AI Summary
5 min readWhen Decagon’s co-founders Jesse Zhang and Ashwin Srinivas started building an AI customer service agent, they used frontier models from OpenAI and Anthropic because the goal was simply to get something working. But as they scaled to larger enterprises with millions of customers and launched a voice agent, a new constraint emerged: latency. The only way to make conversations fast and responsive was to use smaller models. And the only way to make those smaller models perform well enough was to fine-tune them. That shift—from consuming frontier models to building their own fine-tuned open-source stack—fundamentally changed how they think about the company, its moat, and its place in the AI landscape.
Why Open Source Models Outperform Frontier Models for Specific Tasks
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
Never miss an episode of a16z Show
Get every new episode summarized in your inbox — free, ~5 minutes to read.
No spam. Unsubscribe anytime.
What you'll learn
- 1 (01:34) **Why Decagon shifted from frontier to open source models** - Jesse Zhang explains the concrete journey that led them to move 90% of their workflow to open source, starting with the need for lower latency in voice agents.
- 2 (04:44) **The false trade-off between cost, intelligence, and latency** - Zhang argues that fine-tuning smaller, “dumber” models can beat frontier models on the specific task while being cheaper and faster.
- 3 (06:57) **Will enterprises eventually fine-tune their own open source models?** - Zhang predicts enterprises will get there, but slowly, due to the difficulty of building custom evals and the inertia of model risk governance.
- 4 (09:10) **Building a “model factory” to keep pace with the changing AI landscape** - Ashwin Srinivas describes how Decagon Labs is structured to rapidly compress the time between a new model’s release and a production-ready fine-tuned version.
- 5 (10:27) **Framework for in-housing vs. buying in the AI stack** - The team explains they build everything tightly coupled to their unique use case internally, but buy commoditized tooling like data labeling.
- 6 (11:57) **Why Decagon doesn’t obsess over tokenomics** - The founders state that cost is a nice side effect, not the primary driver, because their focus is on performance (latency and accuracy) for customer outcomes.
- 7 (14:44) **The false dichotomy: AI app company vs. infrastructure company** - Zhang and Srinivas argue that application-layer companies will survive because they build deep, vertical-specific software and business logic that frontier labs cannot easily replicate.
+ Full timestamped outline available in the app
Show Notes
Sarah Wang and Kimberly Tan are joined by Jesse Zhang and Ashwin Sreenivas, co-founders of Decagon, to discuss the evolution of enterprise AI agents, why the company increasingly relies on open-source models, and how it is helping some of the world’s largest companies deploy AI in production.
Decagon has become one of the fastest-growing AI companies by building agents that automate customer support, sales, and operational workflows. Jesse, Decagon’s CEO, and Ashwin, its president, explain how the company is building enterprise AI at scale.
They unpack why Decagon moved most of its inference to open-source models, how latency, evaluation, and fine-tuning shape production AI systems, and why enterprise AI requires far more than simply plugging into frontier models. The conversation also explores forward-deployed engineering, enterprise sales, AI’s impact on jobs, and why application companies will continue to thrive alongside the foundation model labs.
Resources:
Follow Jesse Zhang on X: https://x.com/thejessezhang
Follow Ashwin Sreenivas on X: https://x.com/AshwinSreenivas
Follow Sarah Wang on X: https://x.com/sarahdingwang
Follow Kimberly Tan on X: https://x.com/kimberlywtan
Stay Updated:
Find a16z on YouTube: YouTube
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
More from this podcast
a16z Show →