AI Summary
5 min read🎙️ The Voices & The Context
- The Format: Structured interview with host-led questions, technical deep dives, and casual banter.
- The Key Players:
- Matt Bornstein: General partner at Andreessen Horowitz (a16z), enthusiastic host drawing parallels to computing history.
- Woosuk Kwon & Simon Mo: Co-founders of Infrakt, lead contributors to open-source vLLM inference engine; ex-PhD students at UC Berkeley under Ion Stoica (Databricks co-founder, future Infrakt co-founder). Famous for turning a side project into a top GitHub repo powering 400K-500K GPUs worldwide.
- The Vibe: Educational and energizing—nerdy excitement about AI infrastructure, with light humor on open-source chaos and rapid growth.
🗝️ Key Themes & Topics
The episode dives into the "hidden layer" of AI: inference (running models), contrasting it with glamorous training. Core discussion: why open-source vLLM is exploding as the standard for efficient, scalable LLM deployment.
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
What you'll learn
- 1 (00:00) **🎙️ Introduction: Simon Mo & Woosuk Kwon**
- 2 (03:18) **Origins of VLLM**
- 3 (06:24) **Why LLM Inference Differs from Traditional Workloads**
- 4 (10:22) **Simon's Involvement & Early Community**
- 5 (12:51) **VLLM Community Growth & Management**
- 6 (19:53) **What is an Inference Engine?**
- 7 (22:20) **Why Inference is Getting Harder**
+ Full timestamped outline available in the app
Show Notes
Inferact is a new AI infrastructure company founded by the creators and core maintainers of vLLM. Its mission is to build a universal, open-source inference layer that makes large AI models faster, cheaper, and more reliable to run across any hardware, model architecture, or deployment environment. Together, they broke down how modern AI models are actually run in production, why “inference” has quietly become one of the hardest problems in AI infrastructure, and how the open-source project vLLM emerged to solve it. The conversation also looked at why the vLLM team started Inferact and their vision for a universal inference layer that can run any model, on any chip, efficiently.
Follow Matt Bornstein on X: https://twitter.com/BornsteinMatt
Follow Simon Mo on X: https://twitter.com/simon_mo_
Follow Woosuk Kwon on X: https://twitter.com/woosuk_k
Follow vLLM on X: https://twitter.com/vllm_project
Stay Updated:
Find a16z on X
Find a16z on LinkedIn
Listen to the a16z Show on Spotify
Listen to the a16z Show on Apple Podcasts
Follow our host: https://twitter.com/eriktorenberg
Â
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.
Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
More from this podcast
a16z Show →