Why AI Compute Comes in Rectangles — Erik Bernhardsson, Modal CEO
October 22, 2026
AI Summary
5 min readErik Bernhardsson, CEO of Modal, describes his company as "the easiest way to put it is we build AI infrastructure." Modal provides a compute and storage platform used by companies like Decagon, Suno, Runway, and Cognition to build AI applications—primarily inference, sandboxes, and some training. Bernhardsson started the company in 2021 after leaving his role as CTO at Better.com and following a seven-year stint at Spotify where he built the first version of the music recommendation system, including Discover Weekly. The core insight that led to Modal was simple: "building AI applications and realizing like the infrastructure fucking sucks."
The Product: Functions and Sandboxes
Modal has two main products. Modal Functions, launched in 2021, are serverless functions designed for easy deployment and scaling. They turned out to be excellent for inference and managing GPU capacity, and now represent about two-thirds of Modal's revenue. Modal Sandboxes launched in 2023 after Bernhardsson and co-founder Akshay realized that as LLMs got better at generating code, users would need a place to run that code. The idea was that Modal already had the primitives for managing containers and isolation. Sandboxes grew slowly at first—the team even discussed shutting the product down—but then "it just started exploding" as LLMs improved at generating full applications. Sandboxes n
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
Never miss an episode of Latent Space: The AI Engineer Podcast
Get every new episode summarized in your inbox — free, ~5 minutes to read.
No spam. Unsubscribe anytime.
What you'll learn
- 1 (00:03) **Company Overview & Origin Story** - Erik introduces Modal as an AI infrastructure platform for inference, sandboxes, and training, used by companies like Descript, Suno, and Runway.
- 2 (04:41) **From Side Projects to Stable Diffusion** - Erik describes the pre-ChatGPT genesis of Modal and the first wave of real demand.
- 3 (10:26) **Why Modal Built Its Own Stack** - The rationale for rebuilding infrastructure from scratch rather than using existing tools like Kubernetes.
- 4 (13:21) **The Changing GPU Market: From Spot to 25 Clouds** - How Modal's capacity strategy has evolved from relying on hyperscaler spot instances to a complex procurement operation.
- 5 (15:44) **Rent vs. Buy: The GPU Asset Class Debate** - Erik's nuanced view on whether Modal should own hardware or continue renting.
- 6 (21:19) **Competing with OpenAI & Anthropic for GPUs** - How Modal finds capacity that the frontier labs don't want.
- 7 (25:58) **Modal's Multi-Product Evolution: Functions & Sandboxes** - The two core products and how Sandboxes became a surprise hit.
+ Full timestamped outline available in the app
Guests on this episode
Show Notes
Warm welcome to Basil Chatha, our newest cohost from the Forward Deployed side of the AI Engineering world! We’ll be ramping up a new show with him and couldn’t pass up an interview with Modal’s CEO Eric on the occasion of their monster Series D!
Two years ago, we spoke with Erik Bernhardsson about his vision for truly serverless AI infrastructure: developers should be able to run code without provisioning clusters, predicting load, or paying for idle resources. CTO Akshat then joined us to talk about Modal’s infra primitives and stack to offer those primitives and agent experience.
A lot has changed in the past 5 months. Since their $5B series C, Modal now reports it crossed $1 billion in annualized run rate, raised a $750 million Series D at a $16B valuation, and has spun up nearly three billion sandboxes. In this conversation, Erik joins Vibhu to explain the infrastructure and economics behind that growth—and why demand for compute rarely looks like the contracts used to buy it.
Watch the full conversation on YouTube →
GPU demand isn’t rectangular
A typical capacity contract gives a customer a fixed number of GPUs for a fixed period. Plot capacity against time and you get a rectangle. Real workloads behave differently: they grow, spike, and fall throughout the day. Buy enough for the peak and you pay for idle hardware; buy less and you risk missing demand.
Erik describes Modal’s bet as a kind of duration transformation: pool demand across thousands of customers, secure capacity underneath, and let each customer use what they need when they need it. The smoother the aggregate demand, the more useful that pool can be. It’s a useful companion to our conversation with SF Compute’s Evan Conrad: Evan focuses on making capacity tradeable; Erik focuses on making it usable in smaller, elastic slices.
There is a financial risk on the other side. If a provider locks in capacity at today’s prices and GPU prices later fall, it can be left holding expensive supply. Erik says he doesn’t expect an imminent collapse, but thinks the possibility matters when building the business. Jump to the GPU price discussion →
The sandbox that almost didn’t make it
Modal launched sandboxes after Erik and cofounder Akshat Bubna noticed that language models coul