Training Data
Training Data

Google's AI Infrastructure Chief, Amin Vahdat, on the Physics & Economics of Frontier AI

October 6, 2026

AI Summary

5 min read

The Physics and Economics of Frontier AI Infrastructure

Amin Vahdat, Google's head of AI infrastructure, oversees what he calls "the biggest CapEx build-out in human history"—Google alone is expected to spend more than $200 billion on capital expenditures this year, mostly on data centers. But the conversation quickly moves past scale to the harder question: how do you build systems that actually deliver useful work, not just theoretical peak performance?

Goodput vs. Throughput: The Real Measure of AI Infrastructure

Vahdat argues that the industry's standard metric—floating point operations per second, or "flops"—is a vanity metric. What matters is goodput: the fraction of theoretical performance that a workload actually delivers under real-world conditions. At 100,000-accelerator scale, something fails multiple times per hour. Each failure can halt a synchronous workload (training, serving, or agentic tasks) because thousands of components coordinate at microsecond granularity. When one fails, the system must detect it, find a checkpoint, restore it, and restart—all of which counts as throughput (work done) but not goodput (progress toward the answer). "If you have to go back to step one because you've made a mistake," Vahdat says, "you're still doing work. That's throughput. But what is the goodput in delivering your answer?"

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Listen to Audio Summary Open in App

What you'll learn

  • 1 (00:11) **The Core Trade-Off: Specialization vs. Flexibility** - The fundamental tension in hardware design is introduced: specialized chips are faster and more efficient, but less flexible, and must be designed for durable workloads.
  • 2 (01:50) **What Makes an AI Data Center Different** - A crash course on how AI data centers diverge from traditional ones, focusing on co-design and power density.
  • 3 (05:41) **The Real Metric: Goodput, Not FLOPS** - The host introduces "goodput" as the key accountability metric, which measures delivered performance on real workloads, accounting for failures and overhead.
  • 4 (10:54) **The Long Tail of Failures** - The challenge of reliability at scale is explored, revealing that failures are a diverse and constantly evolving problem.
  • 5 (12:05) **Reference Stacks vs. Specialization** - The discussion covers how much of the system design is provided by accelerator vendors versus done in-house by the operator.
  • 6 (13:04) **The Insane Pace of Capacity Doubling** - The host reveals the directive to double serving capacity (token generation) every six months, driven primarily by software and model improvements.
  • 7 (15:36) **The TPU Program: From Contrarian Bet to Co-Design Engine** - The history of Google's custom silicon program is reviewed, highlighting its evolution and the pivotal moment of the Transformer architecture.

+ Full timestamped outline available in the app

Show Notes

At 100,000-accelerator scale, something fails multiple times an hour, which is why Google's Chief Technologist for AI Infrastructure Amin Vahdat thinks FLOPS is a vanity metric. The metric that matters is what he calls goodput: the useful work a workload actually delivers through real failures. Amin walks through the calculus that split the TPU line into 8i and 8t for the first time, why the TPU's core primitives haven't changed since v1, and how Google and DeepMind co-design in the same rooms, intercepting chip architectures mid-flight before tape-out. He explains why long-horizon agents are sending demand for CPUs and storage through the roof alongside accelerators, how optical circuit switches reroute light to a spare rack in milliseconds, and why Google would rather wait on a utility than build its own gigawatt. We also cover orbital data centers and the multi-megawatt rack of 2036.


Hosted by Sonya Huang, Sequoia Capital



0:00 – Introduction

1:47 – What makes a data center an AI data center

5:30 – Goodput, not FLOPS: holding yourself accountable when something fails every hour

11:52 – Doubling token capacity every six months, and where the gains actually come from

15:32 – The TPU bet: from a contrarian call in 2013 to splitting 8i and 8t

23:30 – The case for and against co-design

26:11 – Shoulder to shoulder with DeepMind: intercepting chips mid-flight

34:16 – Long-horizon agents change the shape of the data center

37:50 – Optical circuit switching and the state of networking

42:35 – Power is the binding constraint: utilities, gigawatts, and sizing a data center

49:23 – Training vs. serving clusters, seven-year-old TPUs, and open standards

58:29 – Orbital data centers and the supercomputer of 2036


Training Data

More from this podcast

Training Data →