#494 – Jensen Huang: NVIDIA – The $4 Trillion Company & the AI Revolution

NVIDIA has shifted from GPU-centric design to extreme co-design across chips, systems, racks, pods, and data centers, enabling AI workloads that exceed simple linear scaling. Jensen Huang explains this evolution as essential for distributed computing at massive scale, where problems like sharding models, pipelines, and data trigger Amdahl's law bottlenecks—networking, CPU, memory, power, and cooling all become critical constraints. The company organizes around output: a flat structure with Huang's 60 direct reports as stack-wide experts (GPUs, CPUs, optics, memory), who tackle problems collectively in group sessions, tuning out irrelevant discussions while contributing cross-domain insights.

Extreme Co-Design Process

Extreme co-design optimizes the full stack—software algorithms to hardware, including power-hungry racks—from first principles. Huang's team anticipates AI shifts: Grace Blackwell racks focused on LLMs, Rubin adds Vera CPUs, storage accelerators, and Grok racks for agentic workloads banging on tools like filesystems and databases. This requires reasoning ahead (e.g., agents need I/O, research access), internal model-building, listening to industry labs, and CUDA's flexibility for evolving architectures like Mixture of Experts (MoE), supported by NVLink 72 for trillion-parameter models in one domain. Pods like Rubin (40 racks, 1.2 quadrillion transistors, 60 ex