AI Summary
5 min readDemis Hassabis, the co-founder of DeepMind and recent Nobel laureate in Chemistry, has been thinking about artificial general intelligence since he was a teenager. In a conversation at Y Combinator, he argued that the current AI paradigm—large-scale pre-training, reinforcement learning from human feedback, and chain-of-thought reasoning—has proven itself too thoroughly to be a dead end. Yet he estimates there is a fifty-fifty chance that one or two fundamental ideas are still missing before we reach AGI. The missing pieces, he said, are continual learning, long-term reasoning, and more sophisticated memory systems.
The missing ingredients: memory, continual learning, and reasoning
Hassabis explained that current models rely on a "brute force" approach to memory, shoving everything into a massive context window. A million-token context window sounds enormous, but he noted that if you are processing live video, it only covers about twenty minutes. The brain, by contrast, integrates new knowledge gracefully during sleep, replaying important episodes to consolidate them. DeepMind’s earliest Atari program, DQN, borrowed this idea with "experience replay"—replaying successful trajectories many times. Today, he said, we are using "duct tape" to approximate this, and there is substantial room for innovation in how models store, retrieve, and prioritize information.
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
Never miss an episode of Y Combinator Startup Podcast
Get every new episode summarized in your inbox — free, ~5 minutes to read.
No spam. Unsubscribe anytime.
What you'll learn
- 1 (00:00) **What’s Still Missing for AGI** - Demis identifies the unsolved components needed for true general intelligence.
- 2 (03:36) **The Duct-Tape Approach to Memory** - The current brute-force use of context windows is unsatisfying and inefficient.
- 3 (06:12) **Reinforcement Learning Is Still Underrated** - DeepMind’s agent-focused history shows RL and search are making a comeback.
- 4 (08:12) **Distillation and the Power of Small Models** - Smaller, faster models are catching up to frontier performance at a fraction of the cost.
- 5 (10:42) **The Thousand-X Productivity Shift** - Engineers are already seeing massive multipliers in their work output.
- 6 (12:20) **Why Agents Need Continual Learning** - The missing piece for fire-and-forget agents is adapting to context over time.
- 7 (13:27) **Where Reasoning Still Breaks** - Models overthink and make basic errors that reveal gaps in introspection.
+ Full timestamped outline available in the app
Guests on this episode
Show Notes
Demis Hassabis has had one of the most extraordinary careers in tech. He started as a chess prodigy and video game designer at 17 before getting a PhD in neuroscience and going on to found DeepMind. His lab cracked Go, solved protein structure prediction with AlphaFold, and then gave it away free to every scientist on earth. That work won him the 2024 Nobel Prize in Chemistry. Today he leads Google DeepMind, pushing toward the same goal he set as a teenager: AGI. On this special live episode of How to Build the Future, he sat down with YC's Garry Tan to talk about what still needs to happen to get us to AGI, his advice for founders on how to stay ahead of the curve and what the next big scientific breakthroughs might be. Chapters:00:00 — Intro00:46 — Demis Hassabis: From Chess Prodigy to DeepMind01:48 — What’s Missing Before We Get To AGI?03:36 — Why Memory Is Still Unsolved06:14 — How AlphaGo Shaped Gemini08:06 — Why Smaller Models Are Getting So Powerful10:46 — The 1000x Engineer12:40 — Continual Learning and the Future of Agents13:32 — Why AI Still Fails at Basic Reasoning15:33 — Are Agents Overhyped or Just Getting Started?18:31 — Can AI Become Truly Creative?20:26 — Open Models, Gemma, and Local AI22:26 — Why Gemini Was Built Multimodal24:08 — What Happens When Inference Gets Cheap?25:24 — From AlphaFold to the Virtual Cells28:24 — AI as the Ultimate Tool for Science30:43 — Advice for Founders33:30 — The AlphaFold Breakthrough Pattern35:20 — Can AI Make Real Scientific Discoveries?37:59 — What to Build Before AGI ArrivesApply to Y Combinator: https://www.ycombinator.com/applyWork at a startup: https://www.ycombinator.com/jobs
More from this podcast
Y Combinator Startup Podcast →