Dwarkesh and Ilya Sutskever on What Comes After Scaling

🎙️ The Voices & The Context

• The Format: This long-form interview podcast dives into AI's core challenges and future paths through probing questions and thoughtful responses, blending technical depth with philosophical speculation on AGI's trajectory. Technical and reflective.
• The Format: An interview between host Dwarkesh Patel and guest Ilya Sutskever.
• The Key Players:
  • Guest: Ilya Sutskever – Co-founder of Safe Superintelligence Inc. (SSI), pioneering AI researcher behind AlexNet, key contributor to GPT series and OpenAI's early breakthroughs, known for exceptional research intuition.

🗝️ Key Themes & Topics

Deep technical dissection of AI limitations, scaling laws' limits, human-AI learning gaps, and paths to safe superintelligence amid economic diffusion.

• Topic 1: Benchmark vs. Real-World Gap – Models excel on evals like coding competitions but falter in practical tasks (e.g., bug-fixing loops), due to RL overfitting to benchmarks and poor generalization.
• Topic 2: Scaling Eras & Generalization – Shift from pre-training "scaling" (2012-2020 research age, 2020-2025 scaling age) to RL-heavy compute; humans generalize better via efficient priors, emotions as value functions, and continual learning, not massive data.
• Topic 3: Continual Learning & Superintelligence – AGI as a learner like a "super-efficient 15-year-old" deploye