AI researchers debate how close we are to recursive self-improvement

September 11, 2026

AI Summary

5 min read

In the opening of the conversation, three AI researchers—John Schulman, Charlie O'Neill, and Baron—are asked to imagine a future where, by 2036, the world has not been radically transformed by superintelligent AI. The most likely technical reason, they suggest, is that progress could asymptote just before crossing a critical threshold: the point where AI systems begin to dominate human research and development capability. The discussion that follows explores the mechanisms, bottlenecks, and competing visions that will determine whether that threshold is crossed quickly, slowly, or not at all.

The Asymptote Scenario: Why Progress Might Stall

The central worry is not that AI will stop improving, but that it will keep improving in ways that feel impressive without triggering a takeoff. John Schulman describes a recurring cycle: a new model comes out, people are blown away, and then after a month of use it starts to feel dumb. The bottleneck is not raw capability on benchmarks but persistent weaknesses in judgment, self-checking, and generalization to messy real-world tasks. Baron adds that if deep learning simply cannot yield a system that dominates human research—including the human ability to invent new paradigms—then progress could plateau just short of the explosive regime.

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Listen to Audio Summary Open in App

Never miss an episode of Dwarkesh Podcast

Get every new episode summarized in your inbox — free, ~5 minutes to read.

No spam. Unsubscribe anytime.

What you'll learn

1 (00:30) **Why AI Might Not Transform the World by 2036** - The panel opens by considering the most likely technical reasons we might not see superintelligence within a decade, setting the stage for the entire debate.
2 (03:27) **The Scaling Discontinuity Problem** - The panel debates whether continued scaling of current methods can discover the next paradigm shift, or whether a human-level breakthrough is required first.
3 (05:54) **The ELO Score Intuition and the Asymptote Hypothesis** - The conversation uses chess ELO scores as an analogy for why AI's economic impact might suddenly spike.
4 (07:39) **The Difficulty of Discovering New Objectives** - A deep dive into whether AI can learn to set its own research goals, which is crucial for autonomous improvement.
5 (10:44) **The Century of Thinking Thought Experiment** - The panel explores how much AI progress could be accelerated by spending massive compute on analysis before and after experiments.
6 (13:53) **The Last Human Job: Defining the Objective** - The conversation converges on the idea that specifying the objective is the hardest part to automate.
7 (16:40) **Ad Break: Antithesis** - A sponsored segment about software testing tools used by Jane Street.

+ Full timestamped outline available in the app

Guests on this episode

John Schulman Charlie O'Neill

Show Notes

New episode with John Schulman, Beren Millidge and Charlie O’Neill. I got together with some of the most insightful AI researchers I know who are at the openish companies, because I wanted to hear the details of what's actually happening at the frontier and what comes next.

Watch on YouTube; read the transcript.

Sponsors

* Antithesis helps you trust your code. As agents generate more and more of your software, the bottleneck shifts from your engineers actually writing code to verifying it. Antithesis does that testing for you. Ron Minsky, who co-leads Jane Street’s tech group, told me that Antithesis was able to help his team shake out bugs in software that had already undergone heavy review. If you want to see how it fits into your development process, go to antithesis.com/dwarkesh

* Grok Bot has been a great way to hand off tasks. My team uses it as a producer: whenever my editor posts a rough cut of an interview in Slack, Grok Bot opens the transcript on its own computer, matches my notes to the exact moments they refer to, and uses a file of my preferences to suggest edits. Then it sends me its top clip candidates so I can review everything from my phone, which saves my editors from sorting through hours of footage. Try Grok Bot for yourself at x.ai/bot

* Jane Street just launched its most ambitious competition yet: design a protocol-emulator ASIC. Basically, if you have a chip you want to test outside of a live system, you should be able to connect it to your design and have it simulate realistic traffic. Jane Street wants general-purpose, reprogrammable designs that can work across multiple protocols and remain useful as new ones emerge. The most novel submissions will actually get taped out, and the winners will receive a physical copy! The competition is open until January 18, 2027, and teams are encouraged. To get started download the template code at janestreet.com/dwarkesh

Timestamps

(00:00:00) – Steelmanning the case against RSI

(00:18:39) – What’s driving the Chinese labs’ progress

(00:28:06) – How will automated AI researchers be trained

(00:33:51) – Will long-horizon RL elicit AGI?

(00:45:24) – The sim-to-real gap

(01:00:33) – Ho