I've Got Questions with Sinead Bovell
I've Got Questions with Sinead Bovell

What We Are Missing In The AI Safety Debate

October 20, 2026

AI Summary

5 min read

"I was on a call the other day about AI safety, alignment, recursive self-improvement, and I think we're missing something important," Sinead Bovell begins, before zeroing in on a blind spot in the typical conversation. The usual debate pits AI labs racing toward ever-more-capable systems against safety researchers warning about loss of control. But Bovell argues that a powerful, underexplored lever exists in an unlikely alliance: the private sector (companies that would buy AI products) and society at large. If both groups refuse to accept uninsurable, uncontrollable AI, the economic incentive for labs to build dangerously could collapse.

The Alignment Problem and Recursive Self-Improvement

Bovell first lays out the technical landscape. Recursive self-improvement (RSI) is a scenario where an AI system begins to autonomously improve itself, creating continuous feedback loops of rapid capability gains. Humans get pushed out of the loop, losing the ability to understand, steer, or even know what the system is doing. Some researchers believe AI labs are approaching RSI; others disagree. But either way, for labs like OpenAI and Anthropic, RSI is a stated goal—a path to far more efficient and intelligent systems.

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Listen to Audio Summary Open in App

What you'll learn

  • 1 (00:00) **Missing Lever in the AI Safety Debate** - Sinead argues the standard focus on AI-lab-to-AI-lab coordination overlooks a powerful lever: coordination between the private sector (non-AI companies) and society.
  • 2 (03:39) **The Private Sector as the Real Customer** - AI companies are not building these products charitably; they need businesses and consumers to buy them, which creates a critical dependency.
  • 3 (04:43) **An Unlikely Alignment of Interests** - For the first time, the private sector and societal interests are aligned against unsafe AI, creating a powerful, underexplored lever.
  • 4 (05:26) **The Lever: Enterprise Refusal to Buy** - If companies refuse to buy uncontrollable AI products due to liability and risk, the entire funding model for unsafe AI collapses.
  • 5 Standout Quotes
  • 6 "There's an unexplored lever of power if society and the private sector can coordinate."
  • 7 "For the first time, the private sector and societal interests are quite aligned and you don't really think about that. It's not often that the private sector and society can be arm in arm at the picket line."

+ Full timestamped outline available in the app

Guests on this episode

Sinead Bovell

Show Notes

AI safety has moved further into public debate following a series of incidents over the summer involving AI systems behaving in concerning ways. But efforts to influence the direction of AI development often focus on coordination between AI labs or competition between the U.S. and China. There’s another relationship that deserves more attention: the businesses buying and using AI, and the society they operate in. Both have an interest in AI that is reliable and accountable. Could that shared interest become a source of leverage over the companies building it? In this episode, I explore what coordination between businesses and society could look like, and why I think it’s an under-explored lever for AI safety. 0:00 A different way to think about AI safety 0:20 What is recursive self-improvement? 1:35 The AI alignment problem 2:38 The race between AI labs 3:23 AI companies still need customers 4:44 Where business and society’s interests align 5:17 The financial pressure on AI companies 6:01 A lever worth exploring Listen to the show on other platforms: Apple Podcasts – https://podcasts.apple.com/ Spotify – https://open.spotify.com/ Follow my work here: Website: https://www.sineadbovell.com Substack: https://sineadbovell.substack.com Instagram: / sineadbovell LinkedIn: / sineadbovell X: https://x.com/sineadbovell YouTube: / @sineadbovell

I've Got Questions with Sinead Bovell

More from this podcast

I've Got Questions with Sinead Bovell →