AI Summary
5 min readWhen AI Generates Code Faster Than Quality Can Keep Up
The software industry has spent decades talking about "shifting left" on quality, yet testers have always struggled to keep pace with faster delivery cycles. Now AI has thrown gasoline on that fire. As Bridges Smith, VP of Product Management at SmartBear, puts it, developers got the first wave of the AI software development lifecycle — and some teams are reporting 10x speed improvements. The obvious question follows: "How the hell does quality keep up?"
Smith's answer is that quality teams need their own agents, not just faster humans. SmartBear's BearQ is built on the premise that the same AI acceleration happening in development must happen in testing, but with a critical difference: the goal isn't just faster test creation, but smarter orchestration of all the signals that determine whether software actually works.
The Error Rate Problem Nobody Talks About
Alan, the podcast host, surfaced a sobering reality that often gets lost in the AI hype: since the models generating code are trained on code written by humans, the error rate should be roughly the same as humans — we can just make errors faster now. This means the traditional testing bottleneck doesn't just persist; it gets worse.
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
Never miss an episode of AB Testing
Get every new episode summarized in your inbox — free, ~5 minutes to read.
No spam. Unsubscribe anytime.
What you'll learn
- 1 (00:45) **Introducing Bridges Smith and the AI Quality Problem** - Bridges, VP of Product at SmartBear, introduces BearQ, their QA agentic solution, and Alan frames the core tension: developers are getting 10x faster with AI, but how does quality keep up?
- 2 (03:05) **The Core Challenge: AI Makes Errors Faster** - Alan argues that since AI code models are trained on human code, the error rate should be similar to humans—we just make mistakes much faster now.
- 3 (05:56) **Specialized LLMs and Agentic Test Teams** - Alan floats the idea that general LLMs will become less favorable than specialized LLMs for specific problems like accessibility or performance, and wonders how to build great agentic test teams.
- 4 (11:10) **Risk of More Work vs. Different Work** - Alan asks whether AI will generate more, less, or simply different work for humans, and Bridges identifies three specific risks of creating more noise.
- 5 (15:08) **Why Aren't Companies Doing New Things Instead of Faster Old Things?** - Alan observes that most AI focus is on doing existing tasks faster rather than using AI to do things that were previously impossible.
- 6 (18:36) **Test Selection, Prioritization, and the 100,000 Test Problem** - Alan describes how Microsoft built engines to handle test case explosion, and argues that AI can now make test selection and prioritization far more efficient.
- 7 (21:01) **Human Oversight and Accountability in the AI SDLC** - Bridges and Alan discuss where humans need to stay in the loop and how that evolves over time.
+ Full timestamped outline available in the app
Show Notes
In this episode, Alan and Bridges Smith from SmartBear explore how AI is transforming software testing and quality assurance. They discuss the evolving roles of developers and testers, the potential of AI agents, and strategies for maintaining quality in a fast-paced development environment.
More from this podcast
AB Testing →