Episode 239: More AI Testing

September 27, 2026

AI Summary

5 min read

The hosts of AB Testing return from a brief hiatus to wade into the chaos of AI adoption in software delivery, and the conversation quickly reveals a familiar pattern: teams are moving fast, shipping more bugs, and struggling to figure out what their jobs even mean anymore. Allen opens with a weary account of being pulled into endless status meetings where he could have solved the problem faster than it took to explain why he would solve it. Brent, newly onboarded somewhere he won't name, is taking the "thirty days to listen" approach, though he suspects it will take sixty. The personal frustrations are a microcosm of the larger dysfunction they want to talk about: the industry is treating AI like a magic wand, and nobody has agreed on the design patterns yet.

The confidence problem and Jason Arbon's provocation

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Listen to Audio Summary Open in App

Never miss an episode of AB Testing

Get every new episode summarized in your inbox — free, ~5 minutes to read.

No spam. Unsubscribe anytime.

What you'll learn

1 (00:20) **Intro and Catching Up** - The hosts banter about a long week of status meetings and organizational dynamics.
2 (09:20) **Community Housekeeping & Infrastructure** - Allen discusses the state of the podcast's backend systems and community channels.
3 (14:03) **Introducing the AI & Testing Discussion** - The hosts pivot to the main topic: AI's impact on testing, prompted by Jason Arbin's concept of "confidence engineering."
4 (17:18) **The MTP (Modern Testing Principles) Legacy** - Brent reflects on how the Modern Testing Principles, written nearly ten years ago, still hold relevance today.
5 (19:10) **Debating "Confidence Engineering"** - The hosts dissect Jason Arbin's proposal that testers should rebrand as "confidence engineers."
6 (24:37) **The Agile Manifesto Parallel** - Allen and Brent discuss how the Agile Manifesto was widely misinterpreted, drawing a parallel to today's AI hype.
7 (28:01) **The Non-Deterministic Nature of Agents** - Brent explains why autonomous agents are fundamentally different from deterministic software.

+ Full timestamped outline available in the app

Show Notes

In this episode, Alan and Brent explore the evolving landscape of AI, confidence engineering, and the future of software testing. They discuss how AI impacts decision-making, testing practices, and the importance of clear principles in a rapidly changing tech environment.