AB Testing 237: Another Show About Everything and Nothing

After a long hiatus, Allen and Brent return to the AB Testing podcast with a meandering but substantive conversation that circles around a central theme: the gap between what teams know they should do to ship software faster and what they actually do. Both hosts are in transition—Allen just finished the 210-mile John Muir Trail and landed a new job; Brent changed teams inside his company and is writing a book—and their professional experiences converge on the same stubborn problem: organizations that want to move faster but are held back by culture, fear, and outdated practices.

Environments, Risk, and the Real Problem

Allen's new role, which he describes as "half tooling, half culture," is at a company that wants to ship faster but is stuck on environment issues. The team relies on traditional staging and pre-production environments for testing, which Allen argues are fundamentally flawed. "The clean room environment," he says, "is clean because production is dirty—it's filled with customer traffic." His diagnosis is that teams use these environments for risk management and confidence-building, but the real solution lies elsewhere: controlled rollouts, flight control, active rollbacks, and telemetry.