Is RAG Dead? Lessons from Building AI for Tax Law with Alex Bowcut

Sales tax compliance is a domain where getting the right answer isn't enough—you also need to know where it came from. Alex Bowcut, head of engineering at Sphere, builds AI systems for exactly this problem. Sphere helps companies handle sales tax across thousands of U.S. states, cities, and international jurisdictions, each with its own rules that change constantly. The traditional approach requires massive teams of tax lawyers reading legislation manually. Sphere’s system, called T-RAM (Tax Review and Assessment Model), supercharges those experts instead of replacing them. In this conversation, Bowcut explains why retrieval augmented generation (RAG) is far from dead for high-stakes applications, how they combine dense and sparse retrieval, and what they learned from fine-tuning reasoning models with human feedback.

Why RAG isn’t dead for high-accuracy applications

As context windows grow larger, some argue that retrieval is becoming obsolete—just dump everything into the model’s context and let it reason. Bowcut disagrees, at least for his use case. “I think for some use cases it’s certainly true,” he says, but for Sphere’s problem, “because we are so sensitive to accuracy and we’re so sensitive to the exact right citation… I don’t think agents are just searching over the file system, grepping over it, is at a point where we could switch over and not lose accuracy.”