Do AI Tokenomics Matter More Than Model Benchmarks? with Chris Potts

“I used to cost me $20 to take a ride share to the airport, and now it costs $90, but it’s more like $20 to $500,” says Stanford professor Chris Potts, describing the sticker shock hitting AI users as providers begin charging what he calls the “true costs” of inference. This economic reckoning is forcing a question the field has largely avoided: what are our tokens actually buying us? Potts’s recent work argues that answering that question requires looking beyond model benchmarks to measure value in a systematic way — and the early data suggests a troubling trend he calls “tokenflation.”

From Swears to Systems

Potts’s path to AI tokenomics began with linguistics — specifically, a PhD on swearing. “What swears are like, why we swear, what information they encode, what kind of taboos exist around them,” he says. That fascination with linguistic context led him to corpora, then NLP toolkits, and eventually to building language models and founding the startup Big Spin. This background shapes his view of current AI: he sees large language models as the first non-human creatures to use human language fluently, which he calls “the most exciting moment anyone could have dreamed of” for linguistics. But for NLP researchers, the rise of massive pre-trained models created a crisis. “You might wake up one morning to find that you had been completely scooped,” he says — a GPT-3 releas