Why experts writing AI evals is creating the fastest-growing companies in history | Brendan Foody (CEO of Mercor)
September 18, 2025
AI Summary
5 min readMercor CEO Brendan Foody explains how his company supplies top experts to AI labs for creating evals and reinforcement learning data, enabling model improvements in specific domains. The firm grew from $1 million to $400 million in annual recurring revenue in 16 months by addressing labs' core bottleneck: measuring capabilities where humans outperform models. Discussion covers evals' mechanics, Mercor's pivot and culture, and implications for work.
Evals as the bottleneck for AI labs
AI labs' progress hinges on evals, which define success criteria like rubrics for tasks models struggle with, such as lawyer-style contract redlining. These serve as product requirement documents for experiments, benchmarks for sales, and verifiers in reinforcement learning from human/AI feedback (RLHF/RLAIF), which is more scalable than traditional supervised fine-tuning. Labs need experts—software engineers, bankers, doctors—to build these across domains, as the market is limited by tasks humans can still do better, from tool use to long-horizon planning.
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
What you'll learn
- 1 **[00:05:57] Era of Evals: Why They're Critical for AI Progress**
- 2 **[00:09:26] AI Training Landscape and Mercor's Positioning**
- 3 **[00:12:21] What Experts Do: Creating Evals and RL Data**
- 4 **[00:17:29] Future of Work: New Jobs in Evals, Elastic Demand Skills**
- 5 **[00:39:09] Mercor's Hypergrowth: Tenets and Discovery**
- 6 **[00:46:04] Culture and Operating Principles**
- 7 **[00:56:55] AI Trajectory: Evals Evergreen, No Near-Term AGI**
+ Full timestamped outline available in the app
Show Notes
Brendan Foody is the CEO and co-founder of Mercor, the fastest-growing company in history to go from $1M to $500M in revenue (in just 17 months!). At 22, he is also the youngest American unicorn founder ever. Mercor works with 6 of the Magnificent 7 and all top 5 AI labs to help them hire experts to create evaluations and training data that improve their models. In this conversation, Brendan explains why evals have become the critical bottleneck for AI progress, how he discovered this massive opportunity, and what the future of work might look like in an AI-driven economy.
What you’ll learn:
1. Why evals are becoming the primary bottleneck for AI progress and what this means for AI startups
2. How Mercor grew to $500M revenue in 17 months (fastest in history)
3. Brendan’s meeting with xAI that changed his company’s trajectory
4. Which skills and jobs will remain most valuable as AI continues to advance (hint: jobs with “elastic” demand)
5. Why Brendan believes AGI and superintelligence are not happening anytime soon
6. The three unique core values that drove Mercor’s success
7. How Harvard Lampoon writers are making Claude funnier
—
Brought to you by:
WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs
Jira Product Discovery—Atlassian’s new prioritization and roadmapping tool built for product teams
Enterpret—Transform customer feedback into product growth
—
Transcript: https://www.lennysnewsletter.com/p/experts-writing-ai-evals-brendan-foody
—
My biggest takeaways (for paid newsletter subscribers): https://www.lennysnewsletter.com/i/173303790/my-biggest-takeaways-from-this-conversation
—
Where to find Brendan Foody:
• X: https://x.com/BrendanFoody
• LinkedIn: https://www.linkedin.com/in/brendan-foody-2995ab10b/
—
Where to find Lenny:
• Newsletter: https://www.lennysnewsletter.com
• X: https://twitter.com/lennysan
• LinkedIn: More from this podcast