Lenny's Podcast: Product | Career | Growth

An inside look at X’s Community Notes | Keith Coleman (VP of Product) and Jay Baxter (ML Lead)

February 27, 2025

AI Summary

5 min read

Community Notes enables X users to add context to potentially misleading posts through a crowdsourced system. Contributors with verified phone numbers propose notes, which others rate for helpfulness. Notes appear publicly only after sufficient cross-group agreement, emphasizing neutral, informative additions over strict fact-checking.

Core Mechanism
The algorithm relies on "bridging" agreement: it prioritizes notes rated helpful by users who have historically disagreed on other topics, using matrix factorization to model rater behavior (threshold around 0.4 on an internal scale). This counters bias and manipulation better than majority vote or PageRank variants, as confirmed by an internal ML bake-off. Only about 8% of proposed notes show, filtered conservatively for quality; bad notes risk revoking writing privileges. Every post qualifies, from Elon's to ads, with media/URL matching for scale—one note can apply to thousands of posts.

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Open in App

What you'll learn

1 `* (00:05:26) **Community Notes Definition and Core Mechanism**`
2 `* (00:09:58) **Eligibility, Contributors, and Examples**`
3 `* (00:13:33) **Scale, Impact, and Key Features**`
4 `* (00:21:31) **Handling Polarization and Proven Effectiveness**`
5 `* (00:30:19) **Origin Story and Thermal Team Setup**`
6 `* (00:46:22) **Algorithm Development and Team Operations**`
7 `* (01:12:08) **Core Principles and Survival Through Changes**`

+ Full timestamped outline available in the app

Show Notes

Keith Coleman (VP of product) and Jay Baxter (founding ML engineer), the minds behind Community Notes, reveal how a small, scrappy team inside Twitter/X built the most trusted crowdsourced information system on the internet—one that’s changing the way we understand truth online. What you’ll learn:

1. How Community Notes actually works—a deep dive into the groundbreaking algorithm that rewards “bridging agreement” instead of majority rule

2. The seemingly crazy yet brilliant way this idea survived multiple CEO changes—from Jack to Parag to Elon

3. How this project started with a dumpster fire GIF (literally)—the untold backstory of its early launch

4. The secret to running ultra-fast, high-impact product teams—no OKRs, no Jira; just one Google Doc

5. What Meta’s adoption of Community Notes means for the future of online (mis)information—why this open source system is becoming the industry standard

—

Brought to you by:

• WorkOS—Modern identity platform for B2B SaaS, free up to 1 million MAUs

• Productboard—Make products that matter

• Wix Studio—The web creation platform built for agencies

—

Find the transcript at: https://www.lennysnewsletter.com/p/how-x-built-the-best-fact-checking-system-on-the-internet

—

Where to find Keith Coleman:

• X: https://x.com/kcoleman

• LinkedIn: https://www.linkedin.com/in/keith-coleman-19b12b46/

—