InfluxDB: The Evolution of a Time Series Database (with Paul Dix)

July 30, 2025

AI Summary

5 min read

In 2020, Paul Dix decided to rewrite InfluxDB from scratch. He had already built one of the most widely used time series databases in the world, first in Go and then with a custom storage engine optimized for the peculiar shape of time-stamped data. But customers wanted three things the existing architecture could not deliver: support for unbounded cardinality (millions of unique time series), cheap object storage instead of expensive local SSDs, and a proper SQL query engine. Rewriting the storage engine and the query engine at the same time in a new language—Rust—was the kind of bet that can sink a company. Dix walked through the engineering decisions, the painful business lessons, and the surprising realization that solving the problems customers talked about made the database worse at what it already did well.

What makes a database time series

Continue reading the full summary in the app — free to try.

Read Full Summary →

Free • No credit card required

Listen to Audio Summary Open in App

Never miss an episode of Developer Voices

Get every new episode summarized in your inbox — free, ~5 minutes to read.

No spam. Unsubscribe anytime.

What you'll learn

1 (00:00) **Introduction: Why Time Series Databases** - The host introduces the episode, framing time-series data problems and the need for a specialized database.
2 (04:49) **Definition: What Makes a Database "Time Series"?** - Paul defines time series data and distinguishes it from a general-purpose database with a time column.
3 (10:12) **Push vs. Poll: Query Models for Real-Time Data** - Paul explains the trade-off between push-based subscriptions and polling models for time series data.
4 (16:47) **The Data Lifecycle: Hot Sets, RAM, and Object Storage** - The discussion shifts to the core architectural challenge: managing the data lifecycle from fast ingest to cheap archival storage.
5 (19:44) **The Storage Engine: TSM (Time Structured Merge Tree)** - Paul introduces the custom storage engine built for InfluxDB, explaining why standard indexes fail.
6 (23:19) **Deep Dive: How the TSM Tree Works** - A detailed, hands-free explanation of the TSM tree's inner workings.
7 (28:04) **The Inverted Index (TSI) for Fast Tag Queries** - Paul explains how the tag-based metadata is indexed for fast lookups.

+ Full timestamped outline available in the app

Show Notes

How hard is it to write a good database engine? Hard enough that sometimes it takes several versions to get it just right. Paul Dix joins us this week to talk about his journey building InfluxDB, and he's refreshingly frank about what went right, and what went wrong. Sometimes the real database is the knowledge you pick up along the way....

Paul walks us through InfluxDB's evolution from error logging system to time-series datasbase, and from Go to Rust, with unflinching honesty about the major lessons they learnt along the way. We cover technical details like Time-Structure Merge Trees, to business issues like what happens when your database works but your pricing model is broken.

If you're interested in how databases work, this is full of interesting details, and if you're interested in how projects evolve from good idea to functioning business, it's a treat.

Support Developer Voices on Patreon: https://patreon.com/DeveloperVoices

Support Developer Voices on YouTube: https://www.youtube.com/@developervoices/join

InfluxData: https://www.influxdata.com/

InfluxDB: https://www.influxdata.com/products/influxdb/

DataFusion: https://datafusion.apache.org/

DataFusion Episode: https://www.youtube.com/watch?v=8QNNCr8WfDM

Apache Arrow: https://arrow.apache.org/

Apache Parquet: https://parquet.apache.org/

BoltDB: https://github.com/boltdb/bolt

LevelDB: https://github.com/google/leveldb

RocksDB: https://rocksdb.org/

Gorilla: A Fast, Scalable, In-Memory Time Series Database (Facebook paper): https://www.vldb.org/pvldb/vol8/p1816-teller.pdf

Paul on LinkedIn: https://www.linkedin.com/in/pauldix/

Kris on Bluesky: https://bsky.app/profile/krisajenkins.bsky.social

Kris on Mastodon: http://mastodon.social/@krisajenkins

Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/