Runway’s WorldPrompt and the Engineering of Real-Time Worlds
September 25, 2026
AI Summary
5 min readRunway’s WorldPrompt and the Engineering of Real-Time Worlds
When Anastasis Germanidis and his co-founders started Runway in 2018, they were looking at the early generative models of 2016–2017 and extrapolating: if resolution and quality improve predictably over time, there will be a point where most content is generated. The initial thesis was that generative models would force a fundamental rethink of how creative tools are built. Seven years later, that bet has expanded far beyond video editing into world models, real-time generation, and robotics simulation.
From Art Practice to Video Generation Infrastructure
Germanidis came to Runway with an unusual background: he maintained an art practice making interactive installations while working as a backend engineer at startups. One early project from 2015 used Markov chain text generation to simulate small-talk conversations between strangers in a gallery space, assigning each person a randomly generated identity and orchestrating their interactions. This thread of simulation — what can we learn about humans by building simple models of their behavior — ran parallel to his interest in generative models.
Continue reading the full summary in the app — free to try.
Read Full Summary →Free • No credit card required
Never miss an episode of Latent Space: The AI Engineer Podcast
Get every new episode summarized in your inbox — free, ~5 minutes to read.
No spam. Unsubscribe anytime.
What you'll learn
- 1 Runway’s WorldModels and the Engineering of Real-Time Worlds
- 2 (00:03) **Opening & Founding Thesis** - Anastasis introduces Runway's origin: generative models will force a rethink of creative tools
- 3 (03:03) **Early Generative Art Projects** - Pix2Pix experiments on self-driving datasets revealed a key insight about repurposing models
- 4 (05:23) **The Seven-Year Journey** - From open-source model aggregator to building a research org
- 5 (08:12) **The Big Bet on Scaling** - How Runway committed to building a 1000-GPU cluster in 2022
- 6 (09:34) **Gen 1: Video-to-Video Breakthrough** - Starting from depth-conditioned video before tackling text-to-video
- 7 (12:27) **The Stable Diffusion Story** - Anastasis recounts the latent diffusion journey from Runway's perspective
+ Full timestamped outline available in the app
Show Notes
Earlier this month, world model company Runway introduced GWM Worlds 2, a research preview that “turns high-fidelity video and audio generation into real-time interactive simulation.” Runway calls this an “autoregressive diffusion” model; with autoregressive describing how it generates over time.
One new feature in particular caught our eye: WorldPrompt, a proposed input format for specifying a generated world and the actions within it. It allows you to fix some aspects of a simulated environment — including the first frame — and then create a series of timestamped events. The events, or actions, can even be prompted in real-time.
To understand the implications of WorldPrompt, we spoke to Kamil Sindi, Runway’s CTO, and Robin Kahlow, its Principal Research Scientist for generative video and multimodal AI. We also have exclusive comments from Anastasis Germanidis, co-founder & co-CEO of Runway, courtesy of a podcast swyx and Vibhu did with him.
Who’s building real-time interactive world models?
First, some context about world models that can generate interactive video and audio in real-time.
Runway is reportedly valued at $5.3 billion, based on its most recent fund raise of $315 million in February. Its first release, GWM Worlds, was launched last December.
Alongside Runway, there are several other notable projects in this domain: Google DeepMind’s Genie 3 (which also generates at 720p and 24 fps), Odyssey-2 Pro, and World Labs’ RTFM (Real-Time Frame Model). We’ve summarized their differences in the following table:
Given the complexity and massive latency demands of real-time video and audio generation (which we’ll get into below), all of the projects listed above have limitations. For instance, Google notes that Genie 3 “can currently support a few minutes of continuous interaction, rather than extended hours.”
But as our interviews with Runway show, real progress is being made.
The central idea of WorldPrompt
WorldPrompt, a new feature in GWM Worlds 2, helps diffe