Stories

Google's Genie 4 Adds Audio, Realistic Worlds, and Real-Time Steering

Published 27 October, 202610 min read

Google DeepMind released Genie 4 in research preview on October 27, 2026. It is the company's most photorealistic and interactive world model to date, and it ships with three changes that matter: real-time audio generated alongside the visuals, freeform edits you can type in mid-simulation, and a consistency window that doubles from roughly one minute to two. Gemini is now wired in as a director that plans steps and checks frames while Genie renders the scene.

This is still early hardware-constrained research. The team is explicit about what does not work yet, and the model is available only to trusted testers before rolling out to Google AI Ultra subscribers in Project Genie. But the trajectory is clear: world models are moving from making pretty videos to running something you can play.

From 2D to a two-minute world

Google's Genie line has moved fast in a short time. The first Genie, announced in March 2024, generated 2D interactive environments at roughly one frame per second from a starting image. Genie 2, in December 2024, made the leap to 3D at 360p with a consistency window of 10 to 20 seconds. Genie 3, in August 2025, was the first to run in real time, generating interactive 720p video at 24 frames per second with about one minute of memory. It also introduced promptable world events, though the public Project Genie prototype that launched in January 2026 did not ship that capability.

Genie 4 keeps the general shape and pushes each dimension forward. The two-minute consistency window is double its predecessor. Physics look more grounded. Characters read as more human. And the audio channel, which Genie 3 lacked entirely, is now produced as part of the same generation stream.

Version Released Output Consistency Audio
Genie 1 Mar 2024 2D, ~1 fps seconds no
Genie 2 Dec 2024 3D, 360p 10-20 s no
Genie 3 Aug 2025 3D, 720p, 24 fps ~1 minute no
Genie 4 Oct 2026 3D, photoreal up to 2 minutes yes

Google has not published updated resolution and frame rate figures for Genie 4, so the numbers in the table for older versions are the ones DeepMind has confirmed publicly. What matters for developers is not the single-frame fidelity but the shape of the interface: you prompt a world, you enter it, you steer it, and the model keeps producing the next frame regardless of whether you are standing still.

Steering the world from inside it

The headline capability in Genie 4 is freeform editing while a simulation is running. You can change the weather, move the camera, or drop new people and objects into the scene, and the world carries on from the edited state. That is a different interaction model from prompt-and-render video tools like Veo, where each change starts a fresh clip. Here the edit is injected into the loop and the world keeps running.

It is the same conceptual split Runway introduced with its WorldPrompt format two months earlier: a persistent world description on one side, a stream of timestamped actions and camera inputs on the other. The model does not nudge an existing render; it conditions the next step on what the persistence layer says should be true. Google's version of this lives inside a chat-style surface. You type, the world shifts, and you keep moving.

The practical consequence is that Genie 4 behaves more like a world you can negotiate with than a clip you author. That matters for storyboarding and ideation, where the value is in rapid iteration rather than in a single polished frame.

Audio that arrives with the scene

Genie 4 generates real-time audio that matches the simulated world. That includes ambient sound, music, and interactive dialogue with characters in the scene. Conceptually this mirrors what Runway did with GWM Worlds 2, which produces 48,000 Hz audio alongside 720p video and lets speech be one of the actions in its event stream. Audio generation is no longer a separate track that gets layered on top; it is produced in lockstep with the visuals so the sound of a storm arrives when the storm arrives.

For anyone building interactive experiences, that matters more than it sounds. A world model that generates video and stutters on audio is a demo. A world model that generates both in step is something you can put in front of a user. The catch, which Google itself flags, is accuracy. The team has not solved how to guarantee that generated audio matches the intended content, and that gap shows up most clearly with dialogue.

Gemini writes the plan, Genie renders it

The more interesting structural change is how Gemini sits alongside Genie. In Genie 4, Gemini acts as the director: it plans a sequence of steps and inspects rendered frames, while Genie produces the visuals. That division of labor is what Tejas Kulkarni, research director at Google, called the start of hybrid world models during a press briefing.

"What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction," Kulkarni said.

The split is worth taking seriously because it maps onto how people actually build interactive software. A language model is good at decomposition, planning, state tracking, and writing the small bits of logic a simulation needs. A world model is good at rendering consequence in pixels and now in audio. Wiring the two together, one as planner and one as simulator, gives you a loop that can adjust to what it sees.

Testing robots before they exist

The clearest proof point is a research experiment Google ran with Gemini Robotics. A Gemini 4 Argon model broke a task down into individual steps, and Genie 4 simulated each step in real time. The pitch is straightforward: instead of shipping a robot policy to hardware, test it in a generated world first. Score what happens, inspect failures, adjust, and only then move to the physical robot.

This is a research experiment, not a product, and the numbers behind it are not public. But the loop is exactly what the well-known bottleneck for embodied AI has needed. Training data for physical environments is expensive, slow, and often dangerous to collect. A world model that can generate varied, plausible, and steerable environments is a cheap way to expand the curriculum for an agent before it goes near real hardware. Google already has precedent here: Waymo built its Waymo World Model on top of Genie 3 to simulate edge cases for its robotaxis, and the resulting simulator was used as the company expanded its fleet to new cities.

Prototyping film without rendering it

One use that trusted testers surfaced early was not on Google's original bill of materials: prototyping film. Diego Rivas, a Google DeepMind engineer, framed the real-time part as the important bit for creative work, because it preserves a state of flow while someone is iterating.

"The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," Rivas said during a press briefing. "For example, in the ideation phase, in the storyboarding phase, anytime you're kind of in need of rapid ideation."

The distinction from Veo is worth spelling out. Veo generates high-definition clips with synchronized audio from text, image, or video prompts, and it has been gaining traction among filmmakers. Genie 4 is not competing on fidelity with Veo; it is competing on the speed of exploration. You walk through a version of the scene, change the weather, move the camera, then walk again. Once the shot feels right, you hand it to a proper video model to render clean. Rivas said users could eventually rely on Gemini Omni, a multimodal model that generates video from images, audio, and text, to make that hand-off automatic, though he was clear that those capabilities are still a way off.

What still breaks

The Genie team is unusually open about where Genie 4 slips. Details and objects drift the longer a simulation runs. The audio channels have no guarantee of matching what the scene actually needs. And in-world text, which matters more than people expect for signs and interfaces, still is not reliably legible.

The two-minute ceiling is also a constraint worth designing around rather than assuming away. Autoregressive generation on this scale is expensive per second, and Genie 3's earlier Project Genie prototype capped sessions at 60 seconds explicitly to keep costs in range. Doubling that window is real progress, but you still cannot drop a user into a Genie 4 world and walk away for an hour. If you are prototyping around this model, the sensible pattern is short loops with strong starting prompts instead of long continuous sessions.

The crowded field around Genie 4

Genie 4 lands in a world model market that has gotten noticeably busier in the last two months. Runway shipped GWM Worlds 2 in September 2026 with 720p video, 24 frames per second, 48,000 Hz audio, WorldPrompt as its control format, and support for agents and multiple human roles in the same session. Meanwhile the field is consolidating. AMD agreed in September to acquire Fei-Fei Li's World Labs for roughly $8.2 billion in an all-stock deal, Amazon has reportedly been in talks to buy Decart, and Rivian acquired Third Dimension to bring simulation in-house for self-driving testing.

The consolidation signals what the next few quarters will look like. World models are becoming infrastructure rather than a feature of a demo. The teams that win will not just have the best single frame; they will have the best loop, the cleanest control contract, and the clearest path from a generated world to something a product can actually rely on.

Google's position in that race rests on two things it can do that most competitors cannot: pairing a strong world model with a strong reasoning model in the same stack, and pointing the whole thing at its own robotics research. Whether that translates into a product anyone outside the trusted tester list can use depends on how quickly the drift and audio issues get solved, and on how Google prices the compute.

Availability

Genie 4 is available now as a research preview for trusted testers. In the coming weeks, the model will roll out to Google AI Ultra subscribers through Project Genie, Google's tool for generating interactive worlds from text or image prompts, and to early access partners of the Gemini API and Google AI Studio. Anyone outside those channels should treat what they are seeing in demos as a research preview with real limitations, not a finished tool.

For developers watching this space, the useful takeaway is not the resolution figure. It is the shape of the interface. Plan with a reasoning model, simulate with a world model, keep the plan and the world in a tight loop, and design sessions around the length the model can actually hold together. Genie 4 closes some of the gaps in that loop. The next round of releases will show which gaps remain hard.