Google's Genie 4 Adds Sound, Steering, and Longer-Lasting Worlds

Google DeepMind released Genie 4 on October 27, 2026, as a research preview for trusted testers. The fourth generation of its world model generates more photorealistic and interactive environments than its predecessor, adds real-time audio that matches the scene, and lets users type prompts in the middle of a simulation to change the weather, move the camera, or drop a new person into the frame. It also holds a scene together for up to two minutes, double the limit of Genie 3.

The headline number is easy to skim past. Two minutes sounds modest next to a feature film or even a long video game cutscene. But world models are not video generators with a play button bolted on. They predict the next state of an environment from the current state and the user's input, frame after frame, which means every second of consistency has to be earned rather than rendered in one pass. Extending that horizon while adding audio and live editing is the substance of this release.

The release lands in a market that is consolidating quickly. Runway shipped its second world model, GWM Worlds 2, roughly two months earlier. AMD agreed to acquire Fei-Fei Li's World Labs for $8.2 billion in September 2026. Amazon was reported to be in talks to buy Decart for around $7 billion. Rivian acquired Third Dimension in late October to bring simulation in-house for self-driving testing. Google is not introducing a category here; it is trying to stay at the front of one that is already crowded and already being carved up.

What a world model is, and why Genie is not just video

A world model is an AI system that learns an internal representation of how an environment behaves, then uses it to simulate what happens next. DeepMind has framed this as a stepping stone toward general-purpose agents, because an agent that can be trained inside an unlimited curriculum of generated environments does not need to be trained on every real-world scenario one at a time.

Genie is built as a spatiotemporal transformer. It models two things at once: the structure within an individual frame and the way that structure changes between frames. It works in a latent space, an internal mathematical representation where text and image prompts can influence the generated environment. Given an initial prompt and a stream of user inputs, it produces frames that respond to what the user does.

That differs from a text-to-video model in an important way. A video model generates a clip, and the clip plays. A world model generates a state, receives an action, and generates the next state. The action matters. If you turn left in Genie 4, the model has to produce a plausible view of what was to the left, and the world has to remain coherent when you turn back. Runway's CTO has described the gap between video models and world models as the problem of counterfactual generation: an action and its alternative should both produce equally realistic outcomes, not just the outcome that appears most often in training data.

The generational progression shows how much of this is engineering rather than a single breakthrough:

The original Genie generated worlds at one frame per second. Genie 2 stretched sessions to 10 or 20 seconds and added three dimensions. Genie 3 was the first to allow real-time interaction, at 720p and 24 frames per second, with roughly a minute of memory. Genie 4 doubles the memory horizon and adds the two capabilities that were most conspicuously missing: sound and live direction.

DeepMind has not disclosed resolution or frame-rate figures for Genie 4 in the briefing coverage, so the honest comparison stops at duration and capability. What is clear is that the model is being pushed along three axes at once, which is harder than pushing one.

Audio changes what a simulation is for

Genie 4 generates real-time audio that matches the simulated world. That includes music, ambient noise, and interactive dialogue. It is not a soundtrack layered on afterward. The audio is produced alongside the visual state, so a scene of wind and surf should sound like wind and surf, and a character addressed by the user should respond in a way that fits the moment.

This matters more than it might seem from a feature list. Presence in a simulated environment is heavily auditory. A first-person walk through a hurricane without sound is a moving image; with wind, rain, and waves hitting the railing, it becomes something closer to an experience. Dialogue raises the stakes further, because conversation imposes timing. A character who answers two seconds after you speak feels wrong even if the words are right.

DeepMind is candid that this is not solved. The team acknowledged during the briefing that it has not quite figured out how to ensure the audio will be accurate, and that in-world text is not reliably legible. Those are two different failure modes with a shared root: the model is generating content that looks plausible at a glance but does not always survive inspection. Illegible signage is a classic artifact of generated imagery. Audio that drifts from the visual event, or dialogue that says something subtly inconsistent with the scene, is the auditory equivalent.

For anyone planning to use Genie 4 in a production pipeline, that distinction matters. Sound that creates atmosphere is already useful for ideation. Sound that has to be accurate, such as a line of dialogue carrying plot information or a specific mechanical noise tied to a specific object, still needs human review.

Steering a world while it runs

Genie 3 could already accept additional prompts to modify an environment without restarting the simulation. Genie 4 turns that into freeform editing during playback. Users can change the weather, shift the camera angle, or add new people and objects, and the simulation continues.

This is where the comparison with Runway becomes instructive. Runway's GWM Worlds 2 introduced WorldPrompt, a format that fixes certain aspects of an environment, including the first frame, and then specifies a series of timestamped events. It is a control layer for characters, cameras, and the environment, expressed as prompting rather than scripting. Runway's researchers describe movement as reliable and more complex instructions as dependent on difficulty.

Genie 4's approach is looser: type what you want, when you want it, and the world adjusts. Neither approach offers the deterministic state control of a game engine. Neither lets you write a script that guarantees a character will be at a specific position at a specific time. What Genie 4 adds is immediacy, and that is the point DeepMind's engineers keep returning to.

Diego Rivas, a Google DeepMind engineer, argued during the press briefing that real-time interaction is most valuable where creative flow matters. In ideation and storyboarding, he said, the ability to make a change and see it instantly keeps a creator inside the loop rather than waiting on a render. A filmmaker who wants to test whether a scene works at dusk instead of noon can find out in the moment instead of queuing a job.

That is a real workflow change, but it comes with a real constraint. The longer a simulation runs, the more details and objects drift. DeepMind acknowledges this. Drift is a structural property of autoregressive generation: each frame is produced from previous generated frames, so small errors compound over time. Runway's team has identified error accumulation as the central challenge of autoregressive world models. Two minutes of consistency is an achievement precisely because the drift problem gets harder the longer you go.

Gemini as the director

One of the more consequential details in the release is not a Genie feature at all. Gemini, Google's AI model, is more tightly integrated with Genie 4, acting as a director that plans steps and checks frames while Genie produces the visuals. In a research experiment tied to Gemini Robotics, a Gemini 4 Argon model broke tasks into steps that Genie 4 simulated in real time.

Tejas Kulkarni, a research director at Google, described where this leads: hybrid world models that combine Gemini writing code and text with Genie ingesting that output in real time and simulating reality at different levels of abstraction. The division of labor is intuitive once stated. A language model is good at decomposing a goal into an ordered sequence of actions and checking whether an outcome matches the intent. A world model is good at rendering what those actions look like. Put them together and you get something that can plan a task and show you the result.

The robotics experiment is the clearest illustration of why this matters. Testing a robot in the physical world is slow, expensive, and sometimes dangerous. Testing it in simulation is cheap and repeatable, but only if the simulation responds realistically to the robot's actions. If Gemini can specify a task as a sequence of steps and Genie can simulate each step with plausible physics, then a robot policy can be evaluated before anything is built or moved. Waymo has already taken this path: its Waymo World Model, a variant built on Genie 3, simulates edge cases for robotaxis, outputs lidar four times faster than the base model, and accepts dashcam or smartphone video as input. Waymo has credited the approach as part of how it introduced robotaxis to 11 US cities.

Where the early users are pointing

The pitched commercial uses for world models have been stable for a while: gaming, interactive learning, and robotics. Google's own speculative example of interactive learning involves entering a painting of a historical scene and talking to the figures in it about the event depicted. With synchronized audio and interactive dialogue, that example moves from whimsical to technically feasible, though still firmly in the demo stage.

What is more interesting is where trusted testers, the people who get first access to Google's cutting-edge products, are actually finding value. According to the briefing, they are using Genie 4 to prototype films. That is a different use case than gaming. Film prototyping does not require a persistent world that a player returns to; it requires fast iteration on how a shot feels. A director wants to see a location, a lighting condition, a camera move, and hear whether the scene has the right atmosphere, then change it and see again.

Google already has Veo, its AI video generation model that produces high-definition clips with synchronized audio from text, image, or video prompts, and it has become popular among filmmakers. Genie 4 sits upstream of that. It is for exploring, not for final output. Rivas suggested that users could eventually rely on Gemini Omni, a multimodal model that generates video from images, audio, and text, to turn prototypes into finished creative assets, though he said those capabilities are still a ways off.

The limits worth taking seriously

The briefing included an unusual amount of hedging, and it is worth repeating rather than smoothing over. Rivas stressed that it is early days for Genie 4 and that it is not yet clear what the end-to-end product experience will be. The team acknowledged that details and objects drift over longer simulations, that audio accuracy is unresolved, and that in-world text is not reliably legible.

There are also constraints inherited from the economics of the technology. Project Genie, the web interface Google launched in January 2026 to let AI Ultra subscribers use Genie 3, limits world exploration to 60 seconds. The reason given was that Genie is autoregressive and requires substantial dedicated compute, making longer sessions too expensive to scale across many users. DeepMind argued that extending the limit would add relatively little testing value while significantly increasing cost. Genie 4's two-minute horizon will run into the same arithmetic.

Quality perception is another open question. When Project Genie launched, reviewers found it could generate worlds resembling Nintendo games but not Disney characters, and described performance as slow. One reviewer argued that 24 frames per second is too choppy for practical use when modern systems achieve far higher frame rates. Genie 4 is more photorealistic and more steerable, but Google has not claimed a frame-rate leap, and photorealism does not by itself fix motion smoothness.

The market reaction is also part of the story. After Project Genie's release, stock prices of several video game companies declined. World models that can generate playable environments on demand are a direct challenge to parts of the game industry's production model, even if the current output is better suited to prototyping than to shipping.

Availability

Genie 4 is available now as a research preview for trusted testers. In the coming weeks, Google plans to roll it out to Google AI Ultra subscribers through Project Genie and to early access partners of the Gemini API and Google AI Studio. The pattern is familiar: a controlled preview first, a subscriber-facing tool second, developer access after that.

What Genie 4 actually signals

The temptation with a release like this is to score it against a checklist. Audio, check. Longer duration, check. Live editing, check. That misses the more interesting shift. Genie 4 is the point where Google stops treating the world model as a standalone generator and starts treating it as one component in a system, with Gemini planning and checking while Genie renders.

That division of labor is what Kulkarni's phrase about hybrid world models points toward. A language model that can decompose a task and evaluate an outcome, paired with a simulator that can show the consequences of each step, is a more useful machine than either part alone. It is also the version of world models that matters for robotics, where the value is not a pretty frame but a reliable prediction of what happens when a machine moves.

The gaps are real. Drift, audio accuracy, legible text, compute cost, and an undefined product experience all sit between this preview and anything that ships as a finished tool. But the direction is clear, and the combination of sound, live steering, and a two-minute memory horizon moves the technology past the stage where a world model is impressive for existing at all. The next question is whether it can be trusted for long enough, and cheaply enough, to be useful for work that has to be right.