Google DeepMind announced Genie 4, which Google AI calls "our most realistic and interactive world model yet", promising "unprecedented realism" and better real-time interactive simulation. It is natively multimodal: music, ambient noises and interactive dialogue are generated simultaneously with the world. You set an event in motion and Genie 4 simulates what happens next: how the world responds, how it sounds, and how the people in it move, react and speak.
Gemini as the brain
Genie 4 incorporates and builds on capabilities from the Gemini family. Gemini Omni anchors the simulations in real-world appearance and behavior, while other Gemini models provide the complex reasoning needed to maintain a simulation's coherence and continuity. Google describes Gemini as the "brain" of the simulation: it translates the user's requests into actions, plans out complex events, watches the scene unfold to keep track of what is happening, and adjusts as the user changes the story. Users navigate and control the camera with standard keyboard controls and steer the scene with text prompts.
What changed
- Real-time steering: natural language can modify a simulation as it runs, such as changing the weather, moving the camera or introducing a new person or object, and Gemini adapts the prompt to the surrounding scene. Between inputs, Gemini also monitors and steers the scene so the world keeps progressing cohesively.
- Realism for physics and people: built on an advanced physical understanding of the world, Genie 4 simulates scenarios such as aerodynamics and collisions. The biggest leap, Google says, is people: characters move naturally, respond organically to their environments and can be steered interactively in real time.
- Sound: Genie 4 is Google's first real-time world model to generate audio natively, with each world carrying the ambient sound of its setting, from wind in the trees to waves at the beach. It also introduces interactive dialogue that adapts on the fly, with characters conversing and reacting to the user's inputs or events in the world. All sounds in Google's demos are produced directly by Genie 4.
Why Google is building it, and access
Google frames understanding and simulating the world as a crucial part of building AGI. Genie 4's "safe, infinitely variable simulations" could help with everything from developing training environments for robots and agents to new interactive experiences. It has started rolling out as a research preview for trusted testers, to gather feedback before release to consumers and developers, starting with Google AI Ultra subscribers in Project Genie and early-access partners of the Gemini API and Google AI Studio. A waitlist is open in Project Genie, which opened to US Ultra subscribers in January with Genie 3.
World models are drawing a crowd: see World Labs' Atlas and, for engineering objects rather than scenes, Skyfall's SKY-001.