AI company Runway has announced its first general world model, GWM-1. GWM-1 is an autoregressive model built on top of the company’s Gen-4.5 Runway AI model.
What’s a world model?
As the name suggests, a world model is an AI system that creates a representation of a world or environment. It attempts to follow the structure, dynamics and physics of the real world. An AI world models also tries to simulate future events within that environment.
The aim of general world models is to simulate a wide range of situations and interactions, as you might encounter in the real world.
What does GWM-1 do?
Runway’s GWM-1 world model runs in real time and generates video frame by frame. It can also be controlled interactively with actions including camera pose, robot commands, and audio. In addition, there are three variants of GWM-1: GWM Worlds, GWM Robotics and GWM Avatars.
GWM Worlds
GWM Worlds is a world model for real-time environment simulation. You can give the model a static scene, and it generates an immersive, infinite, explorable space in real time. As a result, Runway says you can “travel to any place, real or imagined.” The world model is also consistent, so that if you turn around, what was behind you before is still there.
Plus, Runway says that it can use GWM Worlds as a sandbox to train an AI system to navigate and act in the physical world.
GWM Robotics
GWM Robotics is a learned simulator that generates synthetic data for scalable robot training. In simple terms, this means that you can test AI models designed to control and operate robotics in a simulated environment, before testing in a physical robot. The AI model then creates video rollouts showing how the robots should behave in the real world.
GWM Avatars
GWM Avatars is an audio-driven interactive video generation model. Runway says that it simulates natural human motion and expression for photorealistic or stylized characters. It also renders realistic facial expressions, eye movements, lip-syncing and gestures during both speaking and listening. Additionally, Runways claims it can run for extended conversations “without quality degradation.”
As part of the announcement of GWM-1 on its website, Runway stated, “We believe that world models are at the frontier of progress in artificial intelligence. Language models alone won’t solve the world’s hardest problems – robotics, disease, scientific discovery. Real progress requires models that experience the world and learn from their mistakes, the same way that humans do. And this kind of trial and error can be massively accelerated when done in simulation, rather than in the real world. World models offer the most clear path to general-purpose simulation.”
What we think
Runway’s GWM-1 world model is another significant development in AI. The uses of world models go far beyond creating movies or video games. Runways is claiming that its GWM Robotics AI model can be used to train and develop real world robotic solutions in a virtual environment. The company also suggests that GWM Avatars could generate lifelike characters for AI-based customer support systems.
Runway sees world models as a way to develop its AI in a simulated environment. This may lead to even more rapid advances in AI technology going forwards.
Pricing and availability
You can request access to the GWM-1 Robotics SDK (Software Development Kit) via the Runway website. Runway is also speaking to several robotics companies and other businesses about uses for GWM-Robotics and GWM-Avatars.
