Niantic, the company behind the 2016 location-based game Pokémon Go, sold that product last year and has redirected its focus toward visual-centric AI known as world models. Chief technology officer Brian McClendon of Niantic Spatial describes the effort as a push to reconstruct the physical world using massive image datasets. Unlike large language models that predict text, world models ingest visual inputs and generate three-dimensional environments that obey physical laws.
McClendon envisions robots rehearsing tasks inside these digital twins instead of moving through real spaces. He suggests that a set of photographs of a target location can be turned into a three-dimensional replica, after which the software runs millions of simulated interactions to teach the machine the area’s constraints. “We’re very much about reconstructing the real world,” he said, emphasizing that the approach lets robots practice without physical risk or travel.
Autonomous vehicle developers are applying the same principle. Jamie Shotton, chief scientist at Wayve, notes that a functional world model must anticipate the consequences of an agent’s actions, understand cause-and-effect, and predict how other road users will respond. Wayve’s system draws on dash-camera footage to recreate dangerous intersections and adverse weather, creating virtual test tracks that would be unsafe or impossible to stage on real streets.
Big technology firms continue to tout progress in this arena while venture capital follows the hype. Google is expected to announce incremental upgrades to its own world-model research, and former Meta AI chief Yann LeCun recently secured more than a billion dollars for his Advanced Machines Intelligence venture. In parallel, Stanford professor Fei-Fei Li launched World Labs in 2024; AMD acquired the startup this year for a multi-billion-dollar deal.
The surge in world-model development also reshapes the labor market, as companies require real-world footage to train their systems. The author of the source article illustrated this by attaching a smartphone to his forehead and recording hours of daily chores, later selling the video clips through an app for a modest compensation. As more firms chase richer datasets, gig workers may find increasing demand for similar on-site recording tasks.
Beyond physical robots, the gaming sector is evaluating how world models could enhance realism and interactivity. Companies such as Tencent have announced investments in the technology, aiming to integrate physics-driven 3D generation into future titles. As developers explore these capabilities, the line between simulated environments for training AI and entertainment experiences may become increasingly blurred.