# DeepMind’s Crazy New AI Masters Games That Don’t Exist Source: https://www.youtube.com/watch?v=-ZFH4oJzCdU Recap page: https://rapidrecap.app/video/-ZFH4oJzCdU Generated: 2025-12-11T08:03:03.864+00:00 --- ## Quick Overview Google DeepMind's SIMA 2 agent demonstrates a significant leap in embodied AI capability over its predecessor, SIMA 1, by successfully understanding, reasoning about, and executing complex, multi-step instructions across diverse and previously unseen 3D virtual environments like No Man's Sky, Wobbly Life, and Valheim, achieving success rates up to 14% in novel scenarios where SIMA 1 failed entirely. **Key Points:** - SIMA 2 significantly outperforms SIMA 1 in previously unseen environments (ASKA and MineDojo), achieving success rates of up to 14% (SIMA 2 in ASKA) compared to SIMA 1's rate of approximately 1-2%. - The new version incorporates multimodal prompting, allowing the agent to follow complex instructions involving visual input (like sketches) and natural language. - SIMA 2 successfully repaired a damaged Pulse Engine in No Man's Sky by correctly interpreting on-screen instructions and navigating the inventory system (00:23). - The agent demonstrates complex reasoning by correctly interpreting a spatial instruction based on a rough sketch in No Man's Sky (00:33). - SIMA 2 exhibits meta-reasoning capability by understanding and executing a 'reverse psychology' trick in Valheim (3:59), where it intentionally performs the opposite of what the user commands. - Unlike SIMA 1, which struggled with new tasks, SIMA 2 successfully navigated and completed novel objectives across multiple distinct game genres, including farming, crafting, and exploration (01:10, 02:00). - The video contrasts SIMA 1's limited, game-specific learning with SIMA 2's generalist approach, which allows it to transfer knowledge between games like Minecraft, Goat Simulator, and others. ![Screenshot at 00:08: The title screen for SIMA 2 appears over a scene from No Man's Sky, defining the agent as 'An agent that plays and reasons in virtual worlds,' setting the context for its expanded capabilities.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-00-08.png) **Context:** This video introduces SIMA 2 (Gemini-based agent that plays and reasons in virtual worlds), the successor to SIMA 1, developed by Google DeepMind. The presentation focuses on comparing the agent's embodied performance in various 3D video game environments—including No Man's Sky, Wobbly Life, Minecraft, Valheim, and others—highlighting SIMA 2's improved ability to interpret natural language, reason about visual context, and execute multi-step tasks compared to the previous version. ## Detailed Analysis The presentation showcases the dramatic improvement of SIMA 2 over SIMA 1 across several embodied AI benchmarks. SIMA 2 is demonstrated to handle complex, multi-step instructions using multimodal inputs, such as interpreting a user-drawn sketch overlaid on the game screen to identify a target object in No Man's Sky (00:33). In tasks requiring inventory management, SIMA 2 successfully accessed the starship inventory to repair the Pulse Engine based on visual cues (00:23). The agent also demonstrated advanced reasoning by correctly interpreting and executing a 'reverse psychology' instruction in Valheim, where it was told to do the opposite of what was commanded (03:59). Performance metrics comparing SIMA 1 and SIMA 2 in previously unseen environments (ASKA and MineDojo) show SIMA 2 achieving success rates around 13-14%, while SIMA 1 barely registered above 1% (04:43). Furthermore, SIMA 2 demonstrates the ability to learn from human instructions in real-time, such as correctly identifying a red mushroom with white spots after failing initially (06:25 vs 06:37), and successfully completing tasks in games it had never encountered during training. The video concludes by contrasting this generalist, fast-learning capability with SIMA 1's inability to perform simple tasks in new environments. ### SIMA 2 Introduction and Capability - SIMA 2 is a Gemini-based agent that plays and reasons across diverse embodied 3D virtual worlds - It handles multimodal prompting, including visual sketches and natural language instructions (00:08) - It shows significant self-improvement capability in unseen environments (01:10). ### No Man's Sky Task Execution - SIMA 2 successfully followed instructions to repair the Pulse Engine by accessing the starship inventory (00:23) - It correctly interpreted a sketch command to find an object on a hill (00:33) - It verbally described its surroundings when asked where it was at night (03:08). ### Multi-Game Generalization - The agent executed tasks in various games, including Wobbly Life (finding the 'tomato house' based on color description at 00:45), Hydroneer (smelting ore at 00:59), and Valheim (following reverse psychology commands at 03:59). ### SIMA 1 vs. SIMA 2 Performance - In previously unseen environments, SIMA 2 achieved success rates around 12-14% while SIMA 1 achieved near-zero success rates (04:43) - SIMA 1 failed to execute simple tasks like finding a campfire (05:04), whereas SIMA 2 provided a structured plan (05:09). ### Learning and Reasoning - SIMA 2 successfully learned to identify flora (naming a poppy at 05:47) and executed complex, multi-step commands like mining Carbon in No Man's Sky (03:34) and mining coal in Minecraft (03:40). ### Limitations and Future - While vastly improved, SIMA 2's success rates (up to 15%) are still below human performance in some tasks (07:01) - The video concludes by showing the infrastructure (Lambda Cloud) that supports training these large models (08:26). ![Screenshot at 00:00: An astronaut character in a green, alien landscape stands near a red and white starship, representing gameplay in No Man's Sky.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-00-00.png) ![Screenshot at 00:13: A scene from a realistic 3D environment showing a character walking towards floating islands with a massive waterfall, labeled 'Genie'.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-00-13.png) ![Screenshot at 00:34: A user prompt overlaying a No Man's Sky scene instructing the AI: 'User: Find the object drawn in the sketch and jump on top of it,' with a rough red sketch overlaying the target structure.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-00-34.png) ![Screenshot at 01:04: A comparison slide showing SIMA 1 \(left\) failing to navigate space debris versus SIMA 2 succeeding in a different environment, contrasting earlier and later model performance.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-01-04.png) ![Screenshot at 02:04: A split screen comparing SIMA 1 \(top\) and SIMA 2 \(bottom\) performance in Minecraft, where SIMA 2 successfully executes the instruction to mine coal.](https://ss.rapidrecap.app/screens/-ZFH4oJzCdU/00-02-04.png)