Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution

Quick Overview

Xiaomi Robotics Zero introduced an open-source Vision-Language-Action (VLA) model that achieves real-time execution by decoupling cognitive prediction from motor control, resulting in significantly faster and smoother action generation compared to previous models, even when tasked with complex actions like towel folding.

Key Points: Xiaomi Robotics Zero developed an open-source VLA model that separates cognitive prediction (what to do) from motor control (how to do it). The model successfully solved the 'Stop-and-Go' problem by achieving continuous, smooth paths for actions like towel folding, unlike models that rely on discrete point-to-point planning. The model was trained on a massive 200 million time-step dataset, mixing open-source sets (like Droid and MoMa) with proprietary data. When tested on the Towel Folding task, the new model achieved a 98.7% success rate, significantly outperforming baseline models like POPE (Object-level Pose Estimation) and PIO-0. The VLA model demonstrated superior performance by maintaining a 1.0 piece-per-minute baseline for the task, compared to 0.5 pieces per minute for the baseline. The researchers found that the best predictor for the next action was the model's own previous action, suggesting that models often get stuck in repetitive loops if not properly constrained. The new architecture forces the model to ground its predictions in reality by paying attention to the actual current state, preventing it from hallucinating or drifting.

Context: The video discusses a research paper from Xiaomi Robotics Zero introducing a novel open-source Vision-Language-Action (VLA) model designed to improve real-time execution in robotics. The central challenge addressed is the 'Stop-and-Go' problem, where robots hesitate or stutter when planning the next step of a continuous movement. The paper proposes a fundamental architectural change to decouple the high-level cognitive planning from the low-level motor control to ensure smoother, faster execution.