# Dream-VL & Dream-VLA: Open VL and VLA Models with Diffusion Language Model Backbone Source: https://www.youtube.com/watch?v=XpeVIWAFftY Recap page: https://rapidrecap.app/video/XpeVIWAFftY Generated: 2026-01-08T14:42:55.996+00:00 --- ## Quick Overview DreamVL and DreamVLA models introduce a fundamental shift in building AI for robotics by replacing the error-prone auto-regressive backbone with a Diffusion Language Model (DLM) backbone, which naturally facilitates long-horizon planning, bidirectional attention, and parallel generation, leading to state-of-the-art performance, including a 27x speedup in generating action chunks for low-level control. **Key Points:** - The primary bottleneck in complex robotic planning is addressed by replacing the auto-regressive (AR) backbone, which suffers from error accumulation due to its sequential nature, with a diffusion language backbone (DLM). - Diffusion models improve planning by starting with noise and iteratively denoising the entire sequence, enabling global coherence and allowing the model to look both backward and forward via bidirectional attention during refinement. - DreamVL, the vision-language model baseline, performed competitively against top-tier open data AR models like Mammoth HVL on general benchmarks, proving the DLM architecture is a viable VLM foundation. - In high-level planning tasks using the Household benchmark, DreamVL clearly outperformed its AR counterpart, demonstrating that the global reasoning capabilities of diffusion translate to smarter strategic planning. - For low-level planning, DreamVL showed extreme robustness; while AR models cratered when forced to predict 12 steps ahead, DreamVL's performance remained stable and actually improved as the action chunk size grew. - The most dramatic result was a 27x speed up achieved by generating a chunk of 12 actions using only a single diffusion step, drastically improving latency for real-time control. - The fully embodied model, DreamVLA, achieved a new state-of-the-art 97.2% average success rate on the Libero benchmark and a 71.4% success rate on real-world tasks using a Widow X robot. **Context:** The video analyzes the paper introducing DreamVL and DreamVLA, which proposes a radical architectural change for embodied AI and robotics planning by moving away from the standard auto-regressive (AR) paradigm. AR models struggle with long horizon planning because they predict actions sequentially, leading to cascading errors, a problem the researchers solve by adopting a diffusion model backbone, similar to those used in image generation, to construct action sequences. ## Detailed Analysis The core innovation presented is the shift from auto-regressive (AR) models, which suffer from error accumulation because they only look at the immediately preceding step, to models utilizing a Diffusion Language Model (DLM) backbone for action planning. Diffusion models function by starting with noise and iteratively refining the entire sequence, which naturally encourages global coherence and allows for bidirectional attention—the ability to look both backward and forward in the plan simultaneously. This architectural change yields three major advantages: bidirectional attention for mid-task corrections, inheritance of strong text-planning skills, and natural parallel generation or action chunking. In testing, DreamVL showed comparable general VLM performance to top AR models but significantly outperformed them in strategic (high-level) planning tasks, especially those requiring common sense like the Household benchmark. The low-level motor control tests were even more revealing: DreamVL maintained performance as action chunks grew while AR models failed, and achieved an astonishing 27x speedup when generating 12 actions in a single diffusion step. The final robotic model, DreamVLA, trained on nearly a million trajectories, set a new state-of-the-art success rate of 97.2% on the Libero benchmark and achieved 71.4% on real-world tasks with a Widow X robot. The coherence of the diffusion architecture, requiring no architectural hacks for parallel execution, leads to more efficient learning and stable performance, though researchers note future work involves scaling data and potentially moving to continuous action representations. ### Problem with Auto-Regressive Models - Error accumulation due to sequential prediction - Loss of global plot akin to only remembering the last written sentence - Struggle with long horizon planning and global reasoning ### Diffusion Model Solution - Replaces AR backbone with a DLM backbone - Starts with pure noise and refines the sequence step-by-step (denoising) - Naturally encourages global coherence by mapping out the whole route before execution ### Architectural Advantages - Bidirectional attention allows looking backward and forward simultaneously for mid-task corrections - Inheritance of strong sequence planning skills from text tasks - Parallel generation (action chunking) achieved naturally without structural hacks ### DreamVL Vision-Language Performance - Trained on 12 million image-text pairs from open source data - Performed competitively against top open data models like Mammoth HVL - Still showed a performance gap compared to massive closed data models due to scale, not architecture ### Planning Benchmark Results - Clear win for diffusion in high-level planning, especially on the complex Household task - Low-level planning showed robustness as chunk size increased, unlike AR models which degraded immediately ### Speed and Efficiency Gains - Achieved an incredible 27x speed up generating 12 actions at once using only a single diffusion step - This low latency control makes real-time interaction suddenly achievable ### DreamVLA Embodied Robotics - Trained on almost a million manipulation trajectories from OpenX - Achieved 97.2% success rate on the Libero benchmark - Reached 71.4% success rate on a real Widow X robot, significantly beating the previous diffusion best of 54.2%