# Planning with Sketch-Guided Verification for Physics-Aware Video Generation

Source: https://www.youtube.com/watch?v=qR-WU9trBDM
Recap page: https://rapidrecap.app/video/qR-WU9trBDM
Generated: 2025-11-29T18:04:19.139+00:00

---
## Quick Overview

The Sketch-Guided Verification (SketchVerify) framework significantly improves the efficiency and physical realism of physics-aware video generation by using a lightweight proxy to check motion trajectories against physical constraints, achieving a 10x speedup over the costly diffusion-based approach while maintaining high accuracy.

**Key Points:**
- SketchVerify reduces generation time for physics-aware video from potentially an hour to about 5 minutes by decoupling the planning/verification step from the resource-intensive rendering step.
- The method uses a lightweight, low-fidelity video proxy to check if the generated motion (e.g., a ball falling) adheres to basic physics like gravity and momentum.
- The core improvement is decoupling the planning/verification from the high-fidelity diffusion backbone, allowing the generator to focus only on photorealism once the motion is verified.
- In ablation studies, SketchVerify improved physics coherence accuracy by 6% compared to methods relying solely on language verification against the underlying diffusion model.
- The verification process checks for violations of physical laws (like momentum or gravity) and structural integrity in the motion trajectory.
- The framework successfully ensures that generated actions, such as a basketball shot, follow a plausible parabolic arc rather than impossible movements like hovering or instantly changing direction.

![Screenshot at 02:24: The final result of the SketchVerify process, where the low-fidelity proxy video accurately shows the basketball following a plausible parabolic arc, contrasting with the failures of pure language verification.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-02-24.png)

**Context:** This video discusses a novel method called Sketch-Guided Verification (SketchVerify) designed to address the challenges in AI video generation, specifically where large generative models often fail to adhere to fundamental physical laws, resulting in unrealistic or impossible motions. The research proposes a three-step pipeline—planning, verification, and generation—to make the process more efficient and physically consistent, especially for complex interactions.

## Detailed Analysis

The video details the SketchVerify framework, which tackles the problem of physics incoherence in AI video generation. The framework operates in three steps: planning, verification, and generation. Step one involves planning and scene parsing, where the system uses a high-level planning model (like Gemini 2.5) to decompose a complex instruction into smaller, actionable sub-instructions, such as 'dribble forward, then stop, then jump and shoot the ball towards the basket.' The second step, verification, is where SketchVerify excels. Instead of immediately rendering a high-fidelity, expensive video for every step, the system generates a cheap, low-fidelity proxy video that accurately depicts the object's movement over time. A verifier then checks this proxy against physical laws (like gravity and inertia) using structural prompts. If the motion violates physics (e.g., an object floats unnaturally or violates momentum conservation), the trajectory is rejected. This verification significantly reduces the computational load. The third step is generation, where the diffusion model only renders the high-fidelity video once the motion plan has been verified as physically plausible. The researchers found that using SketchVerify improved physics coherence accuracy by 6% over language-only verification methods when tested against benchmarks that include complex physical scenarios like car crashes or water splashes. This iterative refinement process—planning, verifying the proxy, and then rendering—is far more efficient than brute-force attempts, saving significant GPU time.

### Planning and Verification Approach

- Single shot planning
- Iterative refinement
- Decoupling planning from diffusion backbone

### Verification Criteria

- Physical laws (inertia, gravity)
- Semantic alignment (checking movement against text description)
- Consistency of visual properties (size, shape)

### Results and Efficiency

- SketchVerify improved physics coherence accuracy by 6% over language-only verification
- Reduced planning time from potentially an hour to about 5 minutes for complex scenes
- Final video quality remains high, focusing GPU power only on rendering verified paths

![Screenshot at 00:00: The opening visual featuring the podcast-style graphic and the call to action to become a member.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-00-00.png)
![Screenshot at 02:24: Visual demonstration of the successful proxy video, showing the basketball following a plausible parabolic arc after verification.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-02-24.png)
![Screenshot at 04:35: Step 1 of the pipeline: Planning and scene parsing using a complex instruction.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-04-35.png)
![Screenshot at 07:00: Step 3: Verification using structured prompts to check four specific physical laws against the motion plan.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-07-00.png)
![Screenshot at 09:35: Comparison phase showing that the validated plan improves performance over less constrained methods.](https://ss.rapidrecap.app/screens/qR-WU9trBDM/00-09-35.png)