# QED-Nano: Teaching a Tiny Model to Prove Hard Theorems Source: https://www.youtube.com/watch?v=bC69l96k9XI Recap page: https://rapidrecap.app/video/bC69l96k9XI Generated: 2026-02-18T00:05:28.81+00:00 --- ## Quick Overview The QED-Nano model, developed by the LM-Provers team, achieves the ability to prove complex mathematical theorems by using a novel, iterative three-stage pipeline involving distillation, reinforcement learning, and a final step that prunes the reasoning path, allowing it to perform at the level of the much larger Gemini 3 Pro model for proof generation at a significantly lower computational cost ($12 per run vs. potentially hundreds of dollars). **Key Points:** - QED-Nano, a small 685 billion parameter model, matches the performance of the much larger Gemini 3 Pro on complex reasoning tasks like theorem proving. - The model achieves this by employing a three-stage pipeline: distillation from the larger model, reinforcement learning (RL), and iterative refinement/pruning. - The cost to run QED-Nano for a proof is approximately $12, compared to potentially hundreds of dollars for the larger models, representing a 3x cost reduction. - Stage 1 involves distilling the reasoning process from the larger model, and Stage 2 uses Reinforcement Learning (RL) to tighten the feedback loop. - Stage 3 involves the model iteratively checking its own work, discarding proofs that fail validation checks (like arithmetic errors) and refining successful ones, which prevents the model from simply memorizing the training data. - The final output is a logically sound proof that doesn't require an external expert to verify the steps, unlike previous models that might generate plausible but incorrect proofs. ![Screenshot at 00:03: The initial slide highlighting the paper's focus on challenging the dominant narrative in AI development regarding model size versus reasoning capability.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-00-03.jpg) **Context:** The video discusses a research paper by the LM-Provers team from institutions including CMU, Hugging Face, and ETH Zurich, detailing a method to create small, highly capable AI models for complex reasoning, specifically mathematical theorem proving. The dominant narrative in AI development has often been that larger models are inherently better, requiring massive scale. This research challenges that by demonstrating an efficient method to distill the reasoning capabilities of a large model (Gemini 3 Pro) into a much smaller model (QED-Nano) while maintaining high performance. ## Detailed Analysis The paper 'QED-Nano: Teaching a Tiny Model to Prove Hard Theorems' presents a method to achieve high-level mathematical reasoning performance using a significantly smaller model. The prevailing wisdom suggested that complex reasoning required massive models and extensive resources. The LM-Provers team developed QED-Nano, a 685 billion parameter model, to compete with larger models like Gemini 3 Pro. The core innovation lies in a three-stage pipeline. Stage one is distillation, learning from the larger model's steps. Stage two introduces Reinforcement Learning (RL) to tighten the feedback loop, forcing the model to learn the process rather than just memorizing the output. Stage three is an iterative refinement loop where the model checks its own generated proofs against validation checks (like arithmetic) and discards or refines them. This results in a model that produces logically sound proofs that mimic the structure of human-verified math proofs, unlike previous models that might generate fluent but flawed reasoning paths. The cost savings are substantial: running QED-Nano costs about $12 per run, whereas running the larger model for the same task costs hundreds of dollars. The key takeaway is that intelligent process engineering (the iterative self-correction loop) can drastically reduce model size requirements while preserving high-quality reasoning ability. ### Challenging the Narrative - The prevailing wisdom held that complex reasoning required massive scale - QED-Nano demonstrates that efficient engineering can achieve high performance with smaller models - The focus shifts from sheer size to efficient process design. ### The QED-Nano Pipeline - Three stages used—Distillation from a large model (Gemini 3 Pro) - Reinforcement Learning (RL) to tighten feedback loops - Iterative refinement where the model checks and corrects its own proofs. ### Performance & Cost - QED-Nano (685B parameters) matches Gemini 3 Pro performance on proving hard theorems - Cost per run is $12, a massive saving compared to the larger model's cost - The model successfully proves complex theorems without external human verification. ### The Iterative Refinement Loop (Stage 3) - The model drafts a proof, then re-evaluates it against criteria (e.g., arithmetic checks) - It discards incorrect steps and refines the good ones, ensuring data fidelity and logical flow - This avoids the 'brute force' approach of simply throwing more data at a model. ### Conclusion and Impact - The approach democratizes high-end reasoning by making it cheaper and faster to run - It suggests a future where specialized, smaller models can handle complex tasks previously reserved for giants. ![Screenshot at 00:00: The introductory screen featuring the podcast graphic and the 'Become A Member Today!' call to action, setting the stage for the discussion.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-00-00.jpg) ![Screenshot at 00:26: The speakers explicitly mention the paper being discussed: 'QED Nano'.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-00-26.jpg) ![Screenshot at 00:47: The speakers mention the comparison model, 'Gemini 3 Pro', highlighting the scale difference being addressed.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-00-47.jpg) ![Screenshot at 01:19: The speaker explains that the smaller model must perform better than the larger one on specific metrics like the IMO \(International Mathematical Olympiad\) benchmark.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-01-19.jpg) ![Screenshot at 02:24: The speaker identifies the benchmark used for comparison: the 'ProofBench' benchmark.](https://ss.rapidrecap.app/screens/bC69l96k9XI/00-02-24.jpg)