# Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematics Source: https://www.youtube.com/watch?v=yx8u0xqR93Q Recap page: https://rapidrecap.app/video/yx8u0xqR93Q Generated: 2026-01-23T02:01:45.662+00:00 --- ## Quick Overview The Numina-Lean-Agent proved highly effective in formal mathematics by achieving a perfect 12/12 score on the notoriously brutal Putnam 2025 competition, showcasing a fundamental shift from pure pattern matching to context-aware, iterative reasoning powered by external tools and self-correction mechanisms. **Key Points:** - The Numina-Lean-Agent achieved a perfect 12/12 score on the Putnam 2025 competition, which is notoriously difficult for mathematical AI agents. - The agent utilized two tools: the Lean proof assistant (LLSP) for formal verification and a custom, result-oriented tool called the NMP (Numina Model Context Protocol) for reasoning. - The NMP framework allows the agent to iterate on failed attempts by using feedback loops, changing its approach based on compiler errors or incorrect logic, avoiding the 'tunnel vision' common in other LLMs. - The cost of solving the hardest problem (Problem A5) was estimated at $1000, compared to the typical $50,000 cost associated with solving similar problems using brute-force methods. - The agent's architecture democratizes advanced mathematical reasoning by effectively teaching the human user the proof structure, contrasting with models that only rely on internal weights. - The system succeeded because it separated the difficulty of mathematical reasoning (proof structure) from the difficulty of formal coding (syntax), allowing the human expert (the librarian) to focus on the former. ![Screenshot at 00:45: The agent successfully proved the Putnam 2025 competition problems, achieving a perfect 12/12 score, which is visually represented by the speakers discussing this major outcome.](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-00-45.jpg) **Context:** The video discusses the performance of a novel AI agent named the Numina-Lean-Agent, designed specifically for formal mathematics proof generation, contrasting its approach with standard large language models (LLMs). The context revolves around the agent's success in the Putnam competition, a notoriously challenging test of mathematical problem-solving ability, highlighting a new methodology that integrates symbolic reasoning tools with iterative self-correction. ## Detailed Analysis The presentation details the success of the Numina-Lean-Agent in solving the Putnam 2025 competition, achieving a perfect 12 out of 12 score, a feat considered a massive achievement, especially since the median score for human math majors is often zero. The agent's architecture relies on a loop where it attempts a solution, receives feedback (like compiler errors or incorrect logic from the Lean proof assistant), and then uses that feedback to refine its next attempt. This contrasts sharply with standard LLMs, which often get stuck in iterative refinement loops or suffer from 'tunnel vision.' The agent specifically uses the Lean proof assistant (LLSP) for formal verification, allowing it to check syntax and logical consistency. The key innovation is the NMP (Numina Model Context Protocol), which acts as a universal plug, enabling the AI to interact with external tools, much like a programmer uses debugging tools. This framework separates the difficulty of formal logic/proof structure from the difficulty of coding syntax. The cost for solving the hardest problem (A5) was estimated at $1000, significantly lower than the $50,000 often required for brute-force attempts by other models. The success highlights a shift where the AI acts as a collaborator, providing the structure, while the human expert can focus on verifying the logic, rather than just pattern matching symbols. ### Putnam 2025 Performance - Agent achieved a perfect 12/12 score - Previous attempts by other models often failed or required human intervention - Cost to solve the hardest problem (A5) was only $1000. ### Agent Architecture and Tools - Agent uses Lean proof assistant (LLSP) for verification - Relies on the NMP (Numina Model Context Protocol) for external tool interaction - NMP acts as a universal plug, allowing interaction with external libraries like Mathlib. ### Reasoning Methodology - Employs an iterative refinement loop based on feedback from the compiler - Agent successfully debugs its own formal code - Focuses on proving the structure/intent rather than just pattern matching symbols. ### Comparison to Other Models - Agent outperforms models relying solely on internal weights (like GPT-4.5.2) which failed to prove the same problems - Avoids the 'tunnel vision' and 'losing the plot' issues seen in pure LLM approaches. ### Implications - Demonstrates that coupling formal systems with iterative feedback creates robust reasoning - Shifts the role of the human from coder to strategic guide/verifier - Success hinges on separating mathematical difficulty from coding difficulty. ![Screenshot at 00:24: The title card introducing the paper "Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematics".](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-00-24.jpg) ![Screenshot at 00:44: A slide highlighting the success of the agent in solving the Putnam 2025 competition, specifically mentioning the 12/12 score.](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-00-44.jpg) ![Screenshot at 01:16: The speaker contrasts the agent's approach with simply training a larger model, emphasizing the importance of the external toolset \(like the Improvver\).](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-01-16.jpg) ![Screenshot at 03:33: Explanation of the agent's architecture, referring to the NMP \(Numina Model Context Protocol\) as a universal plug.](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-03-33.jpg) ![Screenshot at 07:27: The speaker contrasts the agent's method with the previous, unsuccessful approach \(independent sampling\) and notes the human's role shifts from coder to strategist.](https://ss.rapidrecap.app/screens/yx8u0xqR93Q/00-07-27.jpg)