Numina-Lean-Agent: An Open and General Agentic Reasoning System for Formal Mathematics

Quick Overview

The Numina-Lean-Agent proved highly effective in formal mathematics by achieving a perfect 12/12 score on the notoriously brutal Putnam 2025 competition, showcasing a fundamental shift from pure pattern matching to context-aware, iterative reasoning powered by external tools and self-correction mechanisms.

Key Points: The Numina-Lean-Agent achieved a perfect 12/12 score on the Putnam 2025 competition, which is notoriously difficult for mathematical AI agents. The agent utilized two tools: the Lean proof assistant (LLSP) for formal verification and a custom, result-oriented tool called the NMP (Numina Model Context Protocol) for reasoning. The NMP framework allows the agent to iterate on failed attempts by using feedback loops, changing its approach based on compiler errors or incorrect logic, avoiding the 'tunnel vision' common in other LLMs. The cost of solving the hardest problem (Problem A5) was estimated at $1000, compared to the typical $50,000 cost associated with solving similar problems using brute-force methods. The agent's architecture democratizes advanced mathematical reasoning by effectively teaching the human user the proof structure, contrasting with models that only rely on internal weights. The system succeeded because it separated the difficulty of mathematical reasoning (proof structure) from the difficulty of formal coding (syntax), allowing the human expert (the librarian) to focus on the former.

Context: The video discusses the performance of a novel AI agent named the Numina-Lean-Agent, designed specifically for formal mathematics proof generation, contrasting its approach with standard large language models (LLMs). The context revolves around the agent's success in the Putnam competition, a notoriously challenging test of mathematical problem-solving ability, highlighting a new methodology that integrates symbolic reasoning tools with iterative self-correction.

Detailed Analysis

The presentation details the success of the Numina-Lean-Agent in solving the Putnam 2025 competition, achieving a perfect 12 out of 12 score, a feat considered a massive achievement, especially since the median score for human math majors is often zero. The agent's architecture relies on a loop where it attempts a solution, receives feedback (like compiler errors or incorrect logic from the Lean proof assistant), and then uses that feedback to refine its next attempt. This contrasts sharply with standard LLMs, which often get stuck in iterative refinement loops or suffer from 'tunnel vision.' The agent specifically uses the Lean proof assistant (LLSP) for formal verification, allowing it to check syntax and logical consistency. The key innovation is the NMP (Numina Model Context Protocol), which acts as a universal plug, enabling the AI to interact with external tools, much like a programmer uses debugging tools. This framework separates the difficulty of formal logic/proof structure from the difficulty of coding syntax. The cost for solving the hardest problem (A5) was estimated at $1000, significantly lower than the $50,000 often required for brute-force attempts by other models. The success highlights a shift where the AI acts as a collaborator, providing the structure, while the human expert can focus on verifying the logic, rather than just pattern matching symbols.