# Semantic Search over 9 Million Mathematical Theorems Source: https://www.youtube.com/watch?v=cpMTHF5Jw_o Recap page: https://rapidrecap.app/video/cpMTHF5Jw_o Generated: 2026-02-08T15:03:46.996+00:00 --- ## Quick Overview Researchers from the University of Washington developed a new semantic search method for mathematical theorems that successfully retrieved the correct theorem for a query in 37.8% of cases, significantly outperforming general large language models like GPT-4 and Gemini 1.0, which often failed or hallucinated when searching for specific mathematical concepts within a corpus of 9 million theorems. **Key Points:** - A new semantic search system for mathematical theorems, developed at the University of Washington, achieved a 37.8% success rate in finding the correct theorem for a query. - General LLMs like GPT-4 (with web search) and Gemini 1.0 clustered their results, achieving only 19.8% and 19.0% success rates, respectively, on the same task. - The new method works by translating the user's natural language query into a formal mathematical statement (like a lemma or proposition) and then searching a vector database containing 9 million theorems. - The authors constructed a corpus of 9 million theorems, sourced from arXiv preprints, and explicitly organized the data to facilitate this specialized search. - The system successfully bridges the gap between the natural language intent of the user and the formal structure of mathematical proofs, something traditional keyword search fails to do. - The cost to build this specialized system was approximately $6,000, demonstrating cost-effectiveness compared to training massive general models. - The paper emphasizes that the success metric is not just accuracy, but the quality of the result, showing that the body plus the introduction (the context) provides a much richer result than the theorem statement alone. ![Screenshot at 00:00: The introduction screen featuring the podcast graphic and the call to action 'Become a Member Today!' signals the start of a discussion about a new development in information retrieval.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-00-00.jpg) **Context:** The video discusses a significant development in information retrieval specifically tailored for mathematics, addressing the difficulty of finding specific mathematical theorems within vast digital libraries. The research, conducted by people from the University of Washington, aims to solve the 'search granularity problem' where standard search tools often fail to pinpoint the exact mathematical statement needed, leading to wasted time or incorrect results. The core focus is on improving how AI systems handle queries involving formal mathematical language. ## Detailed Analysis The presentation details a significant advancement in information retrieval for mathematical theorems, stemming from research at the University of Washington. The researchers faced the fundamental friction point in how science gets done: effectively searching vast mathematical literature. They found that existing tools, including Google Scholar and basic arXiv searches, struggle with the specificity required by mathematicians. A study on 9 million pre-print mathematical theorems showed that only 2.5% of retractions happened just because the results were already in the prior literature, suggesting a retrieval issue is often the cause of wasted effort. The new system, which they call 'Sloganization,' translates natural language queries into formal mathematical statements, which are then checked against a vector database containing 9 million theorems. This approach yielded a 37.8% success rate for finding the correct theorem, significantly outperforming large language models like GPT-4 (with web search at 19.8%) and Gemini 1.0 (at 19.0%). The system’s success is attributed to indexing theorems based on their formal structure rather than just keywords, effectively creating a searchable database of mathematical facts. The authors note that for the system to work, the context (the theorem's introduction) is as important as the theorem itself, leading to a richer result. The overall cost for this specialized system was only about $6,000, showing that targeted, smart architecture can outperform large, general models in niche tasks. ### Research Context and Problem - Development of a new semantic search method for mathematical theorems from the University of Washington - Addresses the friction point in scientific research where finding specific theorems is difficult - Traditional search tools like Google Scholar are inadequate for precise mathematical queries ### The New Method (Sloganization) - System translates natural language queries into formal mathematical units (lemmas, propositions) - Uses a vector database containing 9 million theorems sourced from arXiv preprints - This method is robust and avoids the errors seen in brute-force keyword searching ### Performance Comparison - The new tool achieved a 37.8% success rate in finding the correct theorem - GPT-4 with web search scored 19.8% and Gemini 1.0 scored 19.0% - The system significantly outperformed general LLMs, particularly when the query involved mathematical notation (e.g., 3.2 nil radical theorem) ### Key Findings and Architecture - The success relies on indexing based on formal structure rather than just text - The combination of the theorem body and its introduction yielded the best results (56.8%) - The system suggests a powerful pattern for domain-specific AI: translating intent into formal logic and searching a structured knowledge base ![Screenshot at 00:00: The introduction slide features podcast hosts and the call to action 'Become A Member Today!' over a grid background.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-00-00.jpg) ![Screenshot at 00:10: The speaker begins discussing the paper from the University of Washington concerning semantic search over 9 million mathematical theorems.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-00-10.jpg) ![Screenshot at 00:36: The speaker cites a study showing that 2.5% of paper retractions happened due to existing literature, highlighting the need for better retrieval.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-00-36.jpg) ![Screenshot at 01:09: The speaker explains the system works at the document level, taking input and returning a list of PDFs, contrasting it with standard search.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-01-09.jpg) ![Screenshot at 02:28: The speaker summarizes the conceptual shift: treating the theorem itself as the primary retrieval object, rather than just the surrounding text.](https://ss.rapidrecap.app/screens/cpMTHF5Jw_o/00-02-28.jpg)