# Teaching LLMs to Ask: Self-Querying Category-Theoretic Planning for Under-Specified Reasoning Source: https://www.youtube.com/watch?v=Ub_nsa8jTjs Recap page: https://rapidrecap.app/video/Ub_nsa8jTjs Generated: 2026-02-02T16:11:49.45+00:00 --- ## Quick Overview The Stanford team's approach, called Self-Querying Bidirectional Category-Theoretic Planning (SQBCP), solves the problem of LLMs failing to reason under uncertainty by introducing a doubt layer that forces the model to verify its assumptions against real-world constraints, drastically reducing hallucinations and improving plan quality compared to standard Chain-of-Thought planning. **Key Points:** - SQBCP (Self-Querying Bidirectional Category-Theoretic Planning) is a new planning method developed by the Stanford team to address reasoning failures in LLMs. - The method introduces a "doubt layer" that forces the LLM to verify its generated steps against real-world constraints, preventing hallucination. - In experiments, SQBCP reduced the hallucination rate to 16.8% (compared to 83.2% for standard Chain-of-Thought) on a toy car task. - The failure metric used for evaluation was resource violations, and the technique successfully pruned plans that would lead to physical damage or system crashes. - The core idea is to move from LLMs generating plans linearly (like standard Chain-of-Thought) to a bidirectional search that verifies feasibility, similar to how GPS uses distance heuristics. - The paper argues that standard LLMs often struggle because they prioritize fluency over factual correctness, leading to plans that look good but fail in reality. ![Screenshot at 00:06: The core concept of the research is illustrated by the title card referencing the paper: "Teaching LLMs to Ask: Self-Querying Category-Theoretic Planning for Under-Specified Reasoning."](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-00-06.jpg) **Context:** The video discusses research from Stanford University addressing one of the most stubborn problems in AI: the gap between LLMs that generate fluent text and those that can reliably execute complex, real-world plans. The primary issue is that standard Large Language Models (LLMs), trained to prioritize fluency, often generate plans based on unfounded assumptions or hallucinations, leading to failure when these plans are executed. The Stanford team proposes a novel planning framework to solve this lack of reliability. ## Detailed Analysis The Stanford research introduces Self-Querying Bidirectional Category-Theoretic Planning (SQBCP) to fix unreliable planning in LLMs. Standard Chain-of-Thought (CoT) planning often fails because models prioritize generating fluent text over factual correctness, leading to plans that sound plausible but are impossible in reality (e.g., asking an agent to cut a table leg into wheels). The key innovation in SQBCP is the introduction of a doubt layer that functions as a pull-back verifier. When the LLM generates a step, the verifier checks it against known physical constraints or domain knowledge. If a step violates a constraint (e.g., the table leg cannot become wheels), the model is forced to stop, ask for clarification from the user (the oracle), or find an alternative path. This process is analogous to how GPS uses distance heuristics to guide navigation. In testing, SQBCP drastically reduced the violation rate from 83.2% (CoT baseline) to 16.8% on a task involving cutting a table leg, and it improved performance on recipe tasks as well. The method forces the system to be rigorous, ensuring that plans are not only linguistically sound but also physically or logically feasible. ### Problem Identification - The gap between AI fluency and real-world reliability - LLMs often hallucinate plans that look plausible but fail when executed (e.g., cutting a table leg into wheels). ### Proposed Solution - Self-Querying Bidirectional Category-Theoretic Planning (SQBCP) introduces a doubt layer that forces verification against constraints, moving from linear CoT to bidirectional planning. ### SQBCP Mechanism - The system checks pre-conditions for every step, tagging unknowns ('unk') or known violations. If a violation occurs (like the table leg becoming wheels), it triggers an emergency break or asks the user (oracle) for clarification. ### Evaluation Results - SQBCP achieved a 16.8% violation rate on the toy car task, compared to 83.2% for standard CoT, demonstrating a massive improvement in reliability. ### Contrast with Standard Methods - Standard LLMs prioritize fluency, leading to costly errors, while SQBCP prioritizes correctness by verifying feasibility using strict mathematical logic rather than just linguistic probability. ![Screenshot at 00:00: The opening screen featuring the podcast promotion graphic and the channel logo.](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-00-00.jpg) ![Screenshot at 00:23: The speaker explains the shift from simple chatbots generating text to agents executing plans.](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-00-23.jpg) ![Screenshot at 00:54: The speaker introduces the toy car example: asking an AI agent to make a toy car from a wooden table.](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-00-54.jpg) ![Screenshot at 03:02: The acronym SQBCP is defined: Self-Querying Bidirectional Category-Theoretic Planning.](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-03-02.jpg) ![Screenshot at 07:24: A comparison is drawn between the distance heuristic used in GPS and the metric used in SQBCP for rating plan quality.](https://ss.rapidrecap.app/screens/Ub_nsa8jTjs/00-07-24.jpg)