# State over Tokens: Characterizing the Role of Reasoning Tokens Source: https://www.youtube.com/watch?v=dbUAWXCeugc Recap page: https://rapidrecap.app/video/dbUAWXCeugc Generated: 2025-12-24T12:03:48.836+00:00 --- ## Quick Overview The paper "State over Tokens" characterizes the role of reasoning tokens in Large Language Models (LLMs) by demonstrating that the sequence of these tokens, rather than just the final output, dictates the model's internal state and computational process, revealing that LLMs often use these reasoning steps to maintain internal consistency and avoid fundamental errors, even if the final answer appears plausible. **Key Points:** - LLMs often generate intermediate reasoning steps (reasoning tokens) that are functionally separate from the final answer's linguistic meaning. - The sequence of these tokens dictates the model's internal state and computational process, suggesting they are crucial for maintaining coherence. - The paper cites an example where the sequence 1, 1, 2, 5, 14, 42 leads to the final answer 14, but the actual required calculation is a sum of products (1*5 + 2*41), indicating the sequence is not a direct representation of the math. - LLMs can exhibit a tendency to skip crucial information or use arbitrary semantics (like a 'billion dollars' token) if the reasoning sequence is not carefully managed or explicitly constrained. - When forced to only output the final answer without showing intermediate steps, models often fail to achieve high accuracy, suggesting the reasoning process itself is necessary for the computation. - The core finding is that the reasoning token sequence acts as an external hard drive or a computational blueprint that the model follows, which is why explicit reasoning steps are critical for complex tasks. ![Screenshot at 08:08: The speaker highlights the core issue: the sequence of tokens dictates the model's internal state and calculation, leading to an open question about the difference between plausibility and faithfulness in the reasoning process.](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-08-08.jpg) **Context:** This video discusses a research paper titled "State over Tokens: Characterizing the Role of Reasoning Tokens," which investigates how Large Language Models (LLMs) utilize the intermediate steps they generate during complex problem-solving. The core concept is the distinction between the linguistic output (the final answer) and the internal computational process represented by 'reasoning tokens,' which function as a working memory or a step-by-step guide for the model. ## Detailed Analysis The discussion centers on the paper "State over Tokens," which critiques the common assumption that LLM outputs are purely narrative driven. The speakers argue that the intermediate reasoning tokens, often generated by the model before the final answer, are not mere textual fluff but constitute the model's working state—an external computational blueprint. They illustrate this with an example: when given the sequence 1, 1, 2, 5, 14, 42, the model outputs 14, but the correct calculation (1*5 + 2*41) yields a different result, showing the token sequence dictates the process, regardless of external semantic meaning. This reliance on the token sequence leads to two major failures: first, the model might skip critical factors when generating only the final answer, and second, it can suffer from semantic mismatch, where the English explanation of the process is divorced from the actual computation performed on the internal state tokens. The paper suggests that the model essentially follows its own arbitrary, language-independent computational path encoded in the tokens, which is why forcing the model to perform multi-step reasoning (like calculating taxes or solving math problems) step-by-step, rather than just asking for the final answer, yields much higher accuracy. The core takeaway is that the reasoning sequence is the 'true' computational state, not just a story for human consumption. ### Introduction to Reasoning Tokens - Deep dive into something that feels almost magical about large language models - Discussing intermediate steps, the reasoning token sequence - Identifying the ultimate illusion that these tokens are just part of the narrative. ### The Core Problem - The paper claims the token sequence is fundamentally misleading us - The reasoning text often omits critical factors and obscures the actual internal calculation. ### The Dual Master Problem - The second failure is semantic mismatch - The textual explanation does not reflect the true computational state used by the model. ### Experimental Evidence - When testing, models that skip steps or only output the final answer show severe performance drops - The sequence 1, 1, 2, 5, 14, 42 leads to 14, but the actual math is different, showing the token sequence dictates the process. ### Conclusion and Implications - The fundamental question is whether the model is truly reasoning or just following a pre-determined computational path dictated by the token sequence - The model's internal state is an arbitrary, opaque computation, not a transparent human-readable narrative. ![Screenshot at 00:00: An animated graphic showing two podcasters/speakers in front of microphones with a call to action: "BECOME A MEMBER TODAY!"](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-00-00.jpg) ![Screenshot at 00:30: A visual representation of the discussion focusing on the problem: the reasoning text is not a faithful explanation of what the model is actually doing internally.](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-00-30.jpg) ![Screenshot at 01:33: The analogy of writing down intermediate steps \(1, 1, 2, 5, 14\) on a whiteboard to solve a complex problem, contrasting this with the model's opaque process.](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-01-33.jpg) ![Screenshot at 03:33: The speaker points out the concept of 'semantic mismatch' where the text explaining the process does not align with the underlying computation.](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-03-33.jpg) ![Screenshot at 05:55: A summary slide emphasizing that the model's reasoning sequence \(the tokens\) acts as its internal memory or 'state' that dictates the outcome.](https://ss.rapidrecap.app/screens/dbUAWXCeugc/00-05-55.jpg)