Do Explanations Generalize Across Large Reasoning Models?

Quick Overview

The hypothesis that explanations generalize across large reasoning models fails because while Model A (DeepSeek Coder) showed high accuracy and consistency in generating step-by-step explanations, Model B (DPO) exhibited a lower consistency score (66% vs. 95% baseline) when evaluated by experts, indicating that the quality of the explanation itself does not automatically translate to accurate or logical reasoning, especially when the underlying logic is flawed or the models are trained on different datasets.

Key Points: Model A (DeepSeek Coder) achieved 95% consistency on the baseline, while Model B (DPO) dropped to 66% consistency when evaluated on complex reasoning tasks. The researchers found that the explanatory chain of thought (CoT) generated by Model A was persuasive enough to trick human experts into rating it highly, even when the underlying math was flawed. The core finding shows that reasoning (like mathematical calculation) and explanation generation are separable skills; high explanation quality does not guarantee high reasoning accuracy. Model A's consistently accurate reasoning (e.g., on MedCalc problems) did not transfer when it was forced to use a specific, flawed decoding strategy (sampling instead of greedy). The study suggests that relying solely on the consistency of an explanation as a proxy for truth is risky, as models can become highly consistent in propagating errors or biases. The experiment involved comparing two models, Model A (DeepSeek Coder) and Model B (DPO), using two metrics: accuracy and consistency of the explanation's logic. The researchers propose that future AI systems require explicit training to check each other's work to prevent the reinforcement of errors, rather than relying on the perceived quality of the explanation.

Context: The video discusses a research paper from Northeastern University and Microsoft Research titled "Do Explanations Generalize Across Large Reasoning Models?". The core concept explores whether the reasoning process (the 'chain of thought') generated by one large language model (LLM) can be reliably used by another model to achieve similar, correct results, or if the quality of the explanation implies true underlying reasoning capability. The experiment involved comparing two models, Model A (DeepSeek Coder) and Model B (DPO), on tasks requiring both mathematical reasoning and logical deduction.