# Do We Always Need Query-Level Workflows? Rethinking Agentic Workflow Gen for Multi-Agent Systems Source: https://www.youtube.com/watch?v=d4SEtNUS0IQ Recap page: https://rapidrecap.app/video/d4SEtNUS0IQ Generated: 2026-01-21T00:07:28.297+00:00 --- ## Quick Overview Researchers propose the SCALE framework, which drastically reduces the token usage for designing multi-agent system workflows—by up to 83%—by replacing expensive, exhaustive testing with a novel surrogate evaluation method combining LLM self-prediction calibrated by a tiny fraction of real-world execution data. **Key Points:** - SCALE claims to cut token usage for agentic workflow generation by up to 83% with a performance degradation of less than 1%, offering a near 'free lunch' in LLM efficiency. - The conventional query-level approach, which designs a custom workflow for every single question, incurs massive inference overhead because it pays to design a new architecture every time. - Task-level workflow search is horrifyingly inefficient in the design phase, chewing through 10 million to 100 million tokens just to find one robust structure. - The researchers found that running the single best static workflow five times achieved the same performance as generating a brand new custom query-level workflow for every query, suggesting adaptability gains are minimal. - Performance of task-level workflows saturates very early in the search process, meaning systems waste millions of tokens searching for marginal improvements (e.g., going from 95.1% to 95.2% accuracy). - SCALE's core innovation is surrogate evaluation: the LLM self-predicts workflow performance, which is then calibrated using a few-shot test on only 1% to 3% of the validation data. - Without the crucial calibration step (the few-shot sanity check), the LLM's self-prediction failed, confirming that the small dose of reality anchors the entire process. **Context:** The discussion centers on the hidden inefficiency plaguing enterprise AI: the massive computational cost associated with designing and optimizing agentic workflows for multi-agent systems, which is often overlooked compared to training costs. The paper analyzed, "Do we always need query level workflows? Rethinking agentic workflow generation for multi-agent systems," challenges the industry's obsession with hyper-customization and brute-force testing by comparing two established paradigms: the generalized, expensive-to-design task-level approach and the expensive-to-run query-level approach. ## Detailed Analysis The video analyzes the economic inefficiency of designing multi-agent system workflows, contrasting the task-level approach (expensive R&D search for one master workflow) with the query-level approach (expensive runtime overhead for custom design per query). Researchers from the Chinese Academy of Sciences proposed the SCALE framework to solve this, claiming an 83% reduction in token usage with negligible performance loss. The analysis revealed that query-level customization offers little benefit over simply running a good generalist task-level workflow multiple times due to LLM stochasticity. Furthermore, the search for task-level workflows suffers from diminishing returns, wasting tokens evaluating near-identical high-scoring candidates. SCALE attacks this by introducing surrogate evaluation: the optimizing LLM reads and self-predicts the performance of a new workflow's code, which is then mathematically corrected ('calibrated') against actual performance on a tiny 1-3% subset of test data. This 'trust but verify' mechanism, where the LLM's intuition is anchored by minimal real-world execution, allows for rapid, cheap workflow discovery. For instance, on the GSM8K math benchmark, SCALE identified a workflow involving a programmer, an ensemble agent, and a dedicated final validation agent, demonstrating that the system learns necessary quality assurance steps without exhaustive failure testing, democratizing the ability to design competitive agent systems. ### Current Workflow Paradigms - Task level focuses on generalization but requires 10M to 100M tokens for design search - Query level offers customization per query but incurs massive inference overhead ### Key Finding on Redundancy - The 'repeat five test' showed that running the single best static workflow five times matched the performance of custom-built query-level workflows, rendering bespoke design largely unnecessary. ### The Problem with Exhaustive Search - Workflow performance saturates early, meaning current methods burn millions of tokens searching for minor rounding error improvements (e.g., 95.1% vs 95.2%). ### SCALE Framework Explained - SCALE stands for Self-prediction of the Optimizer with Calibration using a Few-shot learning for Evaluation - It uses two stages: warm-up and surrogate evaluation. ### Surrogate Evaluation Mechanism - The LLM evaluates generated workflow code via self-prediction, which is then calibrated by running the workflow on only 1-3% of the validation set to create a mathematically corrected surrogate score. ### Performance Metrics - SCALE achieved 54% to 83% reduction in token usage across benchmarks like GSM8K and Hot Pot QA, with an average degradation of only 0.61%. ### Discovered Workflow Structures - For math problems, SCALE evolved a structure including a programmer, an ensemble voter, and a final validation agent - For multi-hop reasoning, it learned the value of a fallback loop for error handling.