AMAP Agentic Planning Technical Report

Quick Overview

The agentic planning framework successfully demonstrated an 80% improvement in training efficiency and a 16% gain in handling complex, ambiguous tasks compared to baseline models like Q3 and Q4, proving its ability to learn domain-specific reasoning and avoid hallucination by adhering to explicit constraints and rewarding truthful outputs.

Key Points: The agentic planning framework achieved an 80% improvement in training efficiency and a 16% gain in handling complex, ambiguous problems compared to baseline models. The framework uses a three-stage process: explicit reasoning (System 1 thinking), tool selection, and reinforcement learning optimization (System 2). The agent was trained to recognize the boundaries of its own toolset, avoiding hallucinations by refusing to answer questions outside its explicit tool scope. The RLA (RL-Agent) model, initialized from an already capable SFT model, showed superior performance by learning to score complex tasks highly only when they were solved truthfully, unlike general models. The three core dimensions evaluated for difficulty were Cognitive Load/Tool Selection, Execution Chain Depth, and Constraint Complexity. For the EV planning case study, the agent successfully planned a 500km route with charging stops, achieving a final score of 7.33, significantly better than the baseline's inability to solve the task without hallucination.

Context: This technical report details the development and evaluation of an agentic planning framework designed to improve the reliability and efficiency of large language models (LLMs) when tackling real-world, multi-step reasoning tasks. The framework addresses the common failure mode of LLMs, which is hallucination or inventing facts, by enforcing adherence to a defined set of tools and knowledge boundaries during the planning and execution process.

Detailed Analysis

The discussion centers on the AMAP (Agentic Map and Plan) framework developed by the Alibaba AMAP AI team, designed to make agents more reliable and specialized for domain-specific tasks. The framework operates in three main stages: System 1 thinking (intuitive problem assessment), tool selection, and System 2 reinforcement learning optimization. A key innovation is training the agent to recognize the boundaries of its available tools, penalizing it for attempting to answer questions outside its scope (hallucinating) and rewarding it for factual accuracy and constraint adherence. The framework was evaluated across three dimensions of difficulty: Cognitive Load/Tool Selection, Execution Chain Depth, and Constraint Complexity. In a crucial case study involving EV route planning with charging stops over a 500km range, the specialized RLA model achieved a score of 7.33, significantly outperforming general models like Q3 and Q4, which scored -1 (unexecutable). This performance gain is attributed to the agent learning to prioritize truthfulness over just finding an answer, directly linking high fidelity to the final reward signal. The overall takeaway is that specialized, constraint-aware training dramatically improves an agent's ability to handle complex, real-world scenarios reliably.