Token-Level LLM Collaboration via FusionRoute

Quick Overview

FusionRoute achieves the best generalist performance by enabling token-level LLM collaboration through a novel two-function router that performs expert selection and adds a corrective complimentary logit signal, successfully overcoming the instability and inefficiency of prior sequence-level and prior token-level collaboration methods.

Key Points: FusionRoute achieved the highest average performance across all five benchmarks, outperforming sequence selection, Collab, merging, and standard fine-tuned models. The core innovation of FusionRoute is its lightweight router performing a dual function: selection of the best expert and providing a corrective 'complimentary logit' signal added to the expert's prediction. Prior token-level collaboration (like Collab) was unstable because it relied only on pure selection, meaning the router was stuck with the expert's fixed output and could not refine suggestions, leading to 'incoherent fast' responses if an early choice was wrong. The router uses a two-phase training strategy: Phase one involves supervised fine-tuning (SFT) focused only on tokens where experts disagree to efficiently teach selection, and Phase two uses Complemented Direct Preference Optimization (CDPO) on base parameters to learn corrections. The training decouples quality learning (CDPO) from selection to prevent 'degenerate routing,' ensuring the selection layer remains focused on specialization while the base parameters learn quality corrections. FusionRoute maintained high accuracy in specialized domains while dramatically improving general instruction performance, proving the complimentary logic fixes both logical errors and general quality issues like fluency and formatting. The performance gap favoring FusionRoute was larger on Llama 3 8B models than on smaller models, indicating the correction mechanism becomes more essential as expert models become more powerful and rigid.

Context: The discussion centers on solving the "LLM dilemma," which pits the high cost and general capability of massive models against the speed and specialization of smaller, hyper-efficient models that fail outside their narrow domain. The goal is to stitch together these efficient specialists to achieve the power of a giant model without the giant cost, leading to the exploration of the paper "Token-Level LLM Collaboration via FusionRoute."