Diffusion LLMs Are Here! Is This the End of Transformers?

Quick Overview

Diffusion LLMs, exemplified by Inception's Mercury 2 model, are demonstrably faster than traditional autoregressive LLMs in inference speed, completing tasks like writing code or generating text significantly quicker by processing tokens in parallel blocks rather than sequentially, although autoregressive models like GPT-4 still hold an edge in complex reasoning benchmarks.

Key Points: Inference speed comparisons show Inception Diffusion LLMs (like Mercury 2) completing tasks (e.g., writing a function) significantly faster than Autoregressive LLMs, taking 14 iterations versus 75 iterations for the same task (00:06:07). Diffusion models generate tokens in parallel blocks, which is inherently faster than the serial, left-to-right generation of autoregressive models, resulting in Mercury 2 being up to 10 times faster in certain inference scenarios (00:12:15). The diffusion approach allows for iterative refinement of the entire sequence simultaneously, overcoming the 'Errors Cascade' problem inherent in sequential generation where early mistakes compound (01:46:18). In a coding benchmark comparing Mercury 2 against leading models, Mercury 2 achieves an Average Quality score of 68.3, outperforming several models like Gemini 2.5 Flash (57) and Claude 3.5 Haiku (62) in this specific metric (11:07). Despite the speed advantage, the benchmark chart shows that models optimized for coding, like Mercury Coder Small, occupy the 'Artificial Analysis' quadrant by achieving high coding index scores while maintaining high output speeds (1008 tokens per second) (11:46). Inception offers three Mercury models (Mercury, Coder, and 2) with varying capabilities and pricing; Mercury 2 costs $0.75 per million output tokens, includes reasoning effort, and supports streaming, tool use, and structured output (12:21). The diffusion architecture enables complex real-time applications, such as rapidly processing a voice query about store hours in real-time, which the model handles instantly (08:51).