Qwen 3.5 - The next NEXT model

Quick Overview

The Qwen 3.5 release introduces the Qwen3.5-397B-A17B flagship vision-language model, which is open-sourced, utilizes an innovative hybrid architecture combining Linear Attention and Sparse Mixture-of-Experts (MoE), and demonstrates significant performance leaps in Coding and Multimodal Understanding benchmarks compared to previous Qwen models and competitors like Claude Opus 4.5 and Gemini 3 Pro, while achieving 8.6x to 19.0x inference throughput boost over Qwen-Max due to only activating 17 billion out of 397 billion total parameters per forward pass.

Key Points: The flagship model Qwen3.5-397B-A17B is now open-sourced, featuring 397 billion total parameters with only 17 billion activated per forward pass. The model achieves a significant performance leap in Coding and Multimodal Understanding benchmarks, often surpassing competitors like Claude Opus 4.5 and Gemini 3 Pro on several metrics (00:07, 00:08, 01:28, 01:37). Inference efficiency sees a major boost: Qwen3.5-397B-A17B achieves 8.6x decode throughput at 32K context length and 19.0x at 256K context length compared to Qwen-Max (00:17). Qwen 3.5 employs an innovative hybrid architecture fusing Linear Attention (via Gated Delta Networks) with a Sparse Mixture-of-Experts (MoE) structure (00:12). Multimodal capabilities are enhanced, supporting 201 languages/dialects, a 250K vocabulary, and showing 10-60% coding/decoding efficiency improvement across most languages (00:10, 06:00). The Qwen3-Next architecture is built upon higher-sparsity MoE, Gated DeltaNet, and Gated Attention, leading to performance comparable to the 1T-parameter Qwen3-Max Base model despite having fewer active parameters (00:07, 04:53). The model is accessible via Qwen Chat (chat.qwen.ai) with modes for Auto, Thinking, and Fast processing, and advanced features like reasoning and web search are enabled via parameters in the ModelStudio (08:11, 08:37).

Context: This video discusses the open-source release of the Qwen 3.5 series of large language models, focusing primarily on the flagship Qwen3.5-397B-A17B model, which is positioned as a native vision-language agent. The presentation highlights architectural improvements, significant performance gains across various benchmarks (especially in coding and multimodal tasks), and massive gains in inference efficiency achieved through a novel MoE architecture.