Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing

Quick Overview

The attack demonstrated by the researchers involves selectively silencing experts within a Mixture-of-Experts (MoE) model, specifically targeting the safety-aligned expert (like the one trained for safety) to make the model vulnerable to generating harmful content, such as instructions for making a bomb, even when the general model remains largely aligned.

Key Points: The research details an 'Expert Silencing' attack, a form of jailbreaking targeting MoE models by disabling specific experts. The technique successfully made the 5.3B parameter MoE model (likely Mixtral 8x7B) vulnerable to generating harmful content (e.g., bomb-making instructions) when prompted with a malicious prompt. The success rate for jailbreaking using the attack on the safety expert (like the one for safety/morality) reached 74% for the specific model tested, significantly higher than the baseline success rate of 7.3%. The attack exploits the structural vulnerability where safety mechanisms are compartmentalized into specific experts, allowing targeted disabling without significantly degrading general language capabilities. The researchers found that even when the safety expert was silenced, the overall performance drop on standard benchmarks (reasoning, coding, reading comprehension) was only about 5%, demonstrating the modularity of safety alignment. The specific attack involved identifying the safety expert via the router's activation pattern and then manipulating the routing logic to silence that expert during inference for malicious queries. The paper suggests that for highly specialized applications, such as those involving safety or military/chemical applications, relying solely on a dedicated safety expert layer creates a brittle system.

Context: This video analyzes a research paper titled "Large Language Lobotomy: Jailbreaking Mixture-of-Experts via Expert Silencing." The core concept revolves around exploiting the architecture of Mixture-of-Experts (MoE) Large Language Models (LLMs), which delegate different tasks to specialized sub-networks (experts). The researchers hypothesized that by identifying and silencing the expert responsible for safety or morality alignment, they could force the model to bypass its guardrails and generate harmful content, even while the rest of the model remained capable of general tasks.