GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs

Quick Overview

GateBreaker successfully demonstrates that removing the safety-aligned layer (the safety neurons) from Mixture-of-Expert (MoE) Large Language Models (LLMs) causes a dramatic increase in the success rate of jailbreaking attacks, specifically showing that GateBreaker attacks increased success from a baseline of 7.4% to 64.9% against the safety-aligned model.

Key Points: GateBreaker attacks achieved a 64.9% success rate against the safety-aligned MoE LLM, up from a 7.4% baseline success rate against the fully aligned model. The removal of safety neurons, which are responsible for enforcing refusals, made the model significantly more susceptible to jailbreaking. The attack framework involves three stages: profiling, localization (identifying safety neurons), and targeted removal. In ablation studies, removing the safety layer alone resulted in a 55.7 percentage point increase in successful attacks, illustrating its critical role. Removing only the gate projection layer resulted in a significant 55.2% average attack success rate, indicating its importance in directing information flow. The research suggests that safety mechanisms in MoE models are often highly localized, making them vulnerable targets for specific removal techniques.

Context: The video discusses the findings of a research paper titled "GateBreaker: Gate-Guided Attacks on Mixture-of-Expert LLMs," which investigates vulnerabilities in Mixture-of-Expert (MoE) Large Language Models (LLMs) concerning safety alignment. The core concept revolves around attacking the internal structure of MoE models, specifically targeting the components responsible for safety guardrails, like safety neurons or routing layers, to bypass refusal mechanisms.

Detailed Analysis

The GateBreaker research reveals a critical vulnerability in Mixture-of-Expert (MoE) LLMs related to their safety alignment. The attack framework has three stages: profiling, localization, and removal. In profiling, the researchers identified which experts handle safety-related responses. The localization stage focused on identifying the specific neurons responsible for enforcing safety refusals. The attack then targeted these safety components for removal or suppression. When GateBreaker was tested against a safety-aligned MoE model (like the FFN structure), the success rate for harmful prompts increased from a baseline of 7.4% to 64.9% after removing the safety-aligned layer. Further ablation studies showed that removing just the gate projection layer—which directs token flow—resulted in a 55.2% success rate, while removing only the safety neurons caused a 55.7% increase in success. This indicates that safety logic is highly localized, making it a fragile component. The technique successfully bypassed refusal logic without compromising the model's general reasoning or linguistic capabilities, as evidenced by the model remaining functional on standard tasks, highlighting a dangerous decoupling between safety and intelligence.