# Ruyi2 Technical Report

Source: https://www.youtube.com/watch?v=Yb-yfcp79qM
Recap page: https://rapidrecap.app/video/Yb-yfcp79qM
Generated: 2026-02-27T19:30:59.741+00:00

---
## Quick Overview

The Ruyi2 framework successfully addresses the computational and memory overheads of large models like those based on the 1.7 billion parameter edge model by employing a novel architecture that allows for highly efficient system-level optimizations, specifically by using a shared transformer backbone for multiple smaller models and employing a highly specific, multi-stage training process to achieve strong performance metrics while drastically reducing resource consumption.

**Key Points:**
- The Ruyi2 framework reduces the computational and memory footprint of large AI models by employing a shared transformer backbone for multiple smaller models.
- The model achieves a 40% reduction in the parameter count of newly added blocks compared to its predecessor, the 1.7 billion parameter edge model.
- The framework successfully solved the problem of gradient interference during training by using a specific multi-stage process involving block expansion and post-training compression.
- The smaller 1.7 billion parameter edge model (with an early exit mechanism) maintains high performance, achieving an MMLU score of 62.7, while the larger 8.1 billion parameter model achieves 86.52 on the GSM8K benchmark.
- Ruyi2 avoids the catastrophic failure mode of older models where a simple 1 billion parameter model trained on the same data set would forget deep structural knowledge when fine-tuned.
- The architectural innovation allows the model to dynamically manage compute resources based on task difficulty, avoiding unnecessary power draw on simple tasks like summarizing an email.
- The research proves that this efficient architecture allows sophisticated reasoning capabilities (like complex logic and natural language tasks) to run locally on mobile edge devices without constant cloud connectivity.

![Screenshot at 00:10: The Ruyi2 framework uses a shared transformer backbone to efficiently manage multiple models, a key architectural change discussed in the report.](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-00-10.jpg)

**Context:** This briefing analyzes the Ruyi2 technical report from the Institute of Artificial Intelligence, focusing on how researchers tackled the significant deployment costs and latency issues associated with large language models (LLMs) when running on resource-constrained devices. The core problem addressed is the inefficiency of deploying massive models for simple, everyday tasks, which was conventionally handled by training separate, smaller models for different capabilities.

## Detailed Analysis

The Ruyi2 technical report introduces a novel architecture designed to overcome the high deployment costs and latency bottlenecks of large AI models, especially on edge devices. The core innovation is using a single, shared transformer backbone to support multiple smaller models, effectively reducing the need to train entirely separate models for every capability. This framework successfully trained an 8.1 billion parameter main branch and a 1.7 billion parameter edge branch that share parameters. The researchers demonstrated that this architecture achieves high performance—the 1.7B edge model scores 62.7 on MMLU, while the main branch scores 86.52 on GSM8K—while being significantly more efficient than previous methods. A critical technical achievement was preventing the catastrophic forgetting that occurred when fine-tuning smaller models on the same data set as larger ones. They achieved this by implementing a two-stage training process: Stage 1 focused on general behavioral alignment using diverse data, and Stage 2 used a technique called Decompose After Expansion (DEE) to compress the model structure. This process allowed the smaller 1.7B model to maintain deep structural knowledge while achieving high performance on mobile edge devices, demonstrated by its ability to handle complex logic and language tasks locally, avoiding high-latency cloud connections. The overall impact is a fundamental shift in AI deployment efficiency, allowing complex reasoning on devices like smartphones.

### Ruyi2 Framework Overview

- Ruyi2 uses a shared transformer backbone for multiple models
- Employs 3D parallel training and a multi-stage training process
- Effectively balances architectural efficiency with high reasoning capabilities.

### Training Methodology

- Stage 1 focuses heavily on general behavioral alignment using diverse data
- Stage 2 uses DEE (Decompose After Expansion) for compression
- Initialization involves randomizing weights across the 1.7B base model layers.

### Performance Metrics

- 1.7B edge model scores 62.7 on MMLU
- 8.1B main branch scores 86.52 on GSM8K
- Achieves 40% parameter reduction in new blocks compared to previous edge models.

### Key Advantage - Stability

- The structure prevents the catastrophic forgetting seen when training on the same data set, maintaining deep structural knowledge.

### Implications for Edge AI

- Enables sophisticated local processing (like complex voice commands) without constant high-bandwidth cloud connection, preventing battery drain and latency.

![Screenshot at 00:00: Introductory screen showing the podcast/discussion format with the call to action 'Become a Member Today!'](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-00-00.jpg)
![Screenshot at 01:40: Visual representation of the 1.7 billion parameter edge model running on a mobile device, contrasting with massive cloud models.](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-01-40.jpg)
![Screenshot at 06:06: A graphic demonstrating the core idea of layer parallelism, splitting layers across multiple GPUs during training.](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-06-06.jpg)
![Screenshot at 08:35: Mention of the specific technique used to avoid catastrophic forgetting: Proximal Policy Optimization \(PPO\).](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-08-35.jpg)
![Screenshot at 11:16: A slide detailing the results, showing the 1.7B model achieving 62.7 MMLU and the 8.1B model achieving 86.52 GSM8K.](https://ss.rapidrecap.app/screens/Yb-yfcp79qM/00-11-16.jpg)