# AndesVL Technical Report: An Efficient Mobile-side Multimodal Large Language Model

Source: https://www.youtube.com/watch?v=V2vCwlOzhdE
Recap page: https://rapidrecap.app/video/V2vCwlOzhdE
Generated: 2026-01-19T18:03:37.202+00:00

---
## Quick Overview

The AndesVL multimodal large language model achieves efficiency for mobile deployment by aggressively quantizing its base model from 16-bit to 4-bit precision, resulting in a 3x speedup in inference while maintaining performance scores (68.4 on MMLU) close to the uncompressed 16-bit model, thereby bridging the gap between large server-side models and on-device intelligence.

**Key Points:**
- AndesVL is an efficient multimodal large language model designed for mobile deployment, achieving high performance despite aggressive quantization.
- The model was quantized from 16-bit precision down to 4-bit precision, resulting in a 3x speedup in inference time.
- Despite quantization, the model maintained a strong MMLU score of 68.4, closely matching the 16-bit baseline (70.9).
- The model employs a novel training method that uses a 'Draft and Verify' approach, where a smaller, faster model drafts responses and a larger model verifies them.
- AndesVL uses Group Relative Policy Optimization (GRPO) for compression and is designed to operate locally on a mobile device, eliminating reliance on the cloud for core functions.
- The model's architecture allows it to perform complex reasoning, like handling screenshots and navigating app UIs, directly on the device, surpassing the capabilities of simpler models.
- The overall strategy is to bridge the gap between massive server models and constrained mobile hardware, enabling powerful AI features locally.

![Screenshot at 00:12: The speaker describes trying to use visual lookup features on his phone while in his car, illustrating the real-world need for efficient on-device AI processing that AndesVL aims to solve.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-00-12.jpg)

**Context:** The video discusses the technical report for AndesVL, an efficient multimodal Large Language Model (LLM) developed by the Andes VL team at the OPPO AI Center. The core focus is on overcoming the constraints of mobile hardware—like limited memory, battery life, and processing power—to run sophisticated AI tasks, such as image understanding and complex reasoning, directly on a smartphone rather than relying on cloud processing.

## Detailed Analysis

The discussion centers on the AndesVL multimodal large language model, which represents a significant step toward powerful on-device AI. The key innovation is aggressive quantization: the model is compressed from 16-bit precision to 4-bit precision, resulting in a 3x speedup during inference while suffering only a minimal performance drop (from 70.9 to 68.4 on the MMLU benchmark). This efficiency is crucial for mobile deployment, as large models typically strain battery and memory. The model uses a two-stage training method involving a faster 'Draft' model and a larger 'Verify' model, specifically employing Group Relative Policy Optimization (GRPO) for compression. This approach allows the model to efficiently handle complex tasks like interpreting screenshots and navigating user interfaces locally, avoiding latency and privacy concerns associated with cloud processing. The success of AndesVL suggests a shift away from relying solely on massive, centralized models toward practical, powerful local intelligence, effectively closing the gap between server-side capabilities and mobile constraints.

### AndesVL Model Overview

- Efficient multimodal LLM for mobile
- Quantized from 16-bit to 4-bit precision
- Achieved 3x inference speedup

### Performance Metrics

- MMLU score of 68.4, comparable to 16-bit baseline of 70.9
- Outperformed VL3.5B by 4.5 points

### Training Methodology

- Utilizes a 'Draft and Verify' method
- Employs GRPO for compression
- Focuses on reasoning, math, and logic puzzles

### Mobile Deployment Advantage

- Runs locally, reducing latency and battery drain
- Avoids sending sensitive data (like photos/location) to the cloud
- UI elements (icons, menus) can be processed locally

### Strategic Implications

- Marks a shift towards on-device intelligence
- Solves the 'bigger is better' narrative by proving efficiency is key for mobile adoption

![Screenshot at 00:00: Video introduction screen showing the podcast image and 'Become A Member Today!' overlay.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-00-00.jpg)
![Screenshot at 00:35: Speaker discussing the different large cloud models \(GPT-4o, Sonnet, Gemini\) that are currently cloud-reliant.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-00-35.jpg)
![Screenshot at 01:18: Visual representation of the paper being discussed, highlighting the concept of 'fresh air' that on-device AI brings.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-01-18.jpg)
![Screenshot at 02:54: Speaker referencing 'Navit' or Native Resolution VIT, which avoids the cropping habit of traditional models.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-02-54.jpg)
![Screenshot at 04:46: Speaker emphasizing that AndesVL's approach is one of the most strategic moves in the paper.](https://ss.rapidrecap.app/screens/V2vCwlOzhdE/00-04-46.jpg)