# EUROLLM-22B: Technical Report Source: https://www.youtube.com/watch?v=HxegNqfHjas Recap page: https://rapidrecap.app/video/HxegNqfHjas Generated: 2026-02-08T16:04:04.211+00:00 --- ## Quick Overview The EuroLM-22B model, developed by a massive coalition of European researchers, significantly outperforms its American and Chinese counterparts on multilingual benchmarks, particularly excelling in tasks requiring complex reasoning and nuance, while also showing a massive leap in performance over previous iterations like the 70B model, despite being smaller. **Key Points:** - EuroLM-22B outperforms larger American (Llama 2 70B) and Chinese models on multilingual benchmarks. - The model was trained on a massive 4 trillion token dataset, including 22 official EU languages and 11 others (Arabic, Russian, Turkish). - The training methodology emphasizes data quality, using a clean, curated dataset rather than just massive volume, leading to better reasoning and less factual errors. - EuroLM-22B successfully passed the MMLU benchmark, achieving a score of 5/5, unlike models trained on noisy data which scored 1. - The model's architecture uses RoPE (Rotary Positional Embeddings) and a three-phase curriculum (bulk, annealing, fine-tuning) for superior performance. - The model demonstrates a strong ability to handle complex logic and nuance, outperforming larger models in specific tests. ![Screenshot at 0:00: The opening frame features the podcast logo for "Really Easy AI" with the prompt "Become A Member Today!", setting the stage for a discussion about AI research.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-00-00.jpg) **Context:** This video discusses the technical report for EuroLM-22B, a large language model developed by a coalition of European researchers in response to the dominance of American and Chinese AI models. The report details the model's architecture, training data, and performance benchmarks, highlighting a strategic shift towards quality over sheer quantity in training data to achieve superior multilingual capabilities and reasoning skills. ## Detailed Analysis The discussion centers on the technical report for the EuroLM-22B model, a large language model developed by a massive European research coalition aiming for AI sovereignty. The model employs a 22B parameter size, which is smaller than giants like Llama 2 70B, yet it shows superior performance in multilingual benchmarks, especially for European languages. The training data was immense, consisting of 4 trillion tokens covering 22 official EU languages plus 11 others, including Arabic, Russian, and Turkish. A key differentiator is the focus on data quality; the researchers filtered out low-quality, noisy data, including pages with bad word-to-token ratios or corrupted code, which led to better performance. The model employs a three-phase curriculum: bulk training, annealing (where the learning rate is reduced), and fine-tuning. The researchers specifically used high-quality data for instruction tuning, even outperforming larger models like Llama 2 70B on benchmarks like MMLU (achieving a 5/5 score). The model uses RoPE positional embeddings and is deliberately designed to avoid becoming a monolithic 'super-brain,' instead favoring a council of smaller, sharper models. This approach allows EuroLM-22B to show strong performance in reasoning and nuance without the massive computational demands of the largest proprietary models. ### EuroLM-22B Overview - 22B parameters - Trained on 4 trillion tokens - Covers 22 EU languages + 11 others (Arabic, Russian, Turkish) ### Training Methodology - Three phases: bulk training, annealing, fine-tuning - Emphasis on high-quality, clean data curation (filtering out low-quality web data and boilerplate code) ### Performance Benchmarks - Outperforms Llama 2 70B on multilingual tests - Achieved 5/5 on MMLU test, while larger models scored 1 due to noisy data ### Architectural Details - Uses RoPE (Rotary Positional Embeddings) - Distilled from a larger model (likely a 70B scale) using the EuroFilter and EuroBlocks methodology ### Implications - Proves that high-quality, curated data can allow smaller models to outperform larger, noisier ones - Supports a future of specialized, sovereign AI models rather than a single super-brain ![Screenshot at 0:00: The opening screen displaying the podcast's call to action to 'Become A Member Today!' over an audio waveform graphic.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-00-00.jpg) ![Screenshot at 0:54: The speaker mentions the specific technical report being analyzed: 'EuroLM 22B Technical Report' by Ramos et al.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-00-54.jpg) ![Screenshot at 2:02: The speaker highlights the model size: '22 billion parameter model' being trained on the 'EuroWeb' dataset.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-02-02.jpg) ![Screenshot at 4:47: The speaker emphasizes the model's superior performance over larger models like the 70B model on certain benchmarks.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-04-47.jpg) ![Screenshot at 11:34: The speaker compares the model's performance to the larger Llama 2 70B model, noting the smaller model's efficiency.](https://ss.rapidrecap.app/screens/HxegNqfHjas/00-11-34.jpg)