Tiny Aya - Cohere's Mini Multilingual Models

Quick Overview

Cohere Labs launched the Tiny Aya family of specialized, small-sized (3.35B parameter) multilingual models, including TinyAya-Base and TinyAya-Global, designed to offer high performance across 67 languages, significantly outperforming larger models on low-resource languages and tokenization efficiency compared to models like Gemma 3 and Qwen 3.

Key Points: Cohere Labs released the Tiny Aya family of models, including the 3.35B parameter TinyAya-Base, which covers over 70 languages, including many lower-resourced ones. TinyAya-Global is a powerful instruction-tuned multilingual model built on TinyAya-Base, achieving strong, balanced performance across 67 supported languages. The development process involved regional specialization: models were trained on region-specific data (Europe, West Asia, Asia Pacific, Africa, South Asia) and then merged, resulting in specialized models like TinyAya-Water, Earth, and Fire. TinyAya models demonstrated superior tokenization efficiency compared to larger models like Gemma 3, Qwen 3, and SimoLLM 3 across various scripts, especially for non-Latin scripts like Greek and Indic languages. In generation quality benchmarks across five regions, TinyAya instruction-tuned models competed effectively with existing massively multilingual models, notably showing superior performance in Africa compared to competitors. The overall goal of Tiny Aya is to bridge the gap between model performance, coverage, and efficiency, enabling powerful, adaptable AI to run locally on devices like phones.

Context: The video discusses the release of Cohere Labs' Tiny Aya suite of multilingual AI models, focusing on how these smaller, efficient models address the performance gap often experienced by lower-resource languages in larger, more common LLMs. The presentation contrasts the Tiny Aya approach, which emphasizes regional specialization and efficient tokenization, against established models like those from the Gemma and Qwen families, using performance charts and diagrams to illustrate their findings.