TranslateGemma Technical Report

Quick Overview

The Google Translate team achieved substantial gains in translation quality and efficiency by fine-tuning the 7B parameter Translate Gemma model using a two-stage process that incorporated both generic and specialized instruction-following data, resulting in better performance across various metrics compared to the baseline model.

Key Points: Translate Gemma was fine-tuned using a two-stage process: Stage 1 (Supervised Fine-Tuning/SFT) and Stage 2 (Reinforcement Learning from Human Feedback/RLHF). The fine-tuning utilized a large, high-quality synthetic dataset (30% of the mix) generated by Gemini 2.5 Flash, alongside human-generated data. The 7B parameter Translate Gemma model achieved an average Metric X score of 3.09, significantly outperforming the 12B parameter baseline model's score of 4.04 (where a lower score is better). The specialized fine-tuning focused on difficult tasks like proper noun translation (names, companies, locations) and multilingual capabilities. The technique successfully taught the model to prioritize high-quality, natural-sounding translations over mere literal accuracy, avoiding catastrophic forgetting. The core technical insight involves using the human evaluation metric (MQM) to guide the RL stage, ensuring the model learns what humans consider good translation quality.

Context: The video discusses a technical report from the Google Translate research team detailing improvements made to their Translate Gemma model, specifically focusing on the fine-tuning methodology used to enhance its translation capabilities, particularly for complex or nuanced language tasks.

Detailed Analysis

The Google Translate team improved translation quality by fine-tuning the Translate Gemma model through a two-stage process involving SFT and RLHF. They mixed 30% large-scale synthetic data generated by Gemini 2.5 Flash with other data. This approach allowed the smaller 7B model to outperform the baseline 12B model on the Metric X score (3.09 vs 4.04). The key was focusing the fine-tuning on difficult areas like proper noun translation and ensuring the model learned to mimic human quality assessments (like MQM scores) rather than just following generic instructions. This resulted in better performance across various facets, including fluency and accuracy, especially for low-resource language pairs like English to Marathi, where the specialized model showed a massive 1.6-point improvement over the baseline.