How AI is trained: Pre-training, mid-training, and post-training explained | Lex Fridman Podcast

Quick Overview

The ML training pipeline involves three distinct stages—Pre-training, Mid-training, and Post-training—where Pre-training builds foundational knowledge on massive data, Mid-training refines reasoning on curated data, and Post-training aligns the model using human feedback, leading to the final robust performance.

Key Points: Pre-training is the initial phase where an ML model builds general knowledge, linguistic fluency, and foundational representations via self-supervised learning on massive, diverse data. Mid-training is an intermediate stage where the model is refined using curated, high-quality data to enhance specific reasoning or technical capabilities. Post-training is the final refinement and alignment phase utilizing human feedback, verifiable rewards, and task-specific supervision to achieve robust reasoning and interactive behavior. The difficulty in scaling LLMs currently lies in finding high-quality, legally permissible data for pre-training, as common crawl data is often unverified or unlicensed. One paper mentioned involved Anthropic paying a record $1.5 billion settlement to authors/publishers in 2025 for using both legally purchased (scanned) and illegally downloaded (pirated) books for training. The concept of 'Mid-training' emerged because training only with Pre-training and Post-training left an awkward gap in specialized reasoning development. Scaling LLMs like GPT-4 often involves training smaller models more effectively on high-quality data rather than simply scaling up the size of the base model.

Context: This segment of the Lex Fridman Podcast features a discussion detailing the three primary stages of training large language models (LLMs): Pre-training, Mid-training, and Post-training. The speakers explore the purpose and data requirements for each stage, highlighting the shift from purely massive datasets in the early stages to more curated and human-aligned data in later stages, particularly focusing on the challenges related to data legality and quality.