# The huge cost of training LLMs | Lex Fridman Podcast

Source: https://www.youtube.com/watch?v=qeAdgD6A19E
Recap page: https://rapidrecap.app/video/qeAdgD6A19E
Generated: 2026-02-04T01:32:59.869+00:00

---
## Quick Overview

The high cost of training large language models (LLMs) is becoming a significant bottleneck, with current estimates suggesting that training models like GPT-4 cost around $100 million, though improvements in efficiency mean future models might cost less per unit of compute, potentially around $1,000 per GPU day, which is still substantial given the scale of compute needed for models in the trillion-parameter range.

**Key Points:**
- Training LLMs like GPT-4 is estimated to have cost around $100 million, highlighting the massive initial investment required.
- Despite increasing model size towards the trillion-parameter range, training efficiency improvements mean the cost per GPU day might drop to around $1,000.
- The speaker points out that while pre-training costs are high, the recurring cost of serving these models to millions of users remains relatively low compared to the initial training expense.
- The ATOM Project, focused on invigorating AI research by building leading open models for the US, aims to counter US lagging behind in open-source models compared to China.
- The speaker references data from the ATOM Project website showing global model momentum, indicating US downloads lagging behind China's surge in usage.
- The massive computational needs for training these models necessitate large, dedicated compute clusters, with contracts for massive Blackwall-scale compute facilities already being sought out by organizations.

![Screenshot at 00:02: Lex Fridman \(left\) and his guest \(right\) are engaged in a podcast discussion, set against a dark background with bookshelves visible, focusing on the technical and economic challenges of scaling AI models.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-00-02.jpg)

**Context:** This segment features a discussion between Lex Fridman and an interviewee, likely related to Artificial Intelligence development, specifically focusing on the economics and infrastructure required to train state-of-the-art Large Language Models (LLMs). The conversation touches upon the sheer scale of computational power needed, the costs associated with this training, and the geopolitical implications of open vs. closed models, referencing initiatives like The ATOM Project.

## Detailed Analysis

The conversation centers on the prohibitive costs associated with training massive AI models, such as GPT-4, which is estimated to have cost around $100 million for its initial training run. The interviewee argues that while training costs are extremely high, efficiency improvements driven by research are pushing the cost per GPU day down to approximately $1,000, even as models scale towards the trillion-parameter range. This cost reduction is crucial because the subsequent serving costs for these models to a large user base are comparatively low. The interviewee mentions that the rapid progress, exemplified by models like Claude Opus 4.5 quickly surpassing previous benchmarks, suggests that this trend of rapidly improving models is likely to continue, meaning future large-scale compute clusters will still be necessary. The discussion also references The ATOM Project, an initiative aimed at strengthening US leadership in open-source AI research, noting that the US currently lags behind China in model download momentum, as evidenced by data presented on screen. The intense computational demand is driving large infrastructure investments, with entities seeking contracts for gigawatt-scale compute clusters.

### LLM Training Economics

- Training GPT-4 cost ~$100 million
- Cost per GPU day might drop to ~$1,000 due to efficiency gains
- Serving costs are low relative to initial training costs

### Model Scaling and Progress

- Models are scaling to trillion parameters
- Newer models like Claude Opus 4.5 quickly surpassed older benchmarks
- Continued progress suggests models will keep getting better, demanding huge compute.

### AI Geopolitics and Infrastructure

- Reference to The ATOM Project driving open models in the US
- US download momentum lags China's
- Companies are seeking contracts for gigawatt-scale compute clusters for 2026.

![Screenshot at 00:03: Lex Fridman in his studio setting, dressed formally, listening to his guest during the podcast discussion.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-00-03.jpg)
![Screenshot at 00:23: The guest, wearing a plaid shirt, gesturing while explaining the cost efficiency of modern LLM training.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-00-23.jpg)
![Screenshot at 01:59: The guest smiles while discussing the rapid progression of AI models and the continuous need for larger compute resources.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-01-59.jpg)
![Screenshot at 02:01: A screenshot of The ATOM Project website is displayed, featuring a graph showing 'Global Model Momentum' with US and Chinese download trends diverging.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-02-01.jpg)
![Screenshot at 02:26: Lex Fridman listens intently, maintaining a serious expression while his guest details the infrastructure demands for future AI training.](https://ss.rapidrecap.app/screens/qeAdgD6A19E/00-02-26.jpg)