# Jet-RL: Enabling On-Policy FP8 RL with Unified Training and Rollout Precision Flow

Source: https://www.youtube.com/watch?v=Jphh6MQySxE
Recap page: https://rapidrecap.app/video/Jphh6MQySxE
Generated: 2026-01-29T22:04:55.933+00:00

---
## Quick Overview

The Jet-RL paper introduces a unified training and rollout precision flow, utilizing 8-bit precision for reinforcement learning (RL) to significantly reduce computational costs and memory footprint without substantial performance degradation, especially by keeping the backward pass gradients in higher precision.

**Key Points:**
- Jet-RL enables on-policy FP8 reinforcement learning by using a unified precision flow for training and rollout.
- The method keeps the backward pass gradients in higher precision (e.g., BF16) while using 8-bit precision for the forward pass and rollout, which accounts for 70% of the total training time.
- This approach resulted in a 1.33x speedup for the 32-billion parameter model and achieved a 10% performance drop compared to the expensive full-precision BF16 training on the GSM8K mathematical reasoning benchmark.
- The backward pass gradient update is crucial; if gradients were also 8-bit, the model failed to converge due to a catastrophic accuracy collapse.
- The forward pass (rollout) is 8-bit quantized, while the backward pass (gradient update) remains at BF16 precision, balancing speed and accuracy.
- Jet-RL achieved a 1.32x speedup on the 32B parameter model while maintaining competitive performance against the full-precision baseline.

![Screenshot at 00:06: The speaker discusses the most expensive bottleneck in modern AI training, referencing the need for a better precision flow for reasoning models like those from OpenAI \(e.g., GPT-4/O1\) or DeepSeek.](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-00-06.jpg)

**Context:** The video discusses the paper 'Jet-RL: Enabling On-Policy FP8 RL with Unified Training and Rollout Precision Flow,' which addresses the high computational cost bottleneck in modern AI training, particularly for reinforcement learning (RL) tasks involving long reasoning chains. The research, involving collaborations from Nvidia, MIT, UC Berkeley, and Stanford, proposes a new precision flow to make training large models more efficient by leveraging 8-bit floating-point (FP8) arithmetic.

## Detailed Analysis

The Jet-RL paper tackles the immense computational cost associated with training large language models, specifically focusing on reinforcement learning (RL) that requires long chains of thought. The core innovation is the 'Unified Training and Rollout Precision Flow,' which leverages FP8 precision. The authors, from top institutions like Nvidia, MIT, UC Berkeley, and Stanford, found that the forward pass and rollout, which consume 70% of training time, can be run in 8-bit precision (FP8) without severe performance loss. Crucially, they determined that the backward pass gradients must remain in higher precision (like BF16); if gradients were also 8-bit, the model experienced a catastrophic accuracy collapse, failing to converge. By keeping the backward pass gradients precise while using 8-bit for the forward pass, they achieved a 1.33x speedup on a 32B parameter model compared to the full-precision BF16 training, with only a small 10% accuracy drop on the GSM8K reasoning benchmark. This method effectively mitigates the computational bottleneck, proving that consistent, low-precision operation during rollout can be beneficial, provided the optimization step (gradient update) retains sufficient precision.

### Jet-RL Core Concept

- Unified Training and Rollout Precision Flow
- Enables on-policy FP8 RL training
- Balances speed from 8-bit forward pass with stability from higher-precision backward pass

### Performance Metrics

- 1.33x speedup on 32B model during rollout
- 10% accuracy drop on GSM8K compared to BF16 baseline
- 70% of training time spent on rollout/forward pass

### Critical Finding

- Gradient Precision: Keeping backward pass gradients in higher precision (BF16) is essential
- 8-bit gradients lead to catastrophic accuracy collapse and failure to converge

### Comparison to Other Methods

- Contrast starkly with standard BF16 training and older chatbot tricks like 'dead reckoning'
- The new method forces models to learn and act within the constraints of the lower precision (FP8) environment.

### Quantization Strategy

- 128x128 block quantization for weights and activations
- Activations are transient and dynamic, while weights are static, leading to tailored quantization.

![Screenshot at 00:00: Title slide featuring podcast imagery and 'Become A Member Today!' call to action.](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-00-00.jpg)
![Screenshot at 00:12: Speaker introduces the paper's title focusing on FP8 RL and unified precision flow.](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-00-12.jpg)
![Screenshot at 01:17: Visual describing the two phases: Rollout \(70% of time, 8-bit\) and Training \(backward pass\).](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-01-17.jpg)
![Screenshot at 02:02: Speaker breaks down the acronyms: BF16 is Brain Floating Point 16, FP8 is 8-bit format.](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-02-02.jpg)
![Screenshot at 07:30: Speaker contrasts the two models: System 1 \(Instinctual, fast, low precision\) vs. System 2 \(Reasoning, slow, high precision\).](https://ss.rapidrecap.app/screens/Jphh6MQySxE/00-07-30.jpg)