GPT Live: Inside OpenAI's New Voice AI Model with Peter Bakkum | Coval | Coval

The Gist

OpenAI's GPT Live model solves voice AI turn detection through a continuous full duplex speech architecture that eliminates cascaded text translation models. It also uses back-end model delegation to handle complex reasoning tasks without introducing conversation latency.

Quick Overview

OpenAI technical staff member Peter Bakkum joins Coval CEO Brooke Hopkins to break down the technical architecture of GPT Live, OpenAI's new full duplex speech model launched for public general availability. The discussion spoils how GPT Live replaces traditional cascaded pipelines of transcription, text models, and text-to-speech with a single native audio model. Bakkum details the shift from turn-based voice activity detection to continuous listening, explaining how back-end delegation handles long-running tasks, how sideband connections secure application control, and why evaluation frameworks built for older architectures must be completely rewritten.

Key Points: OpenAI launched GPT Live as a full duplex voice model with native audio input and output for public general availability. Traditional voice architectures string together a transcription model, a text model, and a text-to-speech model, which creates latency and unnatural turn-taking. GPT Live operates as a single continuous audio model that handles noise reduction, echo cancellation, and intelligent turn detection natively. The model utilizes back-end delegation to call text models for complex tasks like code analysis while maintaining a responsive conversation stream. Sideband connections like WebSockets and WebRTC allow application backends to control live voice sessions securely without exposing API keys on the frontend. Legacy voice agent evals built for turn-based models fail because GPT Live is termless and features no definitive cut-off point. Bakkum concludes that by 2026 developers should abandon manual prompt writing in favor of getting models to write prompts and manage evals programmatically.

Context: OpenAI recently launched GPT Live for public general availability, introducing a native speech-to-speech architecture designed to replace legacy cascaded voice agents. Coval, an evaluation platform for voice AI agents, has been testing the model as an early design partner, positioning this interview to dissect the exact engineering differences between legacy systems and OpenAI's newest real-time stack.