# Shaping Model Behavior in GPT-5.1— the OpenAI Podcast Ep. 11 Source: https://www.youtube.com/watch?v=GXAAzKX6oaQ Recap page: https://rapidrecap.app/video/GXAAzKX6oaQ Generated: 2025-12-02T18:43:57.876+00:00 --- ## Quick Overview The OpenAI team shaped model behavior in GPT-5.1 by implementing a system that balances maximizing user freedom with minimizing harm, primarily through improved instruction following, context window management, and continuous iteration based on user feedback, which resulted in models that feel warmer and are more steerable than previous versions. **Key Points:** - All models in the GPT-5.1 chat are reasoning models, meaning they can decide how much to think based on the prompt, unlike earlier models. - The primary goals for GPT-5.1 included addressing feedback on GPT-5, such as making models feel warmer and improving steering capabilities without sacrificing steerability. - A key technical change was introducing an auto-switcher that moves between the standard chat model and reasoning models, allowing for complex reasoning tasks to be handled by specialized models. - The team focused on improving instruction following, context window capacity, and ensuring models could handle subtle cues across subjective domains like personality and tone. - The work on personality features, like style and tone, is seen as a crucial step toward creating more usable and helpful models for all users. - Data gathered from user feedback, including screenshots of problematic interactions, is used to guide iterative improvements, ensuring models adhere to safety boundaries while maintaining utility. ![Screenshot at 00:23: Christina Kim explains that for the first time ever, all models in the chat are reasoning models, which allows them to dynamically decide how much to think based on the prompt.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-00-23.png) **Context:** This episode of The OpenAI Podcast features host Andrej Karpathy interviewing Christina Kim, a Research Lead at OpenAI working on post-training for models like GPT-5.1, and Laurentia Romaniuk, a Product Manager focused on model behavior. The discussion centers on the engineering and research efforts involved in shaping the behavior of the new GPT-5.1 model, specifically focusing on techniques to make the AI more steerable, less 'cold,' and better at following complex instructions while minimizing harmful outputs. ## Detailed Analysis Andrej Karpathy introduces guests Christina Kim (Research Lead, Post-training) and Laurentia Romaniuk (Product Manager, Model Behavior) to discuss shaping model behavior for GPT-5.1. Kim notes that a major goal was to make models feel 'warmer' and more steerable while maintaining performance gains. A key development in GPT-5.1 is that all models in the chat are now reasoning models, capable of deciding how much computation to spend on a prompt based on its complexity, unlike previous models. Romaniuk explains that part of their work involves creating a 'conversation link' to see exactly what happened during a user's interaction, which helps debug issues where the model might output something cold or inappropriate, like refusing to discuss sensitive topics or giving overly clinical answers. Karpathy mentions the analogy of System 1 vs. System 2 thinking from Daniel Kahneman to describe the difference between the base model and the reasoning models. Kim details that the team uses user feedback, including screenshots of negative experiences, to guide iterative improvements. She emphasizes that the goal is not to make the model perfectly safe by refusing all borderline requests, but rather to allow for maximum user freedom while minimizing harm. They are constantly working on instilling an understanding of context and intent so the model can better navigate subjective domains like personality and tone. Kim highlights that they are trying to engineer models that proactively help users by inferring intent, rather than simply reacting to explicit instructions. She mentions that one of the key improvements in 5.1 is the auto-switcher mechanism that directs queries between the standard chat model and the more capable reasoning models, allowing for more nuanced responses. Karpathy asks how they measure progress in shaping personality, and Kim explains that they look at signals from user interactions, such as whether the model's response style matches the user's expectations (e.g., warm vs. cold, detailed vs. concise). She notes that the inherent challenge is balancing user freedom (maximizing freedom) with minimizing harm, which requires constant iteration and tuning the context window to carry the right amount of information forward. ### GPT-5.1 Model Architecture - All models in the chat are reasoning models - Models use an auto-switcher to move between chat and reasoning capabilities - This allows models to dynamically allocate computation based on prompt complexity. ### Behavior Shaping Goals - Focus on making models feel 'warmer' and more steerable - Improving instruction following and context window management - Reducing overtly harmful outputs while maintaining user freedom. ### User Feedback and Iteration - User feedback, including screenshots of poor interactions, guides improvements - The team uses this feedback to test and refine models across various subjective domains like tone and personality. ### Personality and EQ - Personality is defined by the model's response style and tone, which is an area of active research - They aim to avoid overly clinical or judgmental responses, even in sensitive areas. ### Future Work and Improvements - The team is actively working on improving models' ability to infer user intent and proactively solve problems - They are also refining how models handle subjective domains and maintain consistency across different outputs. ![Screenshot at 00:03: Andrej Karpathy introduces guests Christina Kim and Laurentia Romaniuk to discuss shaping model behavior.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-00-03.png) ![Screenshot at 00:23: Christina Kim explains that for the first time, all models in the chat are reasoning models, allowing them to dynamically allocate compute resources.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-00-23.png) ![Screenshot at 00:43: Andrej Karpathy asks how they measure progress in guiding model behavior over time.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-00-43.png) ![Screenshot at 01:26: Christina Kim elaborates on how the auto-switcher moves users between the standard chat model and the reasoning model.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-01-26.png) ![Screenshot at 04:55: Andrej Karpathy discusses the interesting aspect of having multiple models available for different tasks.](https://ss.rapidrecap.app/screens/GXAAzKX6oaQ/00-04-55.png)