# Orchestrating Agents at Scale Source: https://www.youtube.com/watch?v=KplSDxYv9xU Recap page: https://rapidrecap.app/video/KplSDxYv9xU Generated: 2025-10-08T18:03:51.979+00:00 --- ## Quick Overview OpenAI introduced AgentKit at DevDay 2025, a complete set of building blocks designed to help developers rapidly build, deploy, and optimize AI agent workflows, as demonstrated by James Hills and Rohan with a complex **Key Points:** - AgentKit was introduced at OpenAI DevDay 2025, providing building blocks for creating, deploying, and optimizing AI agent workflows. - James Hills demonstrated AgentKit's Agent Builder, a visual flow editor allowing users to connect nodes like 'File Search' and 'MCP Tool' without writing extensive code. - A live demonstration involved creating an agent workflow to resolve a 'low fuel economy' issue for a truck maintenance company called 'Truck Brothers', which required accessing maintenance manuals and parts lists. - Rohan showed how the system handles complex logic, including external tool calls like a GraphQL query via the 'MCP Tool' node, and integrates guardrails for safety and correctness checks (e.g., Hallucination detection). - The optimization feature allows users to iterate quickly on agent performance, demonstrated by improving the 'Correctness' score in the Synthesis Agent grader from 40% to 80% by updating the prompt. - AgentKit supports self-hosting models via the ChatKit protocol, allowing users to run agents on their own infrastructure rather than solely on OpenAI's cloud. - The session concluded by showcasing the ease of exporting the entire workflow as code (TypeScript or Python) using the OpenAI Agents SDK. ![Screenshot at 00:05: The title slide for the session, displaying "OpenAI DevDay \[2025\]" followed by the presentation title "Orchestrating Agents at Scale", sets the context for the announcement of new agent development tools.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-00-05.png) **Context:** The presentation, delivered by James Hills and Rohan from OpenAI's Forward Deployed Engineering team during OpenAI DevDay 2025, focused on the launch of AgentKit. AgentKit is presented as a solution to simplify the engineering process of creating production-ready, reliable AI agents that can perform multi-step reasoning and interact with external tools and data sources. ## Detailed Analysis The session introduced AgentKit, a new platform feature designed to streamline the development, deployment, and optimization of AI agents. James Hills kicked off the presentation by showcasing the Agent Builder, a visual workflow editor that allows developers to construct complex agent logic by connecting pre-built nodes like 'Start', 'Query Expansion', 'File Search', 'Transform', 'MCP Tool', and 'Synthesis Agent'. The demonstration centered on solving a real-world maintenance problem for a fictional company, 'Truck Brothers', concerning low fuel economy. This involved using tools to search maintenance manuals (vector store search) and query an internal system via a GraphQL tool ('MCP Tool') to retrieve necessary parts and procedures. Rohan followed up by detailing the importance of reliability and optimization, showing how built-in 'Guardrails' nodes (like hallucination detection) ensure outputs are grounded in trusted data. He also demonstrated the evaluation interface, where agents are graded against correctness and formatting criteria, and how optimization involves iterating on prompts or tool configurations based on these metrics. A key feature highlighted was the ability to export the entire visual workflow as runnable code (TypeScript or Python) using the OpenAI Agents SDK, allowing for self-hosting on private infrastructure, thereby addressing compliance and data residency concerns. ### AgentKit Introduction - Launch of AgentKit - Complete set of building blocks for agents - Supports build, deploy, and optimize phases ### Agent Builder Demonstration - Visual workflow creation using nodes like File Search, MCP Tool, and Synthesis Agent - Solved a 'low fuel economy' maintenance issue for 'Truck Brothers' ### Tool Integration (MCP Tool) - Demonstrates calling external tools via GraphQL query, requiring specific procedure IDs retrieved from previous steps. ### Safety and Evaluation - Integrated 'Guardrails' node for checks like hallucination detection - Synthesis Agent grader shows correctness and formatting scores based on ground truth data. ### Optimization and Iteration - Ability to easily iterate on prompts and agents directly in the UI, resulting in significant score improvements (e.g., Correctness from 40% to 80%). ### Deployment Flexibility - Workflows can be exported as code (TypeScript/Python) via the Agents SDK, enabling self-hosting outside the OpenAI cloud. ![Screenshot at 00:15: Introduction of AgentKit as a complete set of building blocks for agent workflows.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-00-15.png) ![Screenshot at 00:37: Visual representation of the Agent Builder workflow, showing connected nodes like 'Start', 'Query Expansion', 'File Search', and 'Guardrail'.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-00-37.png) ![Screenshot at 00:44: Demonstration of ChatKit, showing UI customization options for embedding chat interfaces directly into applications.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-00-44.png) ![Screenshot at 01:21: Scenario visualization: A semi-truck on a desert road, representing the 'Truck Brothers' maintenance use case being solved by the agent.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-01-21.png) ![Screenshot at 02:01: The 'Truck Brothers' landing screen with the flaming truck logo, used as the initial context for the agent query.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-02-01.png) ![Screenshot at 03:05: The agent workflow executing, with progress bars showing 'Searching files', 'Finding parts', and 'Running guardrails' completion.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-03-05.png) ![Screenshot at 05:45: Configuration window for the 'MCP Tool' node, showing the definition of a GraphQL query tool with a specific endpoint and allowed tools.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-05-45.png) ![Screenshot at 06:17: Configuration dialog for the 'Hallucination guardrail' node, showing the required vector store ID pointing to the Truck Brothers repair manuals.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-06-17.png) ![Screenshot at 07:22: The final output display showing the agent successfully summarized the 'Oil Warning Rescue for Your Tractor' procedure, including parts and instructions.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-07-22.png) ![Screenshot at 10:35: Viewing the generated code in an editor \(VS Code\), demonstrating that the workflow logic can be exported as TypeScript or Python code.](https://ss.rapidrecap.app/screens/KplSDxYv9xU/00-10-35.png)