# Build Hour: Built-In Tools Source: https://www.youtube.com/watch?v=8St8elOrRPY Recap page: https://rapidrecap.app/video/8St8elOrRPY Generated: 2025-09-03T01:01:19.421+00:00 --- ## Quick Overview OpenAI's Build Hour on built-in tools demonstrated how to easily integrate advanced capabilities like web search, file search, MCP (Model Context Protocol) for external APIs, and code interpretation into applications without custom coding. These tools automate complex tasks, leverage OpenAI's infrastructure, and allow for rapid iteration, as showcased by integrating them into a data exploration dashboard and by Hebia using web search for financial and legal workflows. **Key Points:** - Built-in tools empower LLMs with external capabilities like web search and file access, automating complex tasks without requiring users to write custom code, as they are executed on OpenAI's infrastructure. - Web search is crucial for LLMs to access real-time information beyond their training data cutoff (May 2024), enabling up-to-date answers, as demonstrated by Hebia for financial and legal use cases. - File search allows LLMs to query private knowledge bases, automating the retrieval-augmented generation (RAG) process, which normally requires significant setup for data preprocessing and retrieval optimization. - The MCP tool enables LLMs to interact with hundreds of external tools by connecting to any MCP server, allowing integration with custom APIs or services like Shopify, as shown in the playground demo. - Code interpreter can execute Python code to solve problems, perform data analysis, and generate charts, integrating seamlessly with other tools like web search for complex data tasks. - Integrating built-in tools into applications is as simple as adding a few lines of code, often by copying configurations from the OpenAI playground, significantly reducing development time and complexity. - Hebia utilizes web search extensively in its 'matrix' product for large-scale data analysis across thousands of companies and in its 'Chat and Deep Research' product to overcome LLM knowledge cutoffs and explore information before exploiting specific data sources. **Context:** This Build Hour session, hosted by Christine and Katya from OpenAI, focuses on 'Built-in Tools,' a feature designed to enhance the capabilities of Large Language Models (LLMs). The goal is to empower developers with best practices and AI expertise to scale their companies using OpenAI APIs. The session covers what built-in tools are, how they differ from function calling, introduces available tools like web search and file search, demonstrates their use in the OpenAI playground, and shows how to integrate them into custom applications, including a live demo from Hebia, a company using these tools for financial and legal workflows. ## Detailed Analysis The session defines built-in tools as a way to add capabilities to LLMs, enabling them to interact with external data or perform actions that LLMs cannot do on their own. Unlike function calling, where developers execute code on their own infrastructure, built-in tools are executed directly on OpenAI's infrastructure, simplifying integration and abstracting away complexity. The core benefit is that users don't need to write or manage the underlying code for these tools. The presentation highlights six available tools: web search for real-time internet data, file search for querying internal knowledge bases (automating RAG), MCP (Model Context Protocol) for accessing hundreds of external tools via MCP servers, code interpreter for executing Python code and generating outputs like charts, computer use for visual interface interaction (requiring local execution), and image generation (not covered in detail). Katya demonstrates these tools in the OpenAI playground, showing how to easily add file search to query uploaded documents, use web search for current events, and connect to Shopify via MCP to find e-bikes. The session also showcases combining web search with code interpreter to analyze weather data and generate charts. For application integration, the demo moves to a Python script and a front-end application, illustrating how tools are added via a simple 'tools' array in the API call. The integration process involves copying tool configurations from the playground and updating the application's system prompt for better model guidance. Hebia's technical lead, Will, shares how his company uses web search to overcome LLM knowledge cutoffs and gather up-to-date information for financial and legal analysis, particularly in their 'matrix' product for large-scale data analysis and their 'Chat and Deep Research' product, emphasizing the 'explore-exploit' trade-off. He also touches on MCP, noting Hebia's preference for custom solutions due to specific indexing needs but acknowledging MCP's value in bringing offline data online. The session concludes by reiterating that built-in tools allow for minimal code integration, reduced complexity, and faster development of powerful agentic experiences. ### Introduction - Welcome and Build Hour Goal: Empowering developers with AI expertise, best practices, and tools to scale companies using OpenAI APIs. - Topics Covered: Explanation of built-in tools, playground demos, app integration, customer spotlight (Hebia), and Q&A. ### What are Built-in Tools? - Tools add capabilities to LLMs, enabling interaction with data and actions beyond their inherent abilities. - Key Distinction from Function Calling: Built-in tools execute on OpenAI's infrastructure, eliminating the need for developers to manage code execution, unlike function calling which requires client-side execution. ### Available Built-in Tools - Web Search: Accesses real-time internet data, overcoming LLM knowledge cutoffs (May 2024). - File Search: Queries internal knowledge bases, automating RAG processes. - MCP Tool: Connects to any MCP server, providing access to hundreds of external tools (e.g., Shopify). - Code Interpreter: Executes Python code for data analysis and chart generation. - Computer Use: Allows models to interact with visual interfaces (requires local execution). - Image Generation: A separate topic not covered in detail. ### Playground Demonstrations - File Search: Uploading PDFs about tools to query for specific information, demonstrating RAG automation. - Web Search: Asking about GPT-5 launch date, showing real-time web querying. - MCP Tool (Shopify): Connecting to a Shopify server to search for e-bikes based on user preferences. - Code Interpreter & Web Search Combination: Checking SF weather, then asking code interpreter to compare historical data and generate a chart. - Stripe MCP Tool: Connecting to Stripe to retrieve account balance. ### Application Integration - Python Script Example: Demonstrates adding web search to a script with minimal code. - Front-end App Integration: Shows adding multiple tools (Stripe MCP, web search, code interpreter) to a dashboard application by updating a 'tools' array. - Custom Function Integration: Using a 'generate component' function with structured output to populate dashboard elements (cards, charts, tables) based on tool results. ### Customer Spotlight (Hebia) - Hebia uses web search to address LLM knowledge cutoffs for financial and legal workflows, enabling up-to-date information retrieval. - Matrix Product: Uses web search at scale for large-scale data analysis across thousands of companies, fetching factual information in parallel. - Chat and Deep Research Product: Employs web search in an 'explore-exploit' strategy to gather broad information before deep dives into specific data sources like SEC filings. ### Benefits and Conclusion - Ease of Integration: Minimal code required, often by copying from playground or documentation. - Reduced Complexity: Abstracts away the underlying technical complexities of tool execution. - Enhanced Agentic Experiences: Enables powerful agents capable of solving complex problems by combining multiple tools sequentially. - Speed of Development: Significantly accelerates the building of AI-powered applications.