# Build Reliable AI Agents Part 1: API Specs, Testing, & Monitoring Source: https://www.youtube.com/watch?v=cFcbOc2KJh0 Recap page: https://rapidrecap.app/video/cFcbOc2KJh0 Generated: 2025-11-14T23:04:14.025+00:00 --- ## Quick Overview The video demonstrates how to build reliable AI agents by integrating Postman for API specification, testing, and monitoring with Langfuse for observability, focusing on four key requirements: creating clean API contracts, ensuring service availability, maintaining response consistency, and implementing agent observability. **Key Points:** - The process for building reliable AI agents requires four key steps: clean API contracts, service availability checks, consistent responses, and AI agent observability. - Postman Agent Mode is used to automate the creation of API specifications (like OpenAPI specs) for collections, as demonstrated by generating a spec for the 'Get Coding Resources' endpoint (0:04). - The presenter added unit tests within Postman's pre-response script to validate status codes, response structure, and content, ensuring correctness before deployment (4:16). - Postman Monitors were configured for the 'Get Coding Resources' endpoint to run hourly, ensuring the endpoint remains healthy and the response structure is consistent (5:07). - Langfuse observability is integrated to track token usage, cost, and monitor performance metrics like response length and relevance scores during agent execution (0:16, 1:55). - The presenter successfully added an API server tool to the agent via Agent Mode to manage the monitoring setup and ensure all components function reliably in production (8:30). ![Screenshot at 0:03: The video title overlay 'Building reliable AI agents \(Postman + Langfuse\)' appears over the Postman interface, introducing the primary tools and objective of the tutorial.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-00-03.png) **Context:** This video is the second part of a series focusing on creating robust and trustworthy AI agents, specifically detailing the implementation of necessary checks and frameworks for reliability. The presenter, Quinton Wall (Head of Developer Relations at Postman), utilizes Postman's features, including Agent Mode and Monitors, alongside the Langfuse observability framework to manage API specifications, testing, and continuous monitoring. ## Detailed Analysis The tutorial outlines four critical requirements for building reliable AI agents: establishing a clean contract between APIs and services, verifying API/service availability, ensuring response consistency across prompts, and implementing AI agent observability. The demonstration begins by showing the Postman interface, where the presenter uses Agent Mode to quickly generate an OpenAPI specification for an existing 'Get Coding Resources' API endpoint (0:04). The presenter then navigates to the pre-response script of a Postman request to show custom tests already in place, which check for status codes (e.g., status code 200) and response array structure (4:16). After confirming the tests pass locally, the presenter sets up a Postman Monitor to run these tests hourly, ensuring the endpoint remains healthy and consistent when deployed (5:07). Furthermore, Langfuse observability is integrated to track agent performance, including token usage and relevance scores, which is crucial for understanding agent behavior in production (0:16). Finally, the presenter demonstrates using Agent Mode within Postman to proactively create an MCP server tool within the agent's capabilities and configure monitoring for new endpoints, ensuring comprehensive reliability checks across the entire system (8:30). The goal is to move from initial API definition to testing, monitoring, and observable performance tracking, all integrated through the Postman/Langfuse combination. ### Key Requirements for Reliable Agents - 1. Clean contract between API and services - 2. API and service availability assurance - 3. Consistent response generation - 4. AI Agent Observability ### API Specification Generation (Postman Agent Mode) - Use Agent Mode to create OpenAPI specs for collections - Example shown: creating an OpenAPI spec for 'Get Coding Resources' (0:04, 2:39) ### Testing and Validation - Added pre-response tests to validate status codes (200), response structure (is array), and custom error messages (4:16, 4:26) ### Monitoring Setup (Postman Monitors) - Configured a monitor to run the collection hourly using the Dev environment to ensure endpoint health and consistency (5:07, 6:37) ### Observability Integration (Langfuse) - Tracking token usage, cost, and performance metrics like response length and relevance score during agent execution (0:16, 1:55) ### Agent Tooling Integration - Created an MCP server tool via Agent Mode to manage monitoring and integrate further capabilities into the agent (8:30, 8:51) ![Screenshot at 0:04: Initial view of the Postman interface displaying the 'Coding Resources API' collection where agent building begins.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-00-04.png) ![Screenshot at 0:02: Code editor showing Python script defining an async function for agent execution, utilizing Langfuse for observability \(e.g., langfuse.log\_span\).](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-00-02.png) ![Screenshot at 0:09: Agent Mode prompt in Postman asking the user to describe what is needed, shown while setting up API specs for a collection.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-00-09.png) ![Screenshot at 4:28: Postman test results screen showing a mix of passed \(green\) and failed \(red\) tests after running the collection, indicating areas needing refinement.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-04-28.png) ![Screenshot at 5:07: Postman Monitor view showing the 'Get Coding Resources Health Check' is 'HEALTHY' after setup, indicating successful continuous monitoring.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-05-07.png) ![Screenshot at 6:06: The monitor filter settings menu expanded, showing options to filter execution types \(e.g., Manual, Scheduled run, Webhook\).](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-06-06.png) ![Screenshot at 8:55: Agent Mode displaying options to export a package of tools, indicating the agent's capabilities can be packaged for wider use.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-08-55.png) ![Screenshot at 9:12: The Postman interface showing the 'MCP Servers' management section where the newly created API server tool is listed and toggled on.](https://ss.rapidrecap.app/screens/cFcbOc2KJh0/00-09-12.png)