# Build Reliable AI Agents Part 2: Prompt Consistency, Tracing, & Costs

Source: https://www.youtube.com/watch?v=mTnXWRmZVlQ
Recap page: https://rapidrecap.app/video/mTnXWRmZVlQ
Generated: 2025-11-14T23:03:16.952+00:00

---
## Quick Overview

Building reliable AI agents requires focusing on four key areas: establishing a clean contract between APIs and services, ensuring API and service availability, maintaining consistent responses, and implementing comprehensive AI agent observability for tracing costs and performance metrics.

**Key Points:**
- The foundation of reliable AI agents involves creating a clean contract between the agent and external APIs/services (01:17).
- Ensuring API and service availability is critical, with the speaker suggesting setting up monitoring and alerts via Postman (06:41).
- Agent responses must be consistent; the speaker uses observability tools like Langfuse to trace and score responses for consistency (06:48).
- Token usage and cost must be tracked, which Langfuse automatically infers and reports based on the model used (04:31).
- The speaker demonstrates setting the model to 'gpt-4' to track its specific cost and usage metrics within the Langfuse dashboard (04:42).
- Four key areas—contract, availability, consistency, and observability—are essential for building reliable agents that scale to production (07:46).

![Screenshot at 00:27: The speaker begins diving into the Python code within VS Code, demonstrating the setup for an agent that connects to an MCP server while enabling Langfuse observability.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-00-27.png)

**Context:** This video, part two of a series on building reliable AI agents, continues the discussion initiated in part one, which focused on API readiness. Quinton Wall, Head of Developer Relations at Postman, details the subsequent steps required for production-ready agents, emphasizing consistency, service availability, and the critical role of observability tools like Langfuse for tracking performance and costs.

## Detailed Analysis

Quinton Wall outlines four critical requirements for building reliable AI agents that can run and scale effectively in production. First, establishing a clean contract between the agent and its services via API specifications is fundamental. Second, ensuring the availability of these external APIs and services is necessary, which he suggests monitoring with alerts, potentially through Postman. Third, responses must be consistent; the speaker demonstrates using Langfuse scoring to ensure that prompts consistently yield similar or identical results. Fourth, robust AI Agent Observability is required to track token usage and performance metrics like latency and cost. Wall shows how to configure the agent code to use specific models (like 'gpt-4') and how Langfuse automatically infers and reports usage and cost data. After running the agent locally and prompting it for Python courses, he switches to the Langfuse dashboard, showing that the custom scores (response length, language presence, course information presence) are being tracked, confirming the observability setup is working correctly and demonstrating that GPT-4 usage is significantly cheaper than GPT-3.5 for the task.

### Four Pillars of Reliable Agents

- 1. Creating a clean contract between API and services
- 2. Making sure API and services are available
- 3. Ensuring response consistency
- 4. AI Agent Observability

### Code Walkthrough & Langfuse Integration

- Agent is defined using the 'Agent' tool and configured with instructions for a 'CodingTeacher' role
- Agent uses the 'get_coding_resources' tool to query courses
- Langfuse observability is enabled via the `@observe` decorator set to 'generation' type (00:33)

### Observability Metrics & Scoring

- Custom scores are sent to Langfuse for context-based scoring, including response length, language reference presence, and course information presence (02:01)

### Cost and Performance Validation

- Running the agent locally shows consistent 100% relevance scores for the defined metrics
- The Langfuse dashboard confirms low total cost ($0.00065) when using GPT-4 compared to GPT-3.5, highlighting token usage tracking (05:11)

![Screenshot at 00:01: Quinton Wall introduces Part 2 of the series focusing on building reliable AI agents.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-00-01.png)
![Screenshot at 00:22: The Postman logo appears during the transition into the coding demonstration section.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-00-22.png)
![Screenshot at 00:27: A detailed view of the Python code in VS Code, showing imports for agents, Langfuse, and the main execution logic.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-00-27.png)
![Screenshot at 01:51: The code snippet highlighting the addition of the @observe\(type='generation'\) decorator to the agent function.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-01-51.png)
![Screenshot at 02:21: The terminal output showing the agent initializing and prompting the user for the programming language.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-02-21.png)
![Screenshot at 03:21: The Langfuse dashboard displaying various metrics charts, including Top Traces and Latency observations.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-03-21.png)
![Screenshot at 04:42: Close-up on the code showing the explicit setting of 'model="gpt-4"' within the generation update call.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-04-42.png)
![Screenshot at 05:07: The speaker navigates the Langfuse console to the 'Dashboards' section, specifically looking at the 'Langfuse Cost Dashboard'.](https://ss.rapidrecap.app/screens/mTnXWRmZVlQ/00-05-07.png)