Build Hour: Agent Memory Patterns

Quick Overview

Agent memory design centers on context engineering, which uses techniques like reshaping/fitting, isolating/routing, and extracting/retrieving information to manage finite context windows, preventing issues like context burst, conflict, poisoning, and noise, ultimately aiming for the smallest high-signal context to maximize desired outcomes.

Key Points: Context engineering is both an art (involving judgment) and a science (with concrete, measurable patterns) that dictates LLM performance beyond just model quality. The three core strategies for context management are: reshape and fit (trimming, compaction, summarization), isolate and route (offloading context to sub-agents), and extract and retrieve (memory extraction and retrieval). Failure modes in context management include context burst (sudden token spikes), context conflict (contradictory instructions), context poisoning (incorrect information propagation), and context noise (redundant tool definitions). Short-term memory involves in-session techniques like trimming (dropping older turns) and compaction (dropping older tool results) to maximize the context window. Long-term memory utilizes extraction and retrieval, demonstrated by summarizing prior sessions into a structured memory object that can be injected into future sessions for continuity. Best practices for prompting include being explicit, structured, giving room for planning, and avoiding conflicts by keeping tool sets small and non-overlapping. The demo showed that context trimming removes older turns/messages when a threshold (e.g., max turns S3) is hit, while summarization compresses prior messages into a structured memory object.

Context: This build hour session, hosted by Michaela with Emry from the solution architecture team, focused on "Agent Memory Patterns" as the latest topic following previous sessions on building agents from scratch and agent RFT. The session aimed to empower attendees with best practices and expertise to scale using OpenAI APIs, starting with an introduction to context engineering as the foundation for agent memory management.