Advice on prompt-engineering with AI agents | Peter Steinberger and Lex Fridman

Quick Overview

The conversation between Lex Fridman and Peter Steinberger explores the 'Curve of Agentic Programming,' illustrating how the complexity of prompt engineering and agent orchestration rises sharply before leveling off as agents become more capable of handling complex tasks with simpler instructions. Steinberger notes that early interactions involve simple prompts like "ple fix this," complexity peaks with orchestration of 8 agents, custom workflows, and 18 slash commands, and eventually simplifies again to high-level instructions like "Hey, look at these files [...] and then do [med size change]."

Key Points: The 'Curve of Agentic Programming' shows complexity peaking when orchestrating multiple specialized agents (e.g., 8 agents at once) with complex workflows and tool usage (like 18 different slash commands). The complexity decreases after the peak, leading to simpler, higher-level prompts like 'Hey, look at these files [...] and then do [med size change]' when agents achieve greater capability. Steinberger shares his past reliance on writing very long, bespoke prompts, contrasting with the current trend of using voice commands for terminal operations. The difficulty lies in the complexity of orchestration, not necessarily the individual LLM's programming ability; the system needs to be built to empathize with the agent's perspective. A major challenge is the 'waterfall model' approach to software development, where agents are expected to handle the whole process, which often fails if the underlying agent is not good. The speaker suggests that building empathy for the AI agent's required context is crucial for effective prompt engineering, as agents often start from 'nothing' or a bad codebase default. The speaker mentions his personal development process involved spending an 'ungodly time' over the year playing with and iterating on his prompts and orchestration layers to improve agent performance.

![Screenshot at 00:04: The 'Curve of Agentic Programming' graph is displayed, showing complexity rising steeply from simple prompts like 'ple fix this' to a peak involving complex orchestration, before declining towards high-level instructions like 'Hey, look at these files \[...\] and then do \[med size change\]'.](https://ss.rapidrecap.app/screens/BuvYFWrHWQ/00-00-04.jpg)