# DSPy is (really) All You Need - Kevin Madura, AlixPartners Source: https://www.youtube.com/watch?v=-cKUW6n8hBU Recap page: https://rapidrecap.app/video/-cKUW6n8hBU Generated: 2026-01-08T21:31:57.816+00:00 --- ## Quick Overview Kevin Madura advocates for DSPy as a declarative framework that treats LLMs as first-class citizens, enabling users to build modular Python programs where logic is decomposed, allowing for quick iteration and optimization while remaining robust to model shifts. **Key Points:** - DSPy is a declarative framework for building modular software that treats LLMs as a first-class citizen, fundamentally calling a function that uses an LLM under the hood with input/output type guarantees. - DSPy sits at a higher level of abstraction than libraries like LangChain, allowing users to focus on declaring intent rather than managing low-level details like string parsing or message construction. - The core concepts of DSPy include Signatures (declaring intent, inputs, outputs), Modules (logical structure), Tools (exposed Python functions), Adapters (prompt formatters), Optimizers (quantitative performance improvement), and Metrics (defining success). - Optimizers are a key benefit, allowing users to quantitatively improve performance and enable model transferability, potentially matching or exceeding fine-tuning performance like GRPO in some situations. - Adapters, such as the built-in JSON adapter or the BAML adapter, control how inputs are formatted into prompts, where using BAML can sometimes improve performance by 5-10% over raw JSON. - DSPy natively supports multimodality (images, audio) via libraries like Attachments, allowing users to feed documents or images directly into signatures for RAG or analysis. - The framework allows users to inject existing good prompts into the docstring of a signature, meaning it does not negate the work of experienced prompt engineers. **Context:** Kevin Madura, a technical consultant at AlixPartners, presented DSPy, a framework designed to structure complex LLM applications by focusing on building proper Python programs rather than just iterating on prompts. He introduced DSPy to an audience where most had heard of it but few used it daily, aiming to demonstrate its utility across various consulting use cases, such as contract analysis and process improvement, emphasizing its value for rapid iteration and modularity. ## Detailed Analysis DSPy functions as a declarative framework that allows developers to decompose logic into a cohesive, modular Python program where Large Language Models (LLMs) are treated as fundamental components. Madura highlighted that DSPy avoids the low-level complexities often found in other frameworks, letting users declare intent via Signatures, which define typed inputs and outputs, deferring the implementation details to the LLM. Key primitives include Modules for structuring logic, Tools which are standard Python functions exposed to the LLM, and Adapters which handle the crucial translation between the structured signature and the final prompt string sent to the model; adapters allow for format customization, like switching from JSON to BAML for potential performance gains. The presentation heavily featured Optimizers, which quantitatively improve performance by iteratively tweaking prompts based on defined Metrics, offering a powerful alternative to traditional fine-tuning, especially for model transferability between models of varying sizes and costs. Madura demonstrated practical applications including simple sentiment classification, complex RAG over SEC filings using the Attachments library for multimodality, and building agents using React for tool calling, ultimately showing how these components assemble into robust data processing pipelines capable of handling diverse file types like contracts and images. ### DSPy Core Philosophy - Declarative framework for building modular software - Treats LLMs as first-class citizens - Focuses on building proper Python programs, not just prompt tweaking - Retains control flow despite rapid model capability shifts ### Key Primitives Explained - Signatures define typed intent (inputs/outputs) often using Pydantic objects - Modules logically structure the program, based on signatures - Tools are Python functions exposed to the LLM - Adapters format the prompt (e.g., JSON vs. BAML) ### Optimization and Measurement - Optimizers quantitatively improve performance iteratively, sometimes rivaling fine-tuning - Metrics define success for the optimizer - Optimization enables model transferability across cost/performance profiles - Omar (founder) stresses DSPy is programming abstractions first, optimization is an added benefit ### Module and Tool Usage - Modules wrap signatures and logic (e.g., time entry standardization) - Tool calling is implemented via modules like React, exposing Python functions (e.g., search web) to the LLM - Trajectory history provides visibility into tool calls and observations for debugging ### Adapters and Multimodality - Adapters format signatures into LLM messages; BAML adapter can improve performance over default JSON adapter - DSPy supports multimodality by default using libraries like Attachments to handle images/PDFs easily - Poor man's RAG example showed processing SEC Form 4 documents ### Code Walkthrough Highlights - Demonstrated using multiple LLMs (Claude, Gemini, GPT-4) with caching enabled by default - Showcased structured output by forcing output types via Pydantic literals - Usage information (token usage) is automatically tracked via late LLM integration