Building interactive UIs for Agent-powered apps with Generative UI

Quick Overview

The video demonstrates how to build interactive UIs for agent-powered applications using the Generative UI framework in Flutter by implementing a workout companion app that dynamically updates based on user input and agent responses, specifically by defining necessary data schemas, implementing stateful widgets to manage rep counts, and utilizing to signal exercise completion back to the agent.

Key Points: The demonstration builds upon a previous example by enhancing the workout tracking logic to handle user input for completed reps and exercise completion. The speaker updated the to include (Integer) and (A2uiSchemas.action) to manage user interaction and agent feedback. The widget was converted to a Stateful Widget to internally manage the count of completed reps using a variable initialized in . In , the code checks if the exercise name changed; if so, it resets to the new . The UI for tracking reps includes up/down arrows to manually adjust and a checkmark icon to trigger the callback, sending the final count back. The callback within the 's definition handles the by resolving context and dispatching a back to the agent, including the data. The demonstration concludes by showing the fully interactive workout flow, where the agent guides the user through exercises and congratulates them upon completion.

Context: This tutorial, titled "Flutter Flight Plans: Interactive UIs for Agent-powered apps with GenUI," is presented by Khanh Nguyen from the Flutter team. It focuses on advancing the integration of Generative UI (GenUI) with Flutter applications by handling user input directly within generated UI components (like the ). The session builds on prior work, showing how to update the data model and widget logic to allow users to manually adjust rep counts and signal completion, ensuring the agent stays aware of the dynamic UI state.

Detailed Analysis

The video explains how to make agent-generated UIs truly interactive, moving beyond static displays to allow users to provide feedback and update the state, which the agent then processes. The presenter first reviews the updated which now includes fields for (Integer) and (A2uiSchemas.action) to manage user interaction and agent callbacks for exercise completion. The presenter then modifies the widget, converting it to a to manage the local state of . In , this local state is initialized from the . The method is implemented to reset if the exercise name changes when the widget is reused. The method is updated to display the current reps completed, along with up/down arrow buttons to manually adjust this count (which calls ), and a checkmark button that triggers . This callback is crucial: it executes the defined in the GenUI data model, which involves resolving context and dispatching a back to the agent, explicitly including the data. This allows the agent to know exactly what the user did, fulfilling the requirement to wait for user confirmation before moving to the next step, as defined in the system prompt.