# Voice to Agents to Sora 2: Gemini Computer Use + Realtime API + Claude Code Source: https://www.youtube.com/watch?v=Ur3TJm0BckQ Recap page: https://rapidrecap.app/video/Ur3TJm0BckQ Generated: 2025-10-13T13:35:38.76+00:00 --- ## Quick Overview The demonstration successfully orchestrates a multi-agent system combining Claude code agents (Sony and Blink), Gemini 2.5 computer vision, and the OpenAI real-time API to build a functional Sora video generation application end-to-end, proving that combining models and providers ("think in ands, not ors") maximizes capability. **Key Points:** - The engineer established two Claude code agents, Sony and Blink, commanded by a voice orchestrator agent, to work on a unified project involving Sora API integration and front-end styling. - The winning strategy in generative AI is to "use them all," combining models like Gemini and OpenAI, and tools like Claude code next to OpenAI's real-time API, instead of optimizing for a single provider. - Multi-agent observability provided a live pulse of agent activities, allowing the engineer to see an agent validating its work with the new Gemini 2.5 computer use model inside a tool. - Agent Sony successfully completed its task by generating a full specification document for Sora video generation integration, which was then used by both agents for subsequent work via a custom slash command (/build). - Agent Blink implemented front-end changes, inverting colors to black text on a white background and applying the Play font, and later corrected interference from Sony by comprehensively reviewing both front-end and back-end components. - The system architecture includes an input layer (engineer/agents), a system layer leveraging the OpenAI real-time API and tools, and an output layer, emphasizing closed-loop design where outputs loop back into inputs for self-correction. - The engineer demonstrated both voice-based and text-based prompting workflows, asserting that voice coding is "right around the corner" for controlling fleets of agents. **Context:** The transcript details an experimental demonstration of a powerful multi-agent system designed to push the boundaries of autonomous software development, specifically focusing on integrating video generation capabilities using the anticipated Sora API. The setup involves an orchestrator voice agent controlling specialized agents, named Sony and Blink, which are built using Claude Code 2.5 and are capable of leveraging external models like Gemini 2.5 and APIs like OpenAI's real-time API to complete complex engineering tasks. ## Detailed Analysis The core value of the demonstration lies in showcasing a highly experimental, powerful multi-agent system where agents are designed to collaborate rather than operate in silos, emphasizing the philosophy to "think in ands, not ors" by combining Gemini, OpenAI, and Claude technologies. The process began by creating two Claude code agents, Sony and Blink, and tasking them: Sony to review documentation and plan Sora API integration, and Blink to handle front-end styling (black text on white background using Play font). Crucially, custom slash commands like /plan and /build provided reusable compute structures for the agents. The system incorporated multi-agent observability, which provided detailed, cheap summaries of every command and allowed real-time monitoring, including seeing an agent validate its work using the Gemini 2.5 computer use model in a closed-loop structure. When initial work caused conflicts (Sony interfering with Blink's UI), the orchestrator agent corrected the issue by re-tasking Blink to re-implement the front end while strictly maintaining boundaries. The final result was the agents successfully building out the front-end and back-end for a content generation platform, culminating in a successful end-to-end cat image generation via a newly created browser agent testing the localhost port 3333, even after the main voice orchestration agent crashed during a remix test. The presenter stresses that scaling impact requires scaling compute, and the engineer remains the bottleneck, advocating for advanced agentic engineering principles taught in the promoted 'Tactical Agentic Coding' course. ### Agent Creation and Initial Tasks - Two Claude code agents, Sony and Blink, were created - Sony was commanded to plan Sora API integration and save the plan to specs - Blink was commanded to invert front-end colors to black text on white background and apply the Play font. ### Multi-Agent System Philosophy - Loyalty to one model is a losing game because it optimizes for business models, not maximum capability - The winning play is to use them all, thinking in "ands, not ors" (e.g., Gemini models AND OpenAI models). ### Observability and Validation - Multi-agent observability provided a live pulse and cheap, fast model summaries of all agent activities - Agents validated their work using tools, including leveraging Gemini 2.5 computer use model for concrete feedback in a closed loop. ### Agent Execution and Custom Commands - Sony completed its specification plan using a custom /plan command - Both agents subsequently executed the build based on that specification using a custom /build command, demonstrating reusable compute structures. ### Conflict Resolution and Iteration - Agent Sony interfered with Blink's front-end work - Blink was re-tasked to comprehensively check both front-end and back-end to restore the initial minimalist theme while connecting to the functioning API. ### Final Application Deployment - Agents successfully built out the FastAPI backend (Sony) and front-end components (Blink) for the Sora video generator - A browser agent was spun up to test localhost port 3333, successfully generating a cat image using Sora 2 Pro in portrait mode for 8 seconds. ### Engineering Takeaways - The key idea is that the engineer is the bottleneck, and scaling impact requires scaling compute - The demonstration supports both voice-based and text-based workflows for agent orchestration.