# Gemini 2.5 Computer Use: BEATS Claude SONNET 4.5 & OpenAI!

Source: https://www.youtube.com/watch?v=PsSYjGchC3Q
Recap page: https://rapidrecap.app/video/PsSYjGchC3Q
Generated: 2025-10-08T00:02:02.73+00:00

---
## Quick Overview

Gemini 2.5 Computer Use model significantly outperforms leading alternatives like Claude Sonnet 4.5 and OpenAI's Computer Using Agent model across multiple benchmarks, achieving top scores in Online-MindWeb (69.0% on the official leaderboard) and WebVoyager (88.9% self-reported), while also delivering leading quality at the lowest latency in the Latency vs Quality chart for Online-MindWeb.

**Key Points:**
- Gemini 2.5 Computer Use achieved 69.0% on the official Online-MindWeb leaderboard, surpassing competitors.
- The model scored 88.9% self-reported on WebVoyager, significantly beating OpenAI's 80.0% score.
- In the Latency vs Quality chart for Online-MindWeb, Gemini 2.5 Computer Use sits in the quadrant indicating high accuracy and the lowest latency compared to other models.
- The Computer Use model operates via an agent loop involving sending requests to the model, receiving responses, executing actions in a client application (like a browser), and capturing the new environment state.
- The video demonstrates the agent successfully completing a multi-step task: navigating to a TinyURL sheet, filtering for California residents, and extracting all required pet information in three turns.
- To implement, users must install 'google-genai' and 'playwright', and set the GEMINI_API_KEY environment variable.

![Screenshot at 0:06: The agent successfully organizes chaotic sticky notes into three correct columns \(Promotion, Setup, Volunteers\) on the sticky-note-jam.web.app interface, demonstrating the core functionality of the Computer Use model.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-06.png)

**Context:** This video introduces and demonstrates the capabilities of Google DeepMind's Gemini 2.5 Computer Use model, a specialized model designed to power agents that interact directly with graphical user interfaces (GUIs) like web browsers, going beyond traditional structured API interactions. The presenter walks through the model's core workflow, showcases benchmark performance against competitors like Claude Sonnet 4.5 and OpenAI models, and provides a live Python code demonstration of an agent organizing chaotic sticky notes on a web application.

## Detailed Analysis

Google DeepMind introduced the Gemini 2.5 Computer Use model, which is available in preview via the API, designed to enable AI agents to interact with GUIs by clicking, typing, scrolling, and filling forms. The model's core capabilities are exposed through the 'computer_use' tool in the Gemini API, operating within a loop that includes sending requests with screenshots, receiving model responses (which suggest UI actions like 'click_at' or 'type_text'), executing those actions using a client application (like Playwright in the demo), and capturing the new environment state to repeat the cycle. Benchmark performance shows Gemini 2.5 Computer Use outperforming leading alternatives: it achieves 69.0% on the official Online-MindWeb leaderboard, compared to OpenAI's 61.3% (Operator). On WebVoyager, it scores 88.9% (self-reported) versus 80.0% for OpenAI. Furthermore, a Latency vs Quality chart highlights that Gemini 2.5 Computer Use offers leading quality (accuracy) at the lowest latency for browser control tasks compared to Claude Sonnet 4.5, Claude Sonnet 4, and OpenAI's model. The demonstration involves an agent organizing tasks on a sticky note board, successfully moving tasks like 'Print volunteer waiver forms' and 'Post about the event on social media' into their correct 'Promotion', 'Setup', or 'Volunteers' columns. A second demonstration successfully extracts pet information for California residents from a Google Sheet after navigating to the link provided in the prompt. The implementation requires installing 'google-genai' and 'playwright' libraries and setting the GEMINI_API_KEY.

### Gemini 2.5 Computer Use Benchmarks

- Online-MindWeb Official Leaderboard hit 69.0%
- WebVoyager self-reported score reached 88.9%
- Outperformed Claude Sonnet 4.5 and OpenAI Computer Using Agent Model across listed benchmarks

### Computer Use Model Flow

- The process cycles through 1. Send request to model (with screenshot/context)
- 2. Receive model response (function_call)
- 3. Execute received action in client app
- 4. Capture new environment state

### Implementation Requirements

- Install necessary packages via 'pip install google-genai playwright' and 'playwright install chromium'
- Set the GEMINI_API_KEY environment variable

### Sticky Note Organization Task

- Agent successfully categorized scattered tasks into 'Promotion', 'Setup', and 'Volunteers' columns using drag-and-drop actions in about 5 seconds per move

### Google Sheet Data Extraction Task

- Agent navigated to a TinyURL sheet, identified the California resident (Onkar Samoyed), and extracted all required details including Pet Name, DOB, Phone Number, and Treatment Requested

### Latency vs Quality Performance

- Gemini 2.5 Computer Use achieved the lowest latency while maintaining high accuracy compared to competitors on the Browserbase harness for Online-MindWeb

![Screenshot at 0:06: The agent is actively dragging a sticky note from an unorganized area into the 'Volunteers' column on the sticky-note-jam.web.app board.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-06.png)
![Screenshot at 0:08: The terminal shows the initial goal: organizing chaotic sticky notes into categories on the specified web application.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-08.png)
![Screenshot at 0:17: The terminal displays the execution log, showing steps like 'open\_web\_browser' and multiple 'drag\_and\_drop' actions being executed over several turns.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-17.png)
![Screenshot at 0:36: The Gemini 2.5 Computer Use Model Flow diagram illustrates the iterative loop involving the Model, Screenshot/Context, and Computer Environment \(client app\).](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-36.png)
![Screenshot at 0:38: Benchmark Performance table highlighting Gemini 2.5 Computer Use scores \(e.g., 79.9% on WebVoyager measured by Browserbase\) against Claude Sonnet 4.5 and OpenAI.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-38.png)
![Screenshot at 0:43: The Latency vs Quality scatter plot shows Gemini 2.5 Computer Use positioned for the best combination of high accuracy \(right\) and low latency \(bottom\).](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-00-43.png)
![Screenshot at 1:56: The Gemini API documentation is shown, outlining the four steps of the 'How Computer Use works' agent loop.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-01-56.png)
![Screenshot at 2:15: Code snippet showing the required installation commands: 'pip install google-genai playwright' and 'playwright install chromium'.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-02-15.png)
![Screenshot at 2:44: The Python code defining the USER\_PROMPT for the second task, instructing the agent to find details for pets with California residency from a TinyURL link.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-02-44.png)
![Screenshot at 2:57: The agent opens the computer terminal within the development environment, preparing to run the orchestration script.](https://ss.rapidrecap.app/screens/PsSYjGchC3Q/00-02-57.png)