# FunctionGemma - Function Calling at the Edge

Source: https://www.youtube.com/watch?v=Zj_cvQKcH4g
Recap page: https://rapidrecap.app/video/Zj_cvQKcH4g
Generated: 2025-12-19T12:34:56.122+00:00

---
## Quick Overview

Google introduced FunctionGemma, a specialized, open-source version of the Gemma 3 270M model fine-tuned for function calling, enabling on-device execution of bespoke API actions like setting reminders or toggling settings, demonstrated by successfully scheduling a calendar event from a voice command.

**Key Points:**
- FunctionGemma is a specialized, open-source version of the Gemma 3 270M model, fine-tuned specifically for function calling (tool use) capabilities.
- The model is engineered for the edge, being small enough to run on devices like NVIDIA Jetson Nano and mobile phones, utilizing Gemma's 256k vocabulary for efficient tokenization of JSON and multilingual inputs.
- Fine-tuning FunctionGemma on the Mobile Actions dataset increased accuracy from a 58% baseline to 85% on that task, confirming its reliability for edge agents.
- The function calling workflow involves defining tools via JSON schema, the model issuing a structured function call based on user prompt, the application executing the tool, and returning the result to the model for a final user-facing response.
- The base Gemma model failed to generate a function call for scheduling a meeting, whereas the fine-tuned FunctionGemma successfully generated the correct `call:create_calendar_event` structure.
- The tutorial provides a complete Colab notebook demonstrating setup, training using Hugging Face TRL, conversion to the `.litertlm` format for deployment, and evaluation.

![Screenshot at 00:28: The introductory slide for FunctionGemma illustrates its role as an agent that translates natural language \(speech bubble icon\) into executable API actions \(represented by circuits connecting to icons for a house, calendar, and user interaction\).](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-00-28.png)

**Context:** This video introduces FunctionGemma, a specialized large language model (LLM) released by Google DeepMind, built upon the open-source Gemma 3 270M architecture. The primary focus is enabling LLMs to perform actions (function calling) directly on edge devices, moving beyond pure conversational interfaces to active agents capable of executing system tasks like managing calendars or controlling device settings.

## Detailed Analysis

The video details FunctionGemma, a specialized, open-source LLM based on Gemma 3 270M, specifically fine-tuned for function calling to enable on-device, edge deployment. The model was trained using a dataset featuring executable API actions, such as those found in the public Mobile Actions dataset, which includes tasks like turning a flashlight on/off, creating contacts, sending emails, showing locations on a map, opening Wi-Fi settings, and creating calendar events. The key benefit demonstrated is improved reliability; fine-tuning boosted accuracy from 58% to 85% on the Mobile Actions evaluation. The core workflow involves a four-stage cycle: the Developer defines available tools via JSON schema, the Model receives the prompt and tool definitions and outputs a structured function call using special control tokens (e.g., `<start_function_call>`), the Application parses this call, executes the necessary logic (like calling an external API or local Android Intent), and returns the result to the model using a function response token structure. The video contrasts the base Gemma model, which failed to schedule a meeting, with the fine-tuned FunctionGemma, which correctly generated the required `call:create_calendar_event` structure. The presentation concludes by showing the steps in a Colab notebook for loading the model, processing the dataset, performing Supervised Fine-Tuning (SFT) using the TRL library, visualizing the loss reduction (showing rapid convergence), and finally converting the model checkpoint to the `.litertlm` format for efficient on-device deployment using LiteRT.

### Introduction to FunctionGemma

- FunctionGemma is a specialized, fine-tuned version of Gemma 3 270M
- trained for function calling (tool use)
- designed for on-device, edge deployment

### Function Calling Workflow

- A four-stage cycle involving Tool Definition (Developer), User Prompt, Model Issues Function Call (Model), Application Logic/Tool Output (Developer), and Final Answer (Model)
- uses specific control tokens to structure communication

### Training and Performance

- Fine-tuning on the Mobile Actions dataset improved reliability from 58% to 85% accuracy
- Base model failed on complex actions like scheduling events, while the fine-tuned model succeeded
- Training takes about 8 minutes per epoch on an A100 GPU using the TRL library

### Implementation Details

- Requires loading model via Hugging Face (gated access needed)
- Dataset is publicly available as `google/mobile-actions`
- Fine-tuned model can be converted to `.litertlm` format for mobile deployment

### Code Walkthrough

- Demonstrates setting up the environment, loading the model, processing the dataset into prompt/completion pairs, testing the base model (failure on calendar event), testing the fine-tuned model (success on calendar event), and the steps for training and conversion.

![Screenshot at 00:07: Google Gemma logo appearing with the text 'Gemma' below it.](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-00-07.png)
![Screenshot at 00:28: The introductory slide for FunctionGemma illustrates its role as an agent that translates natural language \(speech bubble icon\) into executable API actions \(represented by circuits connecting to icons for a house, calendar, and user interaction\).](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-00-28.png)
![Screenshot at 01:02: Sequence diagram showing the 5-turn flow for a successful function call, from User Prompt to Final Answer.](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-01-02.png)
![Screenshot at 04:00: Bar chart comparing FunctionGemma 270M Base accuracy \(~58%\) versus Fine-tuned FunctionGemma 270M accuracy \(~85%\) on the Mobile Actions task.](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-04-00.png)
![Screenshot at 06:19: Python code snippet showing the essential system prompt structure that activates the model's function calling logic, including the developer and user roles.](https://ss.rapidrecap.app/screens/Zj_cvQKcH4g/00-06-19.png)