This AI Model Runs On Your Phone (With No Internet)!

Quick Overview

The video demonstrates that the Locally AI application allows users to run powerful AI models, like the Qwen 3.5 series, directly on an iPhone without an internet connection, showcasing its capability to perform complex reasoning tasks like counting letters in a word and providing detailed advice offline.

Key Points: Locally AI enables running AI models locally on an iPhone, functioning entirely offline without needing to connect to cloud services like OpenAI or Anthropic (0:04, 4:26). The demonstration uses the Qwen 3.5 (2B) model, which runs quickly on the device, as shown by its ability to correctly count the 'R's in 'strawberry' by breaking down the word letter by letter (4:06). The app supports various models including Apple Foundation, Gemma 2, Qwen 3, Llama 3.2, and others, allowing users to select based on their device's capabilities (2:11, 2:56). The Qwen 3.5 Small Model Series includes 0.8B, 2B, 4B, and 9B parameter versions, with the 4B model being recommended for iPhone 15 Pro and newer due to its high CPU usage (0:53, 2:53). The app offers personalization settings, allowing users to customize responses via custom instructions and adjust the 'Temperature' setting to control creativity versus determinism (3:39). The presenter successfully used the app offline to generate detailed advice on handling a child's tantrum after taking away their iPad, confirming the model's reasoning capability without external data access (8:43).

Context: The video explores the capabilities of the Locally AI application, a tool designed to run large language models (LLMs) directly on mobile devices like the iPhone, thus enabling powerful AI interactions without reliance on internet connectivity. The presenter references a recent announcement regarding the Qwen 3.5 Small Model Series, highlighting its performance benchmarks against other models like GPT-4, and proceeds to demonstrate the app's functionality using a downloaded Qwen model.

Detailed Analysis

The video confirms that the Locally AI app allows users to execute advanced AI models, such as the Qwen 3.5 series, entirely on an iPhone without any internet connection, contrasting sharply with cloud-based services like OpenAI or Anthropic. The presenter first shows a tweet from Adrien Grondin demonstrating Qwen 3.5 running on an iPhone 14 Pro, optimized using MLX for Apple Silicon. The demonstration then moves to the Locally AI app itself, which offers a selection of models to download, including Apple Foundation, Gemma 2, Qwen 3, and Llama 3.2, noting that larger models require newer hardware. The presenter downloads the Qwen 3.5 (2B) model (taking about 5 minutes on Wi-Fi) and tests its capabilities offline. The initial test involves asking the model to count the 'R's in 'strawberry,' which it solves by correctly breaking down the word step-by-step (4:06). Next, the presenter tests its reasoning by asking for advice on calming a child having a tantrum after their iPad was taken away; the model provides a structured, multi-step guide focusing on de-escalation, validation, and setting consequences, performing well even without internet access (8:43). The presenter also briefly explores settings like personalization (custom instructions and temperature) and the voice mode feature, concluding that the ability to run such capable models locally is a significant advancement, surpassing previous benchmarks from models like GPT-4 a year and a half prior.