# Llama.cpp Gets a New Web UI Source: https://www.youtube.com/watch?v=SWtIjKwIdfk Recap page: https://rapidrecap.app/video/SWtIjKwIdfk Generated: 2025-11-11T02:09:41.585+00:00 --- ## Quick Overview The Llama.cpp project has integrated a new, minimalist Web UI that allows users to run local LLMs, including multimodal models, with advanced customization options for context length, sampling, penalties, and developer settings, offering a significant upgrade over previous methods like relying solely on Ollama's web UI or command-line arguments. **Key Points:** - Llama.cpp now features a dedicated, minimalist Web UI accessible locally (e.g., at 127.0.0.1) for interacting with local LLMs. - The new UI supports multimodal inputs, allowing users to attach images, audio files, text files, and PDFs as context for queries (05:10). - Extensive customization is available through settings, including General (API key, system message, theme), Sampling (Temperature, Top-K, Top-P), Penalties (Repeat penalty, DRY multipliers), and Reasoning (Show thought process) (03:57). - The model used for testing the PDF context retrieval was Qwen1.5-NL-30B-433B, demonstrating context processing speed around 60 tokens/sec with a 22KB context (06:35). - The UI supports running multiple parallel chat sessions, allowing for side-by-side comparison of different models or prompts (08:54). - The developer settings allow enabling a model selector and showing raw LLM output, as well as defining a custom JSON schema for structured output extraction (09:55). - When querying the model about the o3-mini system card PDF, the model provided a detailed reasoning process before generating a concise answer (06:27). ![Screenshot at 00:14: The primary interface of the new llama.cpp Web UI is shown, featuring the model name, the input prompt area, and evidence of attached context files \(the 'gemma-2b-it' model is selected\).](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-00-14.png) **Context:** The video demonstrates the introduction and features of a new, dedicated Web User Interface (Web UI) for llama.cpp, a popular project for running large language models (LLMs) locally. This new interface aims to provide a richer, more powerful local LLM experience compared to previous methods like using Ollama's web UI or relying entirely on command-line tools. The presenter focuses on showcasing the UI's capabilities, particularly its support for multimodal context input and extensive configuration options, while also benchmarking performance on an Apple M2 Max machine. ## Detailed Analysis The video unveils the new Web UI for llama.cpp, which is designed to be minimalist yet highly functional for local LLM inference. The UI defaults to a dark theme and shows basic model information, context token count, and output speed. A major feature highlighted is multimodal input support (05:10), where users can attach images, audio, text, and PDFs to provide context for queries; for instance, a PDF named 'o3-mini-system-card.pdf' is attached and processed (05:32). Performance is benchmarked using an Apple M2 Max, showing generation speeds around 60 tokens/sec when processing a 22KB context (06:35). The settings menu is comprehensive, covering General, Sampling, Penalties, Reasoning, Import/Export, and Developer sections, allowing fine-grained control over model behavior, such as temperature, Top-K, and enabling structured JSON output via custom JSON schema definition (09:55). The presenter demonstrates running multiple chat sessions in parallel (08:54) and queries the model about the attached PDF, showing the model's internal reasoning process before producing a final answer (06:27). ### New Web UI Features - Minimalist dark theme interface - Local LLM execution - Support for parallel chat sessions ### Multimodal Context Support - Accepts Images (requires vision model) - Audio Files (requires audio models) - Text Files - PDF Files (PDFs converted to text) (05:10) ### Advanced Settings - General (API key, System Message, Theme) - Sampling (Temperature, Top-K, Top-P) - Penalties (Repeat penalty, DRY multipliers) - Reasoning (Show thought process) (03:57) ### Performance Benchmarking - Testing on Apple M2 Max - Generation speed around 58-60 tokens/sec for context-heavy prompts (06:35) ### Structured Output & Developer Tools - Ability to define custom JSON schema for structured output - Option to show raw LLM output in chat - Enable model selector (09:55) ### Demonstration & Comparison - Model o3-mini's performance is compared against GPT-4o and o1-mini across safety, reasoning, and multilingual benchmarks in the system card PDF (06:43) ![Screenshot at 00:14: The primary interface of the new llama.cpp Web UI is shown, featuring the model name, the input prompt area, and evidence of attached context files \(the 'gemma-2b-it' model is selected\).](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-00-14.png) ![Screenshot at 00:30: The 'llama.cpp' README detailing supported backends like Metal, BLAS, SYCL, and Vulkan, showcasing broad hardware compatibility.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-00-30.png) ![Screenshot at 00:38: The terminal showing the installation command for llama.cpp via Homebrew: 'brew install llama.cpp' on an M2 Mac \(1.213s context size\).](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-00-38.png) ![Screenshot at 02:08: Example command for starting the llama-server using the 'gpt-oss-20b-GGUF' model, specifying host and port.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-02-08.png) ![Screenshot at 03:07: ChatGPT's reasoning output is displayed, explaining its identity, capabilities, and limitations, contrasting with the local model's output.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-03-07.png) ![Screenshot at 04:26: The Sampling settings menu in the UI, showing extensive control over generation parameters like Temperature, Dynamic Temperature Range, Top-K, and Top-P.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-04-26.png) ![Screenshot at 05:32: The chat interface now shows two attached PDF files \('invoice-1.pdf' and 'invoice-2.pdf'\) ready for context-aware querying.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-05-32.png) ![Screenshot at 06:02: The GPU History monitor shows significant utilization spikes on the Apple M2 Max during model processing.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-06-02.png) ![Screenshot at 07:13: A table from the o3-mini system card comparing performance metrics \(Safety, Bias, Instruction Hierarchy\) against GPT-4o and o1-mini.](https://ss.rapidrecap.app/screens/SWtIjKwIdfk/00-07-13.png)