# Kelix Technical Report: Closing the Understanding Gap of Discrete Tokens in Unified MM Models Source: https://www.youtube.com/watch?v=uRiW3CqA9Cc Recap page: https://rapidrecap.app/video/uRiW3CqA9Cc Generated: 2026-02-14T17:04:31.031+00:00 --- ## Quick Overview The Kelix architecture effectively closes the understanding gap between discrete tokens (like text) and continuous signals (like images) by treating visual data as a stream of continuous signals, which is then processed by a Vision Transformer (ViT) and encoded using product quantization, achieving superior OCR performance (86.7 score) compared to models treating images as discrete tokens. **Key Points:** - Kelix addresses the understanding gap in multimodal models by treating images as continuous signals rather than discrete tokens. - The architecture uses a Vision Transformer (ViT) to scan the image and convert it into dense, continuous numerical values. - Product quantization is used to compress this continuous data into discrete tokens, allowing the LLM to process it efficiently. - The resulting model scored 86.7 on the OCR benchmark, significantly outperforming discrete models that scored 73.4. - The modular system separates the tokenization (ViT + Quantizer) from the LLM brain, allowing for easy upgrades, such as swapping in a better tokenzer next year. - The model successfully handles complex tasks like counting items (three red roses and one daisy) and describing images with fine detail, unlike simpler models. - The overall goal is to make images 'speak' the same language as text, achieving a unified understanding across modalities. ![Screenshot at 0:05: The presentation begins by introducing the Kelix technical report from Quaxu Technology, which tackles the fundamental structural problem in multimodal AI regarding discrete tokens.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-00-05.jpg) **Context:** This video discusses a technical report from Quaxu Technology introducing a new multimodal AI model architecture called Kelix, designed to bridge the conceptual gap between discrete tokens, commonly used for text, and continuous signals, which characterize visual data. The conventional approach of treating images as discrete tokens (like words) causes information loss, which Kelix aims to resolve through a hybrid architecture involving a Vision Transformer (ViT) and product quantization. ## Detailed Analysis The Kelix technical report introduces a novel architecture to close the understanding gap that arises when multimodal models treat images as discrete tokens, similar to words. Traditional models like GPT-4o treat images as sequences of tokens, leading to information loss, especially with fine details. Kelix adopts a hybrid architecture where the image is first processed by a Vision Transformer (ViT) to convert it into dense, continuous numerical values, treating the image like a continuous signal (analogous to a vinyl record groove). This continuous representation is then compressed using product quantization into discrete tokens, which the LLM can process. This approach results in a massive information fidelity improvement; Kelix scores 86.7 on an OCR benchmark, significantly beating the previous best discrete model (X-Comni) score of 73.4. The modular design separates the tokenization (ViT and quantizer) from the LLM's 'brain,' allowing for future upgrades to components without retraining the entire system. The model successfully performs complex tasks like counting objects (e.g., three red roses and one daisy) and retaining fine detail, which older discrete models often fail at. The fundamental goal is to enable the model to generate images and text using the same underlying language representation, effectively making images 'speak' the same language as text. ### Kelix Architecture Overview - Treats images as continuous signals processed by a ViT - Compresses continuous data via product quantization into discrete tokens for the LLM - Modular system allows swapping components like the tokeniser or LLM independently ### Performance Benchmarks - OCR benchmark score of 86.7 - Significantly outperforms discrete models (e.g., X-Comni score of 73.4) - Demonstrates high fidelity in retaining fine detail, like counting specific objects ### Comparison to Discrete Models - Discrete models fail at fine detail (e.g., counting whiskers or reading small text) - Discrete models are essentially 'blind' to fine details - Kelix avoids the 'context explosion' problem of overly long sequences by using quantization ### Modular Components - Three parts: Tokenizer (ViT + Quantizer) - LLM (Brain) - Decoder (Diffusion Renderer) - The encoder/decoder parts are trained separately from the LLM's brain ![Screenshot at 0:12: Introduction of the Kelix model, designed to address the structural problem in multimodal AI by bridging the gap between discrete text tokens and continuous visual data.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-00-12.jpg) ![Screenshot at 0:40: The speaker begins breaking down the architecture, contrasting the discrete token approach \(like GPT-4o\) with Kelix's hybrid approach.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-00-40.jpg) ![Screenshot at 1:05: Visual explanation of how Kelix uses a Vision Transformer \(ViT\) to scan the image and convert it into continuous numerical values, treating it like an analog signal.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-01-05.jpg) ![Screenshot at 2:13: Summary of the unified goal: enabling the model to predict the entire block of image tokens simultaneously, rather than one token at a time.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-02-13.jpg) ![Screenshot at 4:48: Visual representation of the modular system, highlighting the separation between the Tokenizer, the LLM Brain, and the Decoder.](https://ss.rapidrecap.app/screens/uRiW3CqA9Cc/00-04-48.jpg)