High-Efficiency Diffusion Models for On-Device Image Generation and Editing [Hung Bui] - 753

Quick Overview

Hung Bui, Qualcomm's VP of Technology, explains that their strategy for efficient on-device AI, particularly for image generation and editing, involves distilling knowledge from large models into smaller, highly efficient models (like their 7-billion parameter model) through distillation techniques, allowing them to perform nearly as well as larger models while running locally on mobile devices, thus addressing privacy and latency concerns.

Key Points: Qualcomm recently acquired VinAI Research, which ranked in the top 25 industrial AI labs based on research output. The team successfully developed a 7-billion parameter model that performs even better than the 70-billion parameter model it was distilled from. A key focus is achieving high-quality image generation and editing using these much smaller models, aiming for performance comparable to larger models. The research focuses on efficiency, specifically enabling models to run directly on mobile devices to address privacy and latency issues. The process involves distilling knowledge from large teacher models (like the 70-billion parameter model) into smaller student models (like the 7-billion parameter model). The approach involves running the inference process on the device itself rather than relying on cloud computation. The research group in Vietnam, formerly Stanford Research Institute (SRI) International's Vietnam AI Center, has a history dating back to 2019.

Context: This interview on the TWiML AI Podcast features Sam Charrington speaking with Hung Bui, Technology Vice President at Qualcomm, who recently joined Qualcomm following the acquisition of VinAI Research. The discussion centers on Qualcomm's strategy for developing high-efficiency diffusion models capable of running generative AI tasks directly on mobile devices, contrasting this approach with the reliance on large cloud-based models.

Detailed Analysis

Hung Bui, VP of Technology at Qualcomm, discusses the strategic focus on creating highly efficient diffusion models for on-device image generation and editing, a focus spurred by Qualcomm's acquisition of VinAI Research. He notes that VinAI Research was highly ranked and that the team's work led to significant achievements, such as creating a 7-billion parameter model that outperforms a 70-billion parameter model it was distilled from. The core technical approach involves knowledge distillation, where a large teacher model trains a smaller student model to retain high performance while drastically reducing size and computational requirements. This efficiency is crucial because it allows these powerful models to run locally on mobile devices, addressing concerns around inference latency and data privacy, as sensitive data does not need to leave the device. Bui mentions that this approach allows for faster iteration cycles and better personalization compared to relying solely on cloud-based large language models (LLMs) like GPT-3. He emphasizes that the goal is to achieve the same or better quality outputs with far fewer parameters and faster inference times, making advanced AI accessible everywhere.