Hugging Face: We got Claude to teach open models how to write CUDA kernels!

Quick Overview

Researchers successfully adapted a large language model, Claude Opus, to teach smaller, local models how to write complex CUDA kernels, achieving a 35% performance lift over the baseline model by distilling expert knowledge.

Key Points: The research, released January 28, 2026, details a method where the large model, Claude Opus, teaches smaller models to write CUDA kernels. The technique involves the larger model interacting with the prompt, correcting outputs, pasting documentation bits, and guiding the smaller model through a three-step workflow. The smaller model, UNSLOTH GLM 4.5 Flash, running locally, achieved an 85% success rate on CUDA tasks after training, compared to a 50% baseline. This process resulted in a 35% performance lift for the small model compared to the baseline model running the same task. The method avoids fine-tuning the large model, instead treating the expert model (Opus) as a teacher that distills its knowledge into a reusable skill package. The success of this approach suggests a future where high-level AI performance can be ported to smaller, more accessible hardware, like the Nvidia H100. The authors suggest this method is more efficient than full fine-tuning and reduces the risk of vendor lock-in by creating a portable skill file.

Context: This video discusses a significant breakthrough in AI knowledge transfer, where a research team leveraged a highly capable, proprietary model (Claude Opus) to teach smaller, locally runnable models how to perform complex, low-level programming tasks, specifically writing CUDA kernels for GPUs. This process, dubbed 'Agent Skills,' aims to democratize high-level AI capabilities by distilling expert knowledge into portable skill files, bypassing the massive cost and complexity of retraining huge foundation models.

Detailed Analysis

The video explains a new methodology for transferring specialized knowledge from large, powerful AI models to smaller, more accessible ones, focusing on the ability to write CUDA kernels. This research, released on January 28, 2026, by Ben Burtonshaw, Sean Smith, Merve, and Pedro Quinca, addresses the difficulty of high-precision programming tasks like writing CUDA kernels, which even large models like Claude Opus struggle with without specific guidance. The core idea is to create 'Agent Skills'—a set of files (like a skill.md and skill.json) that encapsulate expert knowledge. The process involves three steps: first, getting the teacher model (Claude Opus) to generate the raw material (the trace/log of the solution); second, using the skill tool to automate the process of distilling this trace into a transferable asset; and third, transferring this asset to the smaller model. When testing this method on a small, locally runnable model (UNSLOTH GLM 4.5 Flash), the performance on CUDA tasks jumped from a 50% pass rate (when simply asked) to an 85% pass rate when using the distilled skill file. This represents a 35% lift over the baseline model's performance. The key benefit is that this process is far more efficient than retraining the large model, reduces latency and cost, and prevents vendor lock-in by creating a portable file. The researchers suggest this trend points toward a future where complex AI capabilities are modular and accessible, rather than tied to massive, expensive foundation models.