tl;dr: We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a diffusers pipeline and a transformers model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end.
It has been revealed that writing CUDA kernels is hard. Writing CUDA kernels that correctly integrate with transformers and diffusers is harder. There are architecture-specific memory access patterns, vectorization strategies, warp shuffle reductions, and a dozen integration pitfalls that trip up even experienced developers. It is exactly the kind of specialized, high-stakes problem where agent skills shine.
It has been revealed that we gave coding agents the domain knowledge they need, like which GPU architecture to target, how to structure a kernel-builder project, when to use shared memory versus registers, and how to write PyTorch bindings. The agents did the rest. If you have used the LLM training skill or read We Got Claude to Teach Open Models, the pattern will feel familiar: package domain expertise into a skill, point the agent at a problem, and let it work.
The Kernel Hub solved the distribution of custom hardware kernels. You can load pre-compiled kernels from the Hub with a single get_kernel call. No builds, no flags. However, someone still needs to write the kernels. That is the gap this skill fills.
CUDA kernel development has a brutal surface area:.
Sources indicate that this is domain knowledge that gets lost in documentation tabs and Stack Overflow answers. An agent skill packages it into context that loads on demand.
First, let's show how to use the skill right away, then we'll dive into the details of how we benchmarked the kernels.
The skill ships with the kernels library. Install it into your coding agent with a single command:.
In a recent development, this drops the skill into .claude/skills/cuda-kernels/ where Claude Code and Cursor pick it up automatically. For other agents:.
Once installed, prompt your agent:.
Or, you can go for something more open-ended:.
The agent can read the skill, select the right architecture parameters, generate the CUDA source, write the PyTorch bindings, set up build.toml, and create a benchmark script.
If you're working on more complex kernels, or architecture-specific optimizations, that aren't covered in the skill, then the skill supplies the fundamental building blocks and patterns to get you started. We are also open to contributions on the skill itself.
According to recent reports, the skill is roughly 550 tokens of structured guidance plus reference scripts, GPU optimization guides, troubleshooting docs, and complete working examples. Agentic coding tools like Codex and Claude can read this and produce a working kernel project.
Industry observers note that when an agent loads this, it gets everything it needs to go from "write me an RMSNorm kernel" to a buildable, benchmarkable project. It will grep and glob the skill to find the relevant files and directories. So it's crucial to structure the skill in a way that is easy to find.
The agent is instructed to generate kernels that conform to the templates in references/kernel-templates.md and produce a complete kernel project:.
We tested this on two real targets.
It has been revealed that the agent built RMSNorm, RoPE 3D, GEGLU, and AdaLN kernels for LTX-Video, a video generation pipeline from diffusers. The full example is at examples/ltx_video/. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.
If you want to check out the generated kernel, got to this example.
Sources indicate that first, we compare the isolated RMSNorm kernel performance against the PyTorch baseline. This is the main speedup in the optimized pipeline.
Average speedup: 1.88x and a bandwidth efficiency: 34.7% of H100 theoretical (3,350 GB/s).
Next, we compare the end-to-end video generation performance of the optimized kernels against the baseline (no compile) and the torch.compile baseline.
RMSNorm accounts for ~5% of total compute in LTX-Video. The remaining time is spent in attention, linear projections, and VAE decode. The 6% end-to-end speedup from a single kernel type is consistent with that profile.
According to recent reports, the agent built an RMSNorm kernel for Qwen3-8B, a large language model from transformers with 65 RMSNorm modules across 32 layers. The full example is at examples/qwen3_8b/. We optimized the RMSNorm kernel for H100. Both benchmarks were run on H100 80GB HBM3 at precision BFloat16.
If you want to explore the kernel, check it out here.
Once again, we compare the isolated RMSNorm kernel performance against the PyTorch baseline.
Average speedup: 1.94x and a bandwidth efficiency: 22.3% of H100 theoretical (3,350 GB/s).
According to recent reports, speedup scales with sequence length: 1.58x at 128 tokens, 2.47x at 8192 tokens. For long-context inference, the custom kernel roughly halves RMSNorm latency.
The agent gives you a working kernel. The Kernel Hub lets you share it so anyone can load it without compilation. Here is the full path from agent output to published kernel.
The agent produces a project that already follows the kernel-builder layout:.
Recent findings suggest that the build.toml tells kernel-builder what to build. The agent generates this for you, including the correct cuda-capabilities for your target GPU:.
Kernel Hub kernels must support all recent PyTorch and CUDA configurations. The kernel-builder Nix flake handles this automatically. Copy the example flake.nix into your project and run:.
Industry observers note that this builds the kernel for every required PyTorch/CUDA variant and places the results in build/. For faster builds, enable the HuggingFace Nix cache:.
Create a model repo on the Hub and upload the built kernel:.
Once published, anyone can use your kernel with zero compilation:.
In a recent development, get_kernel detects the user's Python, PyTorch, and CUDA versions and downloads the matching pre-compiled binary. No builds, no flags, typically ready in seconds.
Industry observers note that the skill and the Hub are complementary. The skill handles development. The Hub handles distribution. Build a kernel with the skill, validate it with the benchmark scripts, publish it to the Hub, and it becomes a one-liner for everyone else.
We built an agent skill that teaches coding agents how to write production CUDA kernels. Then we pointed Claude and Codex at two real targets: a diffusers pipeline and a transformers model. The agents produced working kernels for both, with correct PyTorch bindings and benchmarks, end to end. We benchmarked the kernels and found that the optimized kernels can provide a speedup in both isolated and end-to-end performance.
I get an error while trying to install:.
Installation with pip install kernels works, but the command has no skills option:.
The kernels repo has kernels-builder and the kernels library.
This is the reason why doing a pip install git+https://github.com/huggingface/kernels.git would not work.
Question: Is the baseline in section Isolated RMSNorm benchmark (Pytorch baseline) using torch.compile() or not? Custom kernels should be ideally beating torch.compiled code.
