KernelArc: A Multi-Agent Framework for GPU Kernel Optimization

KernelArc introduces a multi‑agent system that autonomously tunes GPU kernels for diverse workloads. Instead of relying on a single optimizer, it runs several strategy‑focused agents in parallel, letting them exchange only the results of their benchmarks. The approach was tested on NVIDIA’s H100 and B200 GPUs using the SOL‑ExecBench suite, producing a set of highly tuned kernels that appear on the public leaderboard.

What You Need to Know

Each agent specializes in a class of optimizations—tile size, loop unrolling, precision scaling, memory‑access patterns, and so on. They work independently but share a conclusions‑only memory space, meaning they write only the final benchmark score of a tried configuration, not the intermediate code or parameters. This keeps coordination lightweight and avoids interference between agents.

The framework includes a deterministic benchmark guard that fixes random seeds, disables asynchronous migrations, and uses a set number of warm‑up and measurement iterations to guarantee repeatable timings despite GPU noise. When an agent’s improvement stalls (its gain falls below a threshold for N consecutive steps), a plateau‑triggered drafting phase copies read‑only snapshots of the best states from all agents into a shared buffer, allowing other agents to start new searches from those points.

Evaluated kernels include a custom BF16 GEMM, static cuBLASLt Expert‑API configuration tables, a fused mixture‑of‑experts backward pass, shape‑gated decoder‑layer fusion, native NVFP4 grouped‑query attention, and paged prefill attention. Each kernel was generated by the framework without manual tuning, demonstrating that the agent‑based search can produce competitive implementations across a range of operations.

Why It Matters📌 Source: Arxiv Ai