CUDA-L2: RL-Optimized HGEMM Kernels Outperform cuBLAS
The recently introduced system, CUDA-L2, outlined in arXiv paper 2512.02551, utilizes large language models (LLMs) alongside reinforcement learning (RL) to enhance Half-precision General Matrix Multiply (HGEMM) CUDA kernels automatically. By treating CUDA execution speed as the reward in RL, CUDA-L2 evaluates 1,000 configurations to identify the best kernel implementations. It consistently surpasses leading matrix multiplication benchmarks, such as torch.matmul and Nvidia's proprietary libraries cuBLAS and cuBLASLt. In offline mode, where kernels execute sequentially, CUDA-L2 records an average speedup of +22.0% over torch.matmul, +19.2% over optimally arranged cuBLAS (NN and TN), +16.8% over cuBLASLt-heuristic, and +11.4% over the highly competitive cuBLASLt-AutoTuning model. The paper's announcement as a replace-cross on arXiv signifies a revised edition. This advancement is crucial for high-performance computing, showcasing AI-driven optimization's ability to exceed manually tuned libraries and potentially transforming software optimization for hardware across various computational kernels.
Key facts
- CUDA-L2 combines LLMs and reinforcement learning to optimize HGEMM CUDA kernels.
- Uses CUDA execution speed as the RL reward.
- Optimizes kernels across 1,000 configurations.
- Outperforms torch.matmul by +22.0% on average in offline mode.
- Outperforms cuBLAS by +19.2% using optimal layout (NN and TN).
- Outperforms cuBLASLt-heuristic by +16.8%.
- Outperforms cuBLASLt-AutoTuning by +11.4%.
- Paper announced as replace-cross on arXiv.
Entities
Institutions
- arXiv
- Nvidia