Kernel Forge: LLM Agent for CUDA Kernel Optimization
Kernel Forge is an open-source, end-to-end agentic harness that uses large language models (LLMs) to generate and optimize CUDA kernels for PyTorch models. It accepts unmodified PyTorch models and supports vision, diffusion, and LLM workloads. The system addresses limitations of existing tools, which are often evaluated on random tensors, emit standalone code requiring manual reintegration, and primarily target LLM PyTorch models. Kernel Forge provides support for inspecting and debugging results, aiming to reduce the need for expert engineers to hand-write low-level GPU code.
Key facts
- Kernel Forge is an open-source agentic harness for LLM-based generation and optimization of CUDA kernels.
- It accepts any unmodified PyTorch model in place.
- Supports vision, diffusion, and LLM workloads.
- Uses Monte Carlo methods for optimization.
- Existing tools are evaluated on randomly generated tensors and isolated kernels.
- Existing tools emit standalone CUDA code requiring manual reintegration.
- Existing tools mostly target only LLM PyTorch models.
- Kernel Forge offers support for inspecting and debugging results.
Entities
—