LLM Agents as Greedy Optimizers in Hardware-Aware Code Optimization
A new study on arXiv (2605.19782) investigates the behavior of LLM agents in hardware-aware code optimization. Through three controlled experiments, researchers found that in pure black-box optimization, LLMs act as greedy optimizers. In zero-shot kernel generation, providing explicit input-size information had no measurable effect, with models converging to the same kernel parameters regardless of size or temperature. Performance sharply degraded when optimizing for uncommon kernel sizes, irrespective of language used. The study highlights limitations in LLM-based optimization systems.
Key facts
- arXiv paper 2605.19782 examines LLM agents in hardware-aware code optimization.
- Three controlled experiments were conducted.
- LLMs act as greedy optimizers in pure black-box optimization.
- Explicit input-size information had no effect in zero-shot kernel generation.
- Models converged to same kernel parameters regardless of size or temperature.
- Performance degraded for uncommon kernel sizes across languages.
Entities
Institutions
- arXiv