CUDAnalyst: A Framework for Attribution in Self-Evolving LLM Agents for CUDA Kernel Generation

A recent study published on arXiv presents CUDAnalyst, a comprehensive analysis framework designed for the controlled attribution of planning decisions to feedback elements in self-evolving LLM agents focused on CUDA kernel generation. This research tackles the challenges of understanding how planning decisions integrate and attribute diverse feedback signals over generations. Traditional end-to-end ablation methods are inadequate, as iterative planning can magnify initial disturbances and mix feedback influences with trajectory-dependent variations. CUDAnalyst employs techniques like trajectory freezing and selective feedback injection to facilitate stable evaluations at the generation level and offers a principled approach to coalitional attribution. Findings indicate that explicit planning is advantageous only when feedback is consistent, with effective planning arising from organized multi-feedback interactions.