VRAM Forecasting in Code-Synthesis Agents: Analytical Models vs. Learned Baselines

A recent paper on arXiv (2608.15117) investigates peak VRAM usage during LLM inference for code-synthesis agents. The research breaks down memory into weight-storage, KV-cache, and activation components, influenced by step count, tool invocations, and context expansion. Utilizing a LangGraph-based CUDA-kernel-synthesis agent (AgentK), a 4-bit quantization family (Q4_K_M), and a single NVIDIA H100 GPU, the study analyzes four LLM backbones across 1,920 trajectories. Notably, closed-form analytical models, when supplied with two empirical constants (loaded-weight VRAM and fixed activation-memory overhead), demonstrate competitive accuracy against learned baselines. For three of the four backbones, the closed-form model either matches or surpasses the best learned baseline, achieving a test MAPE of 2.2-4.4% compared to 3.4-6.5% (p=0.76). The only exception is the smallest backbone, where the learned baseline excels. This research underscores the promise of analytical models for predicting peak memory in agentic tasks, offering advantages in interpretability and reduced computational demands compared to learned methods. The paper is officially submitted to arXiv as 2608.15117v1.