LLM-Agent Control: GPU Opportunity and Avoiding Host Round Trips

A new study on arXiv (2608.12123) dives into how LLM-agent services manage control, focusing on their predictable, small transitions during model and tool interactions. The authors introduce the concept of the 'ready-cohort boundary,' using metrics like fixed-partition share F, offline share P*, local upper bound U, and online share A. They apply dynamic programming to compute P* under ideal conditions, such as no service time and limitless capacity. In a Poisson replay with 851 sessions, aiming for 100,000 active sessions, they found F=30.19%, P*=43.00%, and U=45.85%. Their method recovers 81.83% of lost opportunities at fixed window boundaries, showing that GPU route decisions can stay on the device without needing host connections. The complete paper is available on arXiv.