Agent JIT Compilation Reduces Latency in Web Agent Planning

A novel technique known as agent just-in-time (JIT) compilation seeks to minimize latency in computer-use agents (CUA) that facilitate browser tasks, such as placing orders at Taco Bell. Existing systems rely on a sequential fetch-screenshot-execute loop, necessitating an LLM call for each iteration, which leads to increased latency and potential errors. In contrast, JIT compilation transforms task descriptions into executable code that encompasses LLM calls, tool calls, and parallel processing. This method comprises three elements: JIT-Planner, which generates and validates multiple code plans to identify the lowest-cost option; JIT-Scheduler, which assesses parallelization methods using Monte Carlo cost estimation; and a protocol that enforces correct tool usage. The research can be found on arXiv with ID 2605.21470.