LLM-Based Coding Agents Enable Just-in-Time System Synthesis
Researchers propose Just-in-Time (JIT) Systems, a new approach where entire systems are synthesized from scratch using LLM-based coding agents, specialized to environment, workload, and required properties. They present Jitskit, a pipeline that iteratively refines system implementations against evolving evaluation test suites. Jitskit synthesizes key-value stores from spec cards covering YCSB workloads, deployment constraints, and system properties. The synthesized systems outperform state-of-the-art systems on all 18 specs tested, achieving up to 4.6x speedup. This challenges the traditional model of building general-purpose systems over years, suggesting a shift toward on-demand, specialized system generation.
Key facts
- JIT Systems are synthesized from scratch using LLM-based coding agents.
- Jitskit is a pipeline for synthesizing key-value stores from spec cards.
- Spec cards cover YCSB workloads, compute resources, consistency, and durability.
- Jitskit iteratively refines implementations against an evolving evaluation test suite.
- Synthesized systems beat state-of-the-art systems on 18 of 18 specs.
- Performance improvement up to 4.6x over comparable systems.
- Traditional systems take years to build and incur performance costs for generality.
- The approach is made tractable by LLM-based coding agents.
Entities
—