Year-Long Study of LLM Serving Workloads from Chutes

A recent paper on arXiv (2608.13573) offers an in-depth examination of real-world workloads for Large Language Model (LLM) serving, utilizing a year-long production trace from Chutes, a cloud service. Titled "A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing," this research addresses the shortcomings of earlier studies that typically focused on brief periods and lacked comprehensive insights into user interactions. The authors analyze the workload from various angles—aggregate, temporal, model-level, and user-level—demonstrating how both workloads and user-model interactions develop over time. Unlike previous research, this trace encompasses complete production behavior involving numerous models and users, including both mainstream and niche models. This paper enhances the understanding of LLM serving systems, vital for cloud workloads, and aims to guide the design and benchmarking of these systems.