Agentic AI Workflows Challenge Datacenter Architecture

A recent study published on arXiv (2608.04458) offers the initial architectural analysis of agentic AI workflows, highlighting notable discrepancies with standard uniform server architectures. Conducted through a production study at Microsoft Azure alongside controlled experiments using open-source frameworks, the research categorizes agentic workflows into a structured taxonomy. The findings indicate that agentic execution is both fragmented and diverse, with requests evolving into workflows involving LLM inferences, tool calls, and orchestration decisions that frequently traverse the CPU-GPU boundary. The CPU's role becomes vital as orchestration and tools operate on the host, leading to low average loads punctuated by sudden spikes. Additionally, model composition influences GPU usage, and the diversity of tasks broadens the range, revealing architectural inadequacies in existing datacenter setups and underscoring the necessity for server designs that cater specifically to agentic workloads.