DDBench: New Benchmark for Evaluating LLM Agents on Distributed-System Bugs

A new benchmark called DDBench has just been introduced to evaluate how well LLM-based coding agents can handle code repairs in distributed systems. While these agents have made significant strides in solo software engineering tasks—scoring high-70s on SWE-bench Verified—debugging in distributed environments is still pretty much unexplored. Problems here often involve various processes and nodes, making it tough to pinpoint issues just by looking at the code. DDBench addresses two main issues: the lack of a benchmark for distributed-system bugs and the need for studies on how additional debugging context affects agent performance. It features 60 historical bugs from 13 open-source projects, organized by difficulty, and evaluates them under two different settings. This research can be found in a paper on arXiv, labeled 2608.14863, and is vital for improving AI in complex software debugging.