Bottleneck-Preserving Witnessing: New Framework for LLM Serving Trace Reduction
A recent submission to arXiv (2608.00423) presents Bottleneck-Preserving Witnessing (BPW), a framework designed to produce compact and reliable LLM serving replay suites under quality constraints. The authors contend that current trace reduction techniques, which aim to maintain workload distributions or representative requests, overlook infrequent workloads that reveal bottlenecks. They emphasize that the absence of evidence in one area cannot be offset by evidence in another, and relying on predicted bottlenecks as a benchmark leads to circular assessments. BPW resolves these challenges by ensuring that evidence for each bottleneck component is preserved, rather than depending on workload representativeness. The framework begins with Workload Candidate Nomination, utilizing response-blind workload features and closed source-side metrics to identify potential scheduling bottlenecks.
Key facts
- Paper arXiv:2608.00423 introduces Bottleneck-Preserving Witnessing (BPW).
- BPW is a quality-constrained framework for compact and diagnostically reliable LLM serving replay suites.
- Existing trace reduction methods preserve workload distributions or representative requests.
- Bottleneck-revealing workloads may be rare and non-representative.
- Evidence for one component cannot compensate for missing evidence in another.
- Using predicted bottlenecks as target truth creates circular evaluation.
- BPW preserves evidence for every bottleneck component.
- BPW performs Workload Candidate Nomination using response-blind workload features and closed source-side measurements.
Entities
Institutions
- arXiv