QUIVER Framework Measures Perturbation Propagation in Compound AI Systems
A new framework called QUIVER, which stands for Quantifying Perturbation Propagation and Bifurcation in Compound AI Systems, has been introduced to evaluate how disruptions spread in complex AI systems that use multiple large language model calls within directed computation graphs. These systems are common in production AI and consist of various nodes with different output types. Previously, there wasn’t an effective way to measure the propagation of perturbations in these unpredictable pipelines. QUIVER includes four key elements: a sensitivity matrix for categorizing edges, trajectory divergence analysis, bifurcation thresholds for identifying minimal perturbations that change execution paths, and distribution faithfulness. This framework can be found on arXiv under the identifier 2605.23956.
Key facts
- QUIVER is a formal framework for measuring perturbation propagation in graph-structured LLM pipelines.
- Compound AI systems that chain multiple LLM calls are the dominant architecture for production AI.
- No existing framework could quantify perturbation propagation through stochastic pipelines with structural path divergence.
- QUIVER defines a sensitivity matrix with type-dispatched distance metrics.
- Edges are classified as amplifiers, absorbers, or threshold-sensitive.
- Trajectory divergence decomposes variation into value drift, structural path divergence, and iteration count divergence.
- Bifurcation thresholds identify the smallest perturbation causing structural execution path changes.
- The framework was announced on arXiv with ID 2605.23956.
Entities
Institutions
- arXiv