PRISM: A New Method to Decompose LLM Drift into Scale, Shape, and Head Mismatches
A new approach called PRISM (Proxy Risk Inference via Structural Mapping) has been introduced by researchers. This method establishes a closed-form upper limit on the cross-entropy risk gap between a target LLM and its modified version (such as quantized, LoRA-adapted, or distilled). By utilizing the linear output head and the nearly isometric backbone structure of LLMs, the method breaks down drift into three independent dimensions: scale mismatch, shape mismatch, and head divergence. Each dimension represents a specific failure mode—shape distortion from low-bit quantization, scale separability due to LoRA forgetting, and head divergence. PRISM aids in variant ranking by diagnosing how a variant has drifted, rather than merely indicating if it has deteriorated. This research is available on arXiv (2605.11608).
Key facts
- PRISM stands for Proxy Risk Inference via Structural Mapping.
- It provides a closed-form upper bound on cross-entropy risk gap.
- Drift is decomposed into scale, shape, and head mismatch axes.
- Scale mismatch relates to LoRA forgetting.
- Shape mismatch relates to low-bit quantization.
- Head divergence is a separate axis.
- The method exploits linear output heads and near-isometric backbones.
- Published on arXiv with ID 2605.11608.
Entities
Institutions
- arXiv