Conditional Collapse in Low-Rank Weight-Space Ablations: Theory and Synthetic Validation
A recent paper on arXiv (2608.03620) introduces a theoretical framework aimed at clarifying the conditions under which activation patching and weight-space ablation align in assigning causal responsibility to components of a model. The researchers examine a simplified residual stream model characterized by additive conditional computation, establishing specific criteria for when the removal of certain carriers causes a matched input pair to yield the same unconditional output. They also derive a deterministic error form applicable even under approximate conditions. Furthermore, the study reveals that patching a carrier alters the readout based on its donor-receiver contrast, while ablating it affects the readout according to its absolute level, with these metrics not being mutually exclusive. The findings contribute to interpretability studies in machine learning, providing precise mathematical tools for intervention analysis.
Key facts
- Paper arXiv:2608.03620
- Studies activation patching and weight-space ablation
- Idealized model: F(x)=F0(x)+sum_i alpha_i(x)v_i
- Deleting carriers collapses input pair iff removal is symmetric and no outside contrast
- Error is deterministic and exact form given for approximate conditions
- Patching moves readout by donor-receiver contrast
- Ablating moves readout by absolute level
- Contrast and absolute level do not bound each other
- Synthetic validation performed
Entities
Institutions
- arXiv