New Research Paper Analyzes Information-Theoretic Limits of Masking-Based AI Explanation Methods
So, there’s this new paper on arXiv called 'The Query Channel: Information-Theoretic Limits of Masking-Based Explanations,' which you can find with the code arXiv:2604.16689v1. It looks at how post-hoc explanation methods like KernelSHAP and LIME act like communication through a query channel. Each masked evaluation is like using that channel. The study links the complexity of these explanations to the entropy of the hypothesis class, while the rate at which information is shared depends on the channel's capacity per query. The authors show that if you try to explain too much too fast, you’ll likely make mistakes. They also found that a sparse maximum-likelihood decoder can still work well if you stay below that capacity. Plus, they came up with a Monte Carlo method to estimate mutual information, which helps in figuring out local feature importance by querying black-box models with random changes.
Key facts
- Research paper published on arXiv with identifier arXiv:2604.16689v1
- Analyzes masking-based post-hoc explanation methods like KernelSHAP and LIME
- Formulates explanation procedures as communication over a query channel
- Each masked evaluation represents a channel use in the framework
- Explanation complexity captured by entropy of hypothesis class
- Query interface supplies information at rate determined by identification capacity per query
- Proves strong converse showing probability of exact recovery converges to one in error when rate exceeds capacity
- Establishes achievability result with sparse maximum-likelihood decoder attaining reliable recovery below capacity
Entities
Institutions
- arXiv