Mathematical Proof of AI Explainability Limits
A new paper on arXiv (2605.24727) mathematically proves a fundamental quadrilemma in explaining AI, showing that AI and its explanation cannot simultaneously satisfy four conditions: complexity of the operation environment, goodness of the AI's performance, interpretability of the AI's explanation, and complete faithfulness of the AI's explanation. This suggests that in most applications, completely faithful and interpretable explanations of large-scale AI systems are theoretically impossible, posing challenges for AI governance.
Key facts
- Paper published on arXiv with ID 2605.24727
- Proves a fundamental quadrilemma in explaining AI
- Four conditions cannot be satisfied simultaneously: complexity of operation environment, goodness of AI's performance, interpretability of explanation, complete faithfulness of explanation
- Large-scale models like LLMs and diffusion models are considered
- Public institutions have emphasized explainability in AI
- Existing methods are not designed for completely faithful explanations
- Theoretical impossibility of completely faithful and interpretable explanations in most applications
- Implications for AI governance
Entities
Institutions
- arXiv