DMDIntel: New Method Interprets LLMs via Dynamic Mode Decomposition
A groundbreaking technique called DMDIntel has been developed by researchers, utilizing dynamic mode decomposition (DMD) to analyze predictions from large language models (LLMs) in classification scenarios. This approach, outlined in a paper submitted to arXiv, establishes an input attribution pipeline that first dissects the hidden states of an LLM into key patterns, or modes, and subsequently ranks input tokens according to their projection values on these modes. Extensive testing across three datasets and three model families indicates that DMDIntel's ranked attributions surpass leading methods like principal component analysis, integrated gradients, and SHAP. This research, which addresses the increasing demand for AI interpretability, is accessible on arXiv with the identifier 2608.13048 in the Computer Science > Artificial Intelligence category.
Key facts
- DMDIntel uses dynamic mode decomposition (DMD) to interpret LLM predictions.
- It decomposes hidden states into modes and ranks input tokens by projection values.
- Experiments were conducted across three datasets and three model families.
- DMDIntel outperforms principal component analysis, integrated gradients, and SHAP.
- The paper is available on arXiv with ID 2608.13048.
- The research is categorized under Computer Science > Artificial Intelligence.
- The method addresses the need for interpretability in large language models.
Entities
Institutions
- arXiv