Metagame Framework Quantifies Second-Order Effects in AI Model Explanations

A new conceptual framework called the metagame has been developed by researchers to assess second-order interaction effects in model explanations. This framework evaluates the directional impact of one feature on the attribution of another, referred to as meta-attribution, by modeling the attribution method as a cooperative game and calculating its Shapley value. The researchers theoretically demonstrate that attributions can be hierarchically broken down into meta-attributions, which serve as directional extensions of current interaction indices. Empirically, the metagame provides valuable insights in various interpretability contexts, including measuring token interactions in instruction-tuned language models, elucidating cross-modal similarities in vision-language encoders, and interpreting concepts in text-to-image multimodal diffusion transformers. This study is available on arXiv in the computer science and machine learning sections.