Graph Kernels for LLM Mechanistic Interpretability

A new framework reframes mechanistic interpretability of large language models as a graph machine-learning problem. Researchers propose representing activation-patching profiles as patch-effect graphs over model components, introducing three graph-construction methods: direct-influence via causal mediation, partial-correlation, and co-influence. Applying graph kernels to GPT-2 Small on Indirect Object Identification tasks shows that patch-effect graphs preserve discriminative structural signals, with localized edge-slot features achieving higher classification accuracy than global methods.