H-Sets: Hessian-Guided Discovery of Set-Level Feature Interactions in Image Classifiers
A recent study presents H-Sets, a two-phase framework designed to identify and attribute higher-order interactions among features in image classifiers. This technique employs input Hessians to find pixel pairs that interact locally, subsequently combining them into semantically meaningful groups with the help of segmentation from Segment Anything (SAM). By doing so, it overcomes the shortcomings of current feature attribution techniques that primarily consider marginal effects and fail to account for joint influences of features. H-Sets seeks to enhance the interpretability of explanations for predictions made by deep neural networks in image classification applications.
Key facts
- H-Sets is a two-stage framework for higher-order feature interactions in image classifiers.
- It detects locally interacting pairs via input Hessians.
- Pairs are recursively merged into semantically coherent sets.
- Segment Anything (SAM) provides spatial grouping prior.
- Existing methods focus on marginal effects, ignoring feature interactions.
- Feature interactions are crucial for semantic meaning in images.
- The paper is published on arXiv with ID 2604.22045.
- The method satisfies core interpretability axioms.
Entities
Institutions
- arXiv