High-Weight Neurons Not Always Critical in Image Classification Neural Networks
A recent study published on arXiv questions the belief that neurons with high weights are the most crucial for image classification in neural networks. Researchers introduced a method to assess neuron importance through three experiments: measuring the overlap between high-weight neurons and those affecting accuracy, examining perturbation impacts, and evaluating post-retraining accuracy following ablation. Tests conducted on CIFAR-10 and Mini-ImageNet revealed that only about 25% of the top 10% high-weight neurons coincided with significant ones. Perturbation assessments indicated that these high-weight neurons could lead to a 45-80% drop in accuracy, while random perturbations resulted in only 3-7%. Additionally, ablation-retraining showed that removing these neurons might enhance accuracy. These results carry important implications for pruning, backdoor defense, and neural network interpretability.
Key facts
- Study challenges assumption that high-weight neurons are most important in image classification neural networks.
- Three experiments: overlap analysis, perturbation effects, and ablation-retraining.
- Experiments conducted on CIFAR-10 and Mini-ImageNet datasets.
- Top 10% high-weight neurons overlap with important ones by only about 25% at maximum.
- Perturbation of top 10% high-weight neurons causes 45-80% accuracy degradation under certain operations.
- Random perturbations cause 3-7% accuracy degradation.
- A third of top 10% high-weight neurons show minimal impact on accuracy.
- Removing top 10% high-weight neurons can sometimes improve accuracy after retraining.
Entities
Institutions
- arXiv