Batch Normalization Increases Privacy Risks in Neural Networks
A new study reveals that Batch Normalization (BN), a widely used technique for stabilizing deep neural network training, significantly amplifies memorization of outlier samples, leading to greater privacy vulnerabilities. The research, published on arXiv, conducted extensive empirical tests using three methods: unintended memorization of out-of-distribution samples, per-sample influence via gradient norms, and membership inference attacks (MIA). Across multiple datasets and architectures, models with BN consistently showed higher memorization of atypical data and were substantially more susceptible to MIAs. The findings highlight a critical privacy risk in BN, which is commonly employed for faster convergence and training stability.
Key facts
- Batch Normalization (BN) is widely used for faster convergence and stable training of deep neural networks.
- The study investigates BN's impact on memorization of atypical or outlier samples.
- Three complementary approaches were used: unintended memorization of out-of-distribution samples, per-sample influence via gradient norms, and membership inference attacks (MIA).
- Across multiple datasets and architectures, BN substantially increases memorization of outliers.
- Models with BN exhibit significantly higher susceptibility to membership inference attacks.
- The research is published on arXiv with ID 2605.24420.
- The paper is a cross-type announcement.
- The study highlights privacy risks associated with BN.
Entities
Institutions
- arXiv