Vanishing Contributions: A Unified Framework for Smooth DNN Compression

The paper introduces Vanishing Contributions (VCON), a unified framework for smooth and iterative compression of Deep Neural Networks (DNNs). Traditional compression techniques like pruning, quantization, and low-rank decomposition reduce memory and computation but often cause accuracy degradation, mitigated by iterative gradual compression. However, different methods require distinct iterative approaches and can lead to unstable fine-tuning. VCON addresses this by running the original and compressed models in parallel during fine-tuning, progressively reducing the contribution of the uncompressed model while increasing that of the compressed one. This affine combination enables a smooth transition without discontinuous jumps. The framework is model-agnostic and applicable to various compression techniques. The paper is published on arXiv under ID 2510.09696, with an announcement type of replace-cross.