FAIR-Pruner: Adaptive Layer-Wise Pruning via Tolerance of Difference

A new framework called FAIR-Pruner has been developed by researchers to improve structured pruning in deep neural networks, and it eliminates the search process entirely. This method features two rankings within each layer: one that points out which units could be removed and another that highlights those crucial for specific tasks. The key component, known as the Tolerance of Difference (ToD), evaluates how much the units being removed overlap with those that need to be kept, allowing for different pruning depths in various layers. It also includes a vision implementation that combines Wasserstein-based U-Score for separating class-specific units and Taylor-based R-Score for task sensitivity. The ToD rule is adaptable to different signals, enhancing structured pruning's practical application.