Error Sensitivity Profile Quantifies Model Data Sensitivity
A new metric known as the Error Sensitivity Profile (ESP) has been introduced by researchers to measure the decline in performance of classification models due to errors in one or more features. To facilitate the calculation of ESP, an integrated tool suite named 'dirty' has been developed. Testing on two datasets involving 14 different models indicates that the deterioration in performance cannot always be anticipated through basic correlations with the target variable.
Key facts
- ESP quantifies sensitivity of model performance to errors in features
- Data-cleaning efforts can be prioritized using ESP
- Integrated tool suite called 'dirty' supports ESP computation
- Experimental study on two datasets using 14 classification models
- Performance degradation not always predictable from simple correlations
Entities
Institutions
- arXiv