Model Cascades Show Robustness and 10-Fold CO2 Reduction Under Data Perturbations

Prediction cascades, which route easy inputs through small AI models and defer difficult cases to larger ones, reduce energy consumption while maintaining performance. However, their effectiveness relies on confidence-based routing, which input degradations like static corruptions and sequential perturbations can disrupt. This paper studies such cascades for image classification, selecting a model at the Pareto-optimum of accuracy, routing quality, and energy efficiency. That cascade achieves competitive predictive performance with up to a 10-fold reduction in CO2 emissions. The findings highlight the sensitivity of confidence-based deferral under data perturbations, informing robust and sustainable AI deployment.