Surrogate Models in Configuration Tuning: Beyond Accuracy via Fitness Landscape Analysis
A recent study published on arXiv (2509.21945) questions the widely accepted notion that greater accuracy in surrogate models always enhances configuration tuning for software systems. The authors contend that "accuracy can lie" and introduce a fresh viewpoint through fitness landscape analysis to assess model effectiveness beyond mere accuracy. They offer a theoretical framework as an alternative metric for evaluating model utility and back their assertions with comprehensive experiments. This paper marks the inaugural systematic investigation into this subject, aiming to reveal the diverse aspects of effective surrogate models in configuration tuning. By addressing the challenge of costly system measurements, the research suggests that relying solely on accuracy may not be sufficient for determining model quality.
Key facts
- Paper arXiv:2509.21945 is a replace-cross announcement.
- The paper focuses on configuration tuning for software system performance, e.g., latency.
- Surrogate models are used to expedite tuning instead of expensive system measurements.
- Prior work found that 'accuracy can lie' in this context.
- The paper provides the first systematic exploration of useful surrogate models beyond accuracy.
- It uses fitness landscape analysis as a novel perspective.
- A theory is presented as an alternative to accuracy for assessing model usefulness.
- Extensive experiments are conducted to support the theory.
Entities
Institutions
- arXiv