Coping with Landscape Ruggedness in Performance Modeling

Develop performance modeling methods for configurable software systems that remain accurate on highly rugged configuration landscapes characterized by abundant local optima and strong fitness fluctuations, explicitly addressing how to mitigate the adverse effects of landscape ruggedness on predictive accuracy across workloads.

Background

The paper’s analysis shows that configuration landscapes for real-world systems (LLVM, Apache, SQLite) are highly rugged, containing large numbers of local optima and pronounced fitness fluctuations. Autocorrelation analyses further corroborate the weak correlation of fitness even among neighboring configurations, underscoring the rugged nature of these landscapes.

Subsequent modeling experiments reveal that the predictive performance of common models (random forests and deep neural networks) is strongly correlated with landscape ruggedness, with moderate R^2 values and systematic biases (overestimating low-fitness and underestimating high-fitness configurations). These findings suggest that ruggedness is a major challenge for accurate performance modeling, motivating the explicit open question on how to cope with ruggedness in the modeling paradigm.