Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
169 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Unlocking the Secrets of Software Configuration Landscapes-Ruggedness, Accessibility, Escapability, and Transferability (2201.01429v2)

Published 5 Jan 2022 in cs.SE and cs.NE

Abstract: Modern software systems are often highly configurable to tailor varied requirements from diverse stakeholders. Understanding the mapping between configurations and the desired performance attributes plays a fundamental role in advancing the controllability and tuning of the underlying system, yet has long been a dark hole of knowledge due to their black-box nature and the enormous combinatorial configuration space. In this paper, using $86$M evaluated configurations from three real-world systems on $32$ running workloads, we conducted one of its kind fitness landscape analysis (FLA) for configurable software systems. With comprehensive FLA methods, we for the first time show that: $i)$ the software configuration landscapes are fairly rugged, with numerous scattered local optima; $ii)$ nevertheless, the top local optima are highly accessible, featuring significantly larger basins of attraction; $iii)$ most inferior local optima are escapable with simple perturbations; $iv)$ landscapes of the same system with different workloads share structural similarities, which can be exploited to expedite heuristic search. Our results also provide valuable insights on the design of tailored meta-heuristics for configuration tuning; our FLA framework along with the collected data, build solid foundation for future research in this direction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. T. Xu, L. Jin, X. Fan, Y. Zhou, S. Pasupathy, and R. Talwadker, “Hey, you have given me too many knobs!: understanding and dealing with over-designed configuration in system software,” in ESEC/FSE’15.   ACM, 2015, pp. 307–319.
  2. N. Siegmund, A. Grebhahn, S. Apel, and C. Kästner, “Performance-influence models for highly configurable systems,” in ESEC/FSE’15.   ACM, 2015, pp. 284–294.
  3. M. Velez, P. Jamshidi, F. Sattler, N. Siegmund, S. Apel, and C. Kästner, “Configcrusher: towards white-box performance analysis for configurable systems,” Autom. Softw. Eng., vol. 27, no. 3, pp. 265–300, 2020.
  4. X. Han, T. Yu, and D. Lo, “Perflearner: learning from bug reports to understand and generate performance test frames,” in ASE’18.   ACM, 2018, pp. 17–28.
  5. T. Chen, K. Li, R. Bahsoon, and X. Yao, “FEMOSAA: feature-guided and knee-driven multi-objective optimization for self-adaptive software,” ACM Trans. Softw. Eng. Methodol., vol. 27, no. 2, pp. 5:1–5:50, 2018.
  6. Y. Zhu, J. Liu, M. Guo, Y. Bao, W. Ma, Z. Liu, K. Song, and Y. Yang, “Bestconfig: tapping the performance potential of systems via automatic configuration tuning,” in SoCC’17.   ACM, 2017, pp. 338–350.
  7. V. Nair, Z. Yu, T. Menzies, N. Siegmund, and S. Apel, “Finding faster configurations using FLASH,” IEEE Trans. Software Eng., vol. 46, no. 7, pp. 794–811, 2020. [Online]. Available: https://doi.org/10.1109/TSE.2018.2870895
  8. A. H. Ashouri, W. Killian, J. Cavazos, G. Palermo, and C. Silvano, “A survey on compiler autotuning using machine learning,” ACM Comput. Surv., vol. 51, no. 5, pp. 96:1–96:42, 2019.
  9. S. Wright, “The roles of mutations, inbreeding, crossbreeding and selection in evolution,” in Proc. of the 11th International Congress of Genetics, vol. 1, 1932, pp. 356–366.
  10. E. D. Vaishnav, C. G. de Boer, J. Molinet, M. Yassour, L. Fan, X. Adiconis, D. A. Thompson, J. Z. Levin, F. A. Cubillos, and A. Regev, “The evolution, evolvability and engineering of gene regulatory DNA,” Nature, vol. 603, no. 7901, pp. 455–463, 2022. [Online]. Available: https://doi.org/10.1038/s41586-022-04506-6
  11. K. M. Malan, “A survey of advances in landscape analysis for optimisation,” Algorithms, vol. 14, no. 2, p. 40, 2021.
  12. D. A. Levinthal, “Organizational adaptation and environmental selection-interrelated processes of change,” Org. Sci., vol. 2, no. 1, pp. 140–145, 1991.
  13. J. P. Doye, “Network topology of a potential energy landscape: A static scale-free network,” Phy. Rev. Lett., vol. 88, no. 23, p. 238701, 2002.
  14. V. K. Vassilev, J. F. Miller, and T. C. Fogarty, “Digital circuit evolution and fitness landscapes,” in CEC’99.   IEEE, 1999, pp. 1299–1308.
  15. P. Jamshidi and G. Casale, “An uncertainty-aware approach to optimal configuration of stream processing systems,” in MASCOTS’16.   IEEE Computer Society, 2016, pp. 39–48.
  16. G. Ochoa, M. Tomassini, S. Vérel, and C. Darabos, “A study of NK landscapes’ basins and local optima networks,” in GECCO’08.   ACM, 2008, pp. 555–562.
  17. M. Huang and K. Li, “Exploring structural similarity in fitness landscapes via graph data mining: A case study on number partitioning problems,” in IJCAI’23.   ijcai.org, 2023, pp. 5595–5603. [Online]. Available: https://doi.org/10.24963/ijcai.2023/621
  18. S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345–1359, 2010.
  19. T. Weise, “Global optimization algorithms-theory and application,” Self-Published Thomas Weise, vol. 361, 2009.
  20. L. D. Whitley, A. E. Howe, and D. Hains, “Greedy or not? best improving versus first improving stochastic local search for MAXSAT,” in AAAI’13.   AAAI Press, 2013, pp. 940–946.
  21. F. H. Stillinger, “A topographic view of supercooled liquids and glass formation,” Science, vol. 267, no. 5206, pp. 1935–1939, 1995.
  22. A. Biedenkapp, M. Lindauer, K. Eggensperger, F. Hutter, C. Fawcett, and H. H. Hoos, “Efficient parameter importance analysis via ablation with surrogates,” in AAAI’17.   AAAI Press, 2017, pp. 773–779.
  23. G. Ochoa and N. Veerapen, “Deconstructing the big valley search space hypothesis,” in EvoCOP’16, vol. 9595.   Springer, 2016, pp. 58–73.
  24. S. L. Thomson, F. Daolio, and G. Ochoa, “Comparing communities of optima with funnels in combinatorial fitness landscapes,” in GECCO’17.   ACM, 2017, pp. 377–384.
  25. E. Weinberger, “Correlated and uncorrelated fitness landscapes and how to tell the difference,” Biol. Cybern., vol. 63, no. 5, pp. 325–336, 1990.
  26. T. Jones and S. Forrest, “Fitness distance correlation as a measure of problem difficulty for genetic algorithms,” in ICGA’95.   Morgan Kaufmann, 1995, pp. 184–192.
  27. J. N. Onuchic and P. G. Wolynes, “Theory of protein folding,” Current opinion in structural biology, vol. 14, no. 1, pp. 70–75, 2004.
  28. M. Harman, S. A. Mansouri, and Y. Zhang, “Search-based software engineering: Trends, techniques and applications,” ACM Comput. Surv., vol. 45, no. 1, pp. 11:1–11:61, 2012.
  29. M. Feurer, J. T. Springenberg, and F. Hutter, “Initializing bayesian hyperparameter optimization via meta-learning,” in AAAI’15.   AAAI Press, 2015, pp. 1128–1135.
  30. H. Chen and H. Koga, “Gl2vec: Graph embedding enriched by line graphs with edge features,” in ICONIP’19, ser. Lecture Notes in Computer Science, T. Gedeon, K. W. Wong, and M. Lee, Eds., vol. 11955.   Springer, 2019, pp. 3–14.
  31. J. Oh, D. S. Batory, M. Myers, and N. Siegmund, “Finding near-optimal configurations in product lines by random sampling,” in ESEC/FSE’17.   ACM, 2017, pp. 61–71.
  32. M. Li, L. Zeng, S. Meng, J. Tan, L. Zhang, A. R. Butt, and N. C. Fuller, “MRONLINE: mapreduce online performance tuning,” in HPDC’14.   ACM, 2014, pp. 165–176.
  33. A. Shahbazian, S. Karthik, Y. Brun, and N. Medvidovic, “equal: informing early design decisions,” in ESE/FSE’20, 2020, pp. 1039–1051.
  34. R. Olaechea, D. Rayside, J. Guo, and K. Czarnecki, “Comparison of exact and approximate multi-objective optimization for software product lines,” in SPLC’18.   ACM, 2014, pp. 92–101.
  35. C. Kaltenecker, A. Grebhahn, N. Siegmund, J. Guo, and S. Apel, “Distance-based sampling of software configuration spaces,” in Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019.   IEEE / ACM, 2019, pp. 1084–1094.
  36. H. Ha and H. Zhang, “Deepperf: performance prediction for configurable software with deep sparse neural network,” in ICSE’19.   IEEE / ACM, 2019, pp. 1095–1106.
  37. P. Valov, J. Guo, and K. Czarnecki, “Empirical comparison of regression methods for variability-aware performance prediction,” in SPLC’15.   ACM, 2015, pp. 186–190.
  38. P. Jamshidi, N. Siegmund, M. Velez, C. Kästner, A. Patel, and Y. Agarwal, “Transfer learning for performance modeling of configurable systems: an exploratory analysis,” in ASE’17.   IEEE Computer Society, 2017, pp. 497–508.
  39. M. Velez, P. Jamshidi, N. Siegmund, S. Apel, and C. Kästner, “White-box analysis over machine learning: Modeling performance of configurable systems,” in ICSE’21, 2021, accepted for publication.
  40. P. Jamshidi, M. Velez, C. Kästner, and N. Siegmund, “Learning to sample: exploiting similarities across environments to learn performance models for configurable systems,” in ESEC/FSE’18.   ACM, 2018, pp. 71–82.
  41. M. J. V. D. Donckt, D. Weyns, F. Quin, J. V. D. Donckt, and S. Michiels, “Applying deep learning to reduce large adaptation spaces of self-adaptive systems with multiple types of goals,” in SEAMS’20.   ACM, 2020, pp. 20–30.
  42. S. A. Fahlberg, C. R. Freschlin, P. Heinzelman, and P. A. Romero, “Neural network extrapolation to distant regions of the protein fitness landscape,” bioRxiv, pp. 2023–11, 2023.
  43. C. Kinneer, D. Garlan, and C. L. Goues, “Information reuse and stochastic search: Managing uncertainty in self-**{}^{\mbox{*}}start_FLOATSUPERSCRIPT * end_FLOATSUPERSCRIPT systems,” ACM Trans. Auton. Adapt. Syst., vol. 15, no. 1, pp. 3:1–3:36, 2021.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now