ConfigCrusher: White-Box Analysis Framework

• ConfigCrusher is a white-box framework that integrates static program analysis, dynamic measurement, and fitness landscape analysis to evaluate configurable systems.
• It employs region-based instrumentation and local regression models to isolate performance hotspots and identify key configuration interactions.
• Empirical evaluations show high sampling efficiency, minimal instrumentation overhead, and rapid conflict detection in large-scale attributed feature models.

ConfigCrusher is a white-box analysis and search framework for configurable systems, integrating static program analysis, region-based dynamic measurement, and rigorous methodologies from fitness landscape analysis to address both configuration validation and performance modeling. It enables stakeholders to isolate and explain performance-impacting option interactions, efficiently construct accurate prediction models, and iteratively explore large configuration spaces for optimization, all while providing actionable insight into localized and systemic behaviors of highly variable software systems (Velez et al., 2019, Lesta et al., 2015, Huang et al., 2022).

1. Motivation and Problem Scope

Software systems frequently expose a large number of configuration options $O = \{o_1, ..., o_n\}$, with the resulting configuration space $C = \{0,1\}^n$ being combinatorially vast. Stakeholders must reason about how $P(c)$, the performance function under configuration $c \in C$, varies as options are toggled, to support informed decisions in tuning, debugging, and optimization. Existing black-box sampling methods, which evaluate system-level performance over sampled configurations, struggle to (1) reveal higher-order interactions without excessive sampling, (2) localize which code regions or logic are responsible for observed effects, and (3) provide actionable explanations for configuration-induced anomalies or dead/false-optional features (Velez et al., 2019, Lesta et al., 2015).

ConfigCrusher directly addresses these deficits by inspecting source code and using static and dynamic analyses to:

• Map configuration options to code regions and predicates they influence.
• Guide measurements to a minimal, interaction-exercising configuration subset.
• Build localized, interpretable performance-influence models.
• Detect and explain conflicts or contradictions in attributed feature models (AFMs).
• Provide empirical methods for efficient and effective search over configuration landscapes.

2. White-Box Performance Analysis Pipeline

ConfigCrusher's core workflow consists of:

1. Static Data-Flow Analysis: Using a lattice $\mathcal{L} = \mathcal{P}(O)$ and context-, field-, object-sensitive taint analysis, build $SI: S \rightarrow \mathcal{P}(O)$, mapping statements to influencing options. Both control- and data-flow dependencies are tracked, with implicit flows processed via taint propagation (Velez et al., 2019).
2. Code Region Identification and Instrumentation: For each predicate $p$, define minimal contiguous code regions $R_p$ bounded by dominator/post-dominator analysis, grouping statements influenced by identical option sets $I$. Nested and adjacent regions are merged and uninfluenced statements are subsumed, dramatically reducing instrumentation overhead.
3. Configuration Set Compression: Given interacting option-sets, reduce the number of configurations to be measured (covering set $CC \subset C$) by focusing only on those required to exercise all detected interactions, supporting tractable measurement even for large $n$.
4. Dynamic Measurement and Local Model Fitting: Instrumented regions collect timing data $T_r(c)$ during configuration runs. For each region $r$ with influencing options $O_r$, fit a local regression model:

$$P_r(c) = \sum_{S \subseteq O_r} \theta_{r,S} \prod_{i \in S} c_i$$

with coefficients $\theta_{r,S}$ capturing marginal and interaction effects. The global model is the sum $P(c) \approx \sum_r P_r(c)$. Low interaction degree (typically pairs/triples) keeps this expansion tractable (Velez et al., 2019).

1. Conflict Detection in Attributed Feature Models: For AFMs, translate the variability specification $(F, A, root, E, CTC)$ to a finite-domain CSP:

$$\text{Vars} = \{x_f \mid f \in F\} \cup \{v_a \mid a \in A\},\quad \text{Dom}(x_f) = \{0,1\},\quad \text{Dom}(v_a) = \text{Dom}(a) \cup \{nil\}$$

with constraints encoding parent–child, attribute, and cross-tree relations. Inconsistencies are isolated by QuickXplain, efficiently pinpointing minimal conflicting constraint sets (Lesta et al., 2015).

3. Empirical Evaluation and Quantitative Performance

ConfigCrusher was evaluated on 10 open-source Java systems, with SLOC ranging from 69 to over 20,000 and options from 5 to 22 per system. The major findings are:

• Sampling efficiency: ConfigCrusher required only $4{-}256$ configuration measurements to build models with Mean Absolute Percentage Error (MAPE) $\leq 5\%$, compared to thousands for black-box and SPLat approaches, which yielded MAPE in $10{-}600\%$ range on the same budget (Velez et al., 2019).
• Instrumentation optimization: Region-merge and expansion strategies reduced event counts by over $90\%$, limiting runtime overhead to $<3\%$ of uncontended baseline.
• Localization: In all cases, only $1{-}5$ regions out of dozens exhibited non-negligible option-induced performance variation. Local coefficients directly exposed “hot spots,” dead options, and misdocumented features.
• AFM Conflict Explanation: In a real-world product line with $497$ features and $1990$ attributes, constraint pruning and module-grouped QuickXplain produced a $\approx32\times$ speedup with $10{-}20$ consistency checks per conflict, robustly isolating contradictions even as the model evolved (Lesta et al., 2015).

4. Fitness Landscape Analysis and Heuristic Search

Extending beyond white-box modeling, ConfigCrusher++ (an Editor's term) generalizes the approach with rigorous fitness landscape analysis (FLA) for scalable configuration tuning (Huang et al., 2022). Key definitions and experimental procedures include:

• Ruggedness: The configuration landscape exhibits $L \approx 2.5\%-4.2\%\times N$ local optima, very short autocorrelation length ($\lambda \ll n$), and weak fitness-distance correlation ($|r|<0.2$).
• Accessibility: Top $0.1\%$ local optima have basins covering $\gg50\%$ of $\mathcal{C}$; best-improvement hill-climbing from random starts reaches top-20\% optima with $50\%$ probability.
• Escapability: Inferior local optima can be escaped with a $k$-kick ($k=2$) with near-$100\%$ probability; even top peaks permit $\approx50\%$ escape chance, facilitating exploration of global structure.
• Transferability: Structural similarity $\tau_G(w_a, w_b) \approx 0.6-0.9$ (GL2Vec) between workloads enables effective warm-starts, saving $20-60\%$ function evaluations.

An empirically validated search heuristic is as follows:

1. Compute $\tau_G$ to select the 3 workloads structurally closest to the target.
2. Seed search with their top-$K$ configurations and 50 random restarts.
3. For each seed, iteratively hill-climb, triggering $k$-kick perturbations after 10 non-improving steps.
4. Return the configuration maximizing $f_w$ within the computational budget. These settings yield $>90\%$ probability of discovering global optima while maintaining exploration efficiency (Huang et al., 2022).

5. Integration of ConfigCrusher in Large-Scale Systems

ConfigCrusher supports comprehensive configuration management through an integrated pipeline structure (Lesta et al., 2015):

• Model Parsing: Converts AFMs and variability DSLs into intermediate feature and attribute graphs.
• CSP Generation: Emits all variables and constraints to industrial CSP solvers such as CLP(FD), Gecode, or Choco.
• Incremental Consistency Monitoring: Reacts to model/constraint changes by re-validating void, dead, or false-optional features using the reduced graphs.
• Conflict Explanation Modules: Modular invocation of QuickXplain, optionally cascading over user/group-defined modules for isolation of systemic contradictions.
• Interactive UI/Reporting: Issues precise, actionable feedback about specific parent–child, attribute, or arithmetic constraints underlying detected errors.

Adopting mature CSP engines and modular grouping enables near-real-time feedback and validation in very large attributed feature models, with empirical evidence of order-of-magnitude improvements in both speed and informational relevance.

6. Principal Insights, Limitations, and Future Directions

ConfigCrusher’s methodological advances rest on three primary insights (Velez et al., 2019, Huang et al., 2022):

• Irrelevance of most configuration options for any given workload: static analysis eliminates unnecessary measurements.
• Orthogonality and sparsity of option interactions: true performance interactions are low-order and regionally localized.
• Landscape structure: large, accessible basins of attraction, high escapability, and workload-transferable optima support rapid, reliable search.

Limitations include:

• Static Analysis Boundaries: FlowDroid’s precision limits can lead to both under- and over-tainting, affecting interaction detection.
• Single-thread Measurement: Current measurements only support single call-stack semantics, potentially missing cross-thread interaction effects.
• External Validity: Empirical evaluations are on Java single-threaded systems, leaving distributed/multi-language settings as future work.
• Input Distribution Fixity: Model accuracy is tied to the specific workload/input sizes chosen for measurement (Velez et al., 2019).

Future enhancements will likely focus on relaxing these limitations through more scalable static/dynamic analyses, multi-threaded region tracking, and broadened applicability to distributed and larger-scale software ecosystems.

7. Relationship to Black-Box and Variability Analysis Methods

ConfigCrusher outperforms black-box sampling and state-of-the-art SPL validation in both sample efficiency and model accuracy by exploiting source-level structural information and fitness landscape properties (Velez et al., 2019, Huang et al., 2022). Unlike family-based or SPLat combinatorial methods, it fully captures load-time and data-flow variability, supports direct region-based attribution, and adapts to evolving constraints in large-scale AFMs (Lesta et al., 2015). Integration with established constraint solvers and formal diagnostic tools for model validation ensures robust support for both prospective tuning and reactive error isolation across software product lines and configurable platforms.