Control Variable Genetic Programming

• Control Variable Genetic Programming (CVGP) is a methodology that partitions variables into controlled and free sets, reducing search space exponentially for symbolic regression and control synthesis.
• The approach achieves state-of-the-art performance by staging variable exploration and freezing stable subtrees, resulting in lower NMSE and improved traffic control metrics.
• CVGP also supports meta-optimization by evolving genetic operators as control variables, enabling co-adaptive strategies for tackling high-dimensional and multimodal problems.

Control Variable Genetic Programming (CVGP) refers to a suite of methodologies in which genetic programming (GP) explicitly models, evolves, or utilizes control variables—either representing control actions in dynamical systems, variables actively manipulated in experimental design, or even the parameters and operators of the GP process itself—as core components of its search and optimization strategies. CVGP combines classical GP's symbolic optimization power with explicit mechanisms for handling the curse of dimensionality, control-law synthesis, or algorithmic meta-optimization through directly modeling or manipulating key control variables using symbolic trees and adaptive workflows.

1. CVGP in Symbolic Regression and Scientific Law Discovery

The canonical application of CVGP is high-dimensional symbolic regression via control variable experiments. In this setting, motivated by classical experimental science, CVGP incrementally builds up multivariate expressions by holding subsets of variables fixed (controlled) while systematically freeing others for active exploration. This staged methodology enables tractable discovery in exponentially large hypothesis spaces (Jiang et al., 2023, Jiang et al., 2023). The approach operates as follows:

• Variables $V = \{x_1, x_2, ..., x_m\}$ are partitioned into "free" and "controlled" at each stage.
• Data are generated (real or simulated) by varying only the "free" variables, holding the others at arbitrary but fixed levels.
• Standard GP evolves expressions $\phi$ to fit this "reduced-form" data, with constants in the expression standing in for sub-expressions of controlled variables.
• Stable subtrees or constants, identified via near-zero fitting error and lack of variation across control settings, are frozen and carried forward; nodes whose fitted value changes significantly across controls are marked as expandable for subsequent rounds.
• At each subsequent stage, additional variables are freed, and GP operates only on the now-expandable regions, iteratively refining the global expression.

Mathematically, the search space reduction is exponential in the number of variables: traditional SR scans $|S(L)| = \Theta((4(m+1)o)^{(L-1)/2})$ trees, while CVGP divides the problem into $m$ subproblems of size $l_0$, for $O(m \cdot |S(l_0)|)$ total effort (Jiang et al., 2023). Empirically, this results in state-of-the-art performance on synthetic and real-world (e.g., Feynman) law discovery problems, with consistently lower normalized mean squared error (NMSE) than standard GP, RL-guided, and commercial baselines (Jiang et al., 2023, Jiang et al., 2023).

2. CVGP for Control-Law Synthesis in Dynamical Systems

CVGP frameworks extend beyond symbolic regression to control synthesis, notably for traffic signal optimization. Here, the control variable is a direct component of the problem—such as phase selection in a signalized intersection:

• At each control timestep $t$, for each possible action (e.g., a signal phase $s_i$), a vector $X_i(t)$ of observable state features (vehicle counts, etc.) is computed.
• A GP-evolved symbolic tree $\Gamma: \mathbb{R}^d \rightarrow \mathbb{R}$ maps features $\phi$0 to an "urgency" score $\phi$1.
• The control action is selected as $\phi$2 (Liao et al., 2024).

Urgecy functions are trees with terminal set $\phi$3 and function set $\phi$4, where $\phi$5 is protected division. Evolution proceeds via standard GP operations—ramped-half-and-half initialization, subtree crossover ($\phi$6), subtree mutation ($\phi$7), tournament selection (size 3), fixed-depth/size constraints, and simulation-based fitness evaluation (e.g., mean vehicle travel time, minimized) using a microscopic simulator (CityFlow) (Liao et al., 2024).

Table: Example final performance on a benchmark (Hangzhou\textsubscript{1})

Method	Mean travel time (s)
Fixed-Time	575.56
Max-Pressure	365.06
MPLight (DRL)	331.15
GPLight (CVGP)	314.12

This direct application of CVGP to learn explicit control laws provides both improved control performance and high interpretability relative to deep RL approaches.

3. Evolution of GP Operators and Meta-Control

An alternative instantiation of CVGP evolves the GP's own search operators (e.g., crossover, mutation) as explicit control variables in the search process, represented as trees and subject to their own evolutionary dynamics (Salinas et al., 2017). In such frameworks:

• Operators (trees built from atomic mutation/crossover/identity nodes) are applied to candidate solutions and selected proportional to empirically measured search improvement.
• Operator rates $\phi$8 evolve through a punish/reward mechanism based on operator-specific success/failure voting, normalized at each generation.
• Operator-trees are themselves genetically adapted (crossover/mutation) over time, yielding a co-evolutionary dynamic where both candidate solutions and the GP search algorithm co-adapt.
• Empirical evaluation shows superior performance, particularly in high-dimensional, multimodal optimization problems versus classic GA and "hybrid adaptive" EAs (Salinas et al., 2017).

A plausible implication is that CVGP is not restricted to modeling domain control variables but can also target meta-optimization of the evolutionary process itself.

4. Algorithmic Details and Pseudocode Structures

Generic CVGP workflows, abstracted over particular domains, share common elements:

• Initialization of GP populations (with variable trees, operator trees, or control laws).
• Structured evolution, often with incremental freeing or staging of variables, or co-evolution of meta-operators.
• Fitness evaluation based on domain-specific cost or error (MSE for regression, simulated control cost for reinforcement contexts, or direct objective for optimization).
• Control variables are explicitly represented and manipulated during evolution—either as parameters to optimize (e.g., phase urgency), arguments to experiment design (e.g., which variables are controlled/free), or evolving GP operator pools.

Below is distilled pseudocode for CVGP in symbolic regression (Jiang et al., 2023):

$\phi$9

For control-law synthesis (Liao et al., 2024):

$|S(L)| = \Theta((4(m+1)o)^{(L-1)/2})$0

5. Comparative Results and Empirical Evidence

Benchmarking across applications illustrates the efficacy of the CVGP paradigm:

• In symbolic regression, CVGP achieves notably lower NMSE on synthetic and physics-inspired benchmarks versus DEAP-GP, Eureqa, and neural-methods, with marked improvements in exact formula recovery rates in noiseless settings (44–64% for CVGP vs. 22–44% for GP) (Jiang et al., 2023, Jiang et al., 2023).
• CVGP in traffic control yields lower average travel times compared to max-pressure heuristics and deep RL (see previous section) (Liao et al., 2024).
• Operator meta-evolution consistently outperforms classical and hybrid evolutionary algorithms on high-dimensional real-valued benchmarks across metrics such as median final fitness (Salinas et al., 2017).

6. Generalization, Limitations, and Future Directions

The core insight underlying CVGP—the explicit manipulation and modeling of control variables—readily generalizes to any setting where the control action space is low-dimensional and observable, and fitness/cost can be evaluated through simulation or experiment. Domains include HVAC (thermostat tuning), robotics (impedance scheduling), chemical processes (feedrate control), and power grids (voltage/reactive power control) (Liao et al., 2024).

Key limitations include dependence on noiseless or low-noise data oracles (particularly in staged SR), scalability bottlenecks in domains lacking adaptive experiment design, and residual challenge in reliably solving constituent GP tasks at each stage (Jiang et al., 2023, Jiang et al., 2023). Advancements such as Racing-CVGP—optimizing the variable-control schedule via racing and pruning of experiment schedules—yield further speedups and accuracy gains, especially as variable dimensionality increases (Jiang et al., 2023).

Suggested future directions:

• Automated schedule learning for variable staging (Jiang et al., 2023).
• Hybridization with neural or sparse regression for robustness to noise (Jiang et al., 2023).
• Extending staged/controlled procedures to differential law and dynamical system discoveries.
• Meta-optimization of GP itself via co-evolution and operator diversity adaptation (Salinas et al., 2017).

7. Connections to Adaptive Parameter-Control in GP

Classical GP parameter-control (e.g., mutation rate, population survivors) can also be understood through the lens of CVGP: mutation rates or survivor fractions are treated as control variables, set via rules triggered by fitness-dynamics signals (e.g., average-fitness delta, stagnation windows) (Gregor et al., 2016). These adaptive schemes can inject diversity (high-mutation or "flood" episodes) or scale selection pressure in response to the observed progress of the evolutionary process. Their application extends to other algorithmic hyperparameters and shares the core CVGP ethos—adapting search via explicit, observable control variable manipulation.

---

References:

• (Jiang et al., 2023): Symbolic Regression via Control Variable Genetic Programming
• (Jiang et al., 2023): Racing Control Variable Genetic Programming for Symbolic Regression
• (Liao et al., 2024): Learning Traffic Signal Control via Genetic Programming
• (Salinas et al., 2017): Self-adaptation of Genetic Operators Through Genetic Programming Techniques
• (Gregor et al., 2016): Fitness-based Adaptive Control of Parameters in Genetic Programming