Neural Network-Accelerated CCG

• The paper demonstrates that NN-CCG achieves speedups up to 130.1× with optimality gaps below 0.1% in large-scale stochastic unit commitment problems.
• The methodology replaces expensive second-stage MILP solves with rapid neural network evaluations while preserving finite convergence and logical feasibility.
• The approach scales efficiently, reducing recourse evaluation times from minutes to microseconds and enabling robust performance in complex energy market applications.

Neural Network-Accelerated Column-and-Constraint Generation (NN-CCG) refers to a family of methods that integrate neural networks as high-fidelity surrogates for computationally intensive second-stage recourse functions within classical column-and-constraint generation (CCG) frameworks for two-stage (mixed-integer) optimization under uncertainty. These techniques have been developed to drastically increase the tractability of large-scale stochastic or robust optimization models in power systems and energy markets, replacing expensive subproblem solves with rapid neural network evaluations while maintaining strong guarantees of solution quality and finite convergence (Shao et al., 14 Aug 2025, Meng et al., 15 Nov 2025).

1. Two-Stage Optimization and the CCG Framework

Two-stage stochastic and robust programs underlie many real-world decisions under uncertainty, notably in power system operation and energy markets. In the canonical setting, first-stage ("here-and-now") decision variables $x$ are chosen before uncertainty $\omega$ (stochastic) or $\xi$ (robust) is realized. Second-stage ("wait-and-see") recourse variables $y$ optimally respond to each scenario. The deterministic equivalent can be computationally infeasible for realistic scenario sets $|\Omega|$ or high-dimensional uncertainty sets $U$. The CCG algorithm iteratively constructs the scenario set or uncertainty support relevant for optimality, solving a master problem (MP) and repeatedly querying second-stage subproblems. Computational bottlenecks arise as the subproblem structure generally requires mixed-integer programming (MIP) or bilinear max–min solves for each candidate $x$ across all (or many) scenarios per iteration, dominating overall runtime (Shao et al., 14 Aug 2025, Meng et al., 15 Nov 2025).

2. Classical Column-and-Constraint Generation (CCG) Algorithm

The classical CCG algorithm alternates between:

• Solving a master problem over a restricted scenario set $S^k\subset\Omega$ or $\{\xi^{(1)},…,\xi^{(k)}\}$, yielding a tentative solution $x^{(k)}$ and lower bound.
• Solving, for every scenario or extreme point in the uncertainty set, the recourse (second-stage) subproblem to evaluate the true cost $Q(x^{(k)},\omega)$ or $Q(x^{(k)},\xi)$, identifying the “worst-case” scenario or point $\omega^*$ or $\xi^*$.
• Enriching the master problem with the newly identified scenario/constraint, and iterating until the difference between upper and lower bounds falls below a prescribed tolerance.

This procedure guarantees finite convergence for polyhedral (finite) $\Omega$ or $U$ and combinatorial first-stage feasible sets. However, as $|\Omega|$ or the size of the uncertainty polytope grows, the cost of solving potentially thousands of large MILPs or max–min LPs per iteration becomes prohibitive, often constituting 60–95% of overall runtime in large-scale settings (Shao et al., 14 Aug 2025).

3. Neural Surrogate Modeling of Recourse Functions

NN-CCG introduces a neural network $\mathrm{NN}_\Theta$ as a surrogate for the recourse function $Q(x,\omega)$ or $Q(x,\xi)$. This surrogate is trained offline on large corpora of $(x,\xi)$ (or $(x,\omega)$) pairs, where true recourse values are obtained by direct solution of the exact subproblem for randomized first-stage decisions and sampled uncertainty realizations.

• For stochastic unit commitment (SUC), the input features typically concatenate all first-stage binary variables $$\mathrm{flatten}(z), \mathrm{flatten}(u), \mathrm{flatten}(v)$$ and the scenario $\xi_\omega$; a multilayer perceptron (MLP) with sizes [1024, 512, 256, 128] with ReLU activations is trained using mean-squared error to target $Q(x,\omega)$, with explicit penalties included for infeasibility (e.g., load-shedding costs) (Shao et al., 14 Aug 2025).
• For adaptive robust/stochastic day-ahead offering, the NN is constructed to be representable as a MILP (using ReLU activations throughout); the architecture embeds low-dimensional representations of $x$ and $\xi$ via dedicated two-layer MLPs and a final single hidden-layer value net. The NN outputs $\widehat Q(x,\xi)$ for use in both master and subproblem roles (Meng et al., 15 Nov 2025).

Empirically, validation errors on held-out samples fall below 1.3% in the day-ahead offering problem, and optimality gaps in downstream optimization tasks are consistently below 0.1% (Meng et al., 15 Nov 2025, Shao et al., 14 Aug 2025).

4. Integration of Neural Surrogates into CCG

The NN-CCG framework replaces explicit (often exact) evaluation of recourse subproblems in the CCG loop with surrogate evaluation and maximization:

1. Surrogate evaluation: For the current $x^{(k)}$ from the master, compute $Q̂(x^{(k)},\omega; \theta)$ or $\mathrm{NN}_\Theta(x^{(k)}, \xi)$ for all candidate scenarios/uncertainties in $\Omega$ or $U$, identifying the maximizer.
2. Master update: Enrich the master scenario set with the identified maximizer, repeating the process.
3. Convergence: Iteration terminates when the surrogate-predicted recourse cost for the current $x^{(k)}$ over all of $U$ is within a small tolerance $\epsilon$ of the maximum over the current scenario set.

The surrogate can be encoded into MILP form via big-M constraints and binary variables for argmax implementations, as in the robust day-ahead offering problem (Meng et al., 15 Nov 2025). In SUC applications, a forward pass of the MLP is used per scenario, and the master problem remains unchanged aside from surrogate-driven scenario selection (no MILP encoding required) (Shao et al., 14 Aug 2025).

Safeguards, such as embedding penalty slacks in the SUC master problem, guarantee logical feasibility of any iteratively generated solution. After convergence, a final certification step can optionally be performed by evaluating the true recourse function at the computed solution across all scenarios.

5. Theoretical Properties and Convergence

NN-CCG maintains the finite termination property of classical CCG when the first-stage feasible set and the scenario/uncertainty support are finite. Proposition 1 in (Shao et al., 14 Aug 2025) formalizes this for the stochastic UC setting. Although the substitution of an inexact NN surrogate for the true recourse breaks the strict optimality guarantee, empirical results indicate that optimality gaps are well-controlled—remaining typically below 0.1% for SUC and at or below 0.001% in power system offering problems (Shao et al., 14 Aug 2025, Meng et al., 15 Nov 2025).

A key complexity reduction arises because a forward pass through the NN (10–100 μs per evaluation) is multiple orders of magnitude faster than solving an exact MILP or max–min LP per scenario (often many seconds to minutes). This effect is magnified as the size of the scenario or uncertainty set grows.

6. Numerical Results and Performance Benchmarks

Empirical results demonstrate the scalability and efficiency of NN-CCG:

• For stochastic unit commitment (IEEE 118-bus system, up to $|\Omega|=1000$ scenarios), Neural-CCG attained up to 130.1× speedup versus Gurobi, with a mean optimality gap not exceeding 0.096%. Compared to standard CCG, Neural-CCG achieved speedups from 17× to 130× as $|\Omega|$ increases (Shao et al., 14 Aug 2025).

Method	Gap (%)	Speedup (1000 scenarios)
Gurobi	0	1.0×
CCG	0.068	5.0×
Neural-CCG	0.096	130.1×

• For DER day-ahead offering (1028-node test system, up to 500 price scenarios), NN-CCG yields typical speed-ups ranging from 22× to over 100× against direct Gurobi solves and more than 30× against classical CCG. Objective value deviations are within 0.001%, showing negligible suboptimality (Meng et al., 15 Nov 2025).

# Scenarios	Gurobi Time (s)	CCG Time (s)	NN-CCG Time (s)	Speedup vs Gurobi	NN-CCG Gap (%)
500	125,382	40,841	1,232	101.74×	0.001

A consistent observation is that NN-CCG runtime grows only mildly with the scenario or uncertainty count, due to the lightweight evaluation and encoding of the neural surrogate.

7. Applications, Limitations, and Extensions

NN-CCG methods are validated in multi-period stochastic unit commitment and large-scale robust/stochastic day-ahead offering in power systems (Shao et al., 14 Aug 2025, Meng et al., 15 Nov 2025). Even with modest neural architectures (e.g., a 4-layer MLP or small ReLU networks), surrogate accuracy is sufficient to support high-fidelity optimization, suggesting strong potential for extension to other two-stage mixed-integer programs such as stochastic security-constrained optimal power flow and robust network design. However, surrogate modeling introduces a trade-off—a small but present risk of suboptimality if the surrogate fails to generalize, and a need for large, representative datasets to ensure accuracy (Shao et al., 14 Aug 2025).

The architecture can be further specialized; replacing the MLP with a graph neural network may improve the learning of topological dependencies in network-constrained optimization (Shao et al., 14 Aug 2025). Tight MILP-representable NN design ensures compatibility with commercial solvers and explicit encoding in optimization pipelines, as demonstrated in robust offering applications (Meng et al., 15 Nov 2025).

Overall, NN-CCG transforms the computational viability of scenario-based decomposition approaches for real-time, large-scale energy system operations by shifting the loop bottleneck from enumerative subproblem solves to a single, high-quality surrogate model evaluation per iteration.