TDCSO: Two-Step Stochastic Optimization

• TDCSO is a stochastic programming framework that partitions operational problems into discrete commitment and continuous scheduling stages for improved tractability.
• It employs a two-stage decomposition where the first stage fixes binary commitment decisions and the second stage optimizes scenario-specific continuous variables.
• TDCSO achieves significant computational scalability and enhanced operational efficiency in large-scale power systems and hybrid desalination applications.

Two-Step Decomposed Commitment-Scheduling Stochastic Optimization (TDCSO) is a stochastic programming framework designed to partition complex, scenario-driven operational problems—characterized by a combination of binary commitment and continuous scheduling decisions—into a tractable two-stage decomposition. TDCSO is rigorously applied in large-scale power and energy system operations, including stochastic unit commitment (UC) and coordinated power-water scheduling for hybrid reverse osmosis (RO) desalination plants. The approach exploits the two-step structure by first determining scenario-coupled commitment variables, then independently scheduling continuous variables conditioned on the commitment, achieving strong computational scalability, solution quality, and flexibility under uncertainty (Shao et al., 13 Jul 2025, Hu et al., 12 Jan 2026, Dumas, 2023, Rigaut et al., 2023, Shao et al., 14 Aug 2025).

1. Mathematical Formulation and Core Structure

TDCSO is formulated as a two-stage stochastic mixed-integer programming (SMIP) problem. The generic model seeks to minimize the total cost by optimizing discrete (commitment) variables in the first stage and scenario-dependent continuous (scheduling/dispatch) variables in the second stage:

• First-stage (commitment):

$$\min_{x\in X} \;\; c^\top x + \mathbb{E}_{\xi} \left[Q(x, \xi)\right]$$

where $x$ contains binary commitment decisions (e.g., on/off), $c$ are fixed costs, and $\xi$ denotes the scenario realization of uncertainty (e.g., net-load, renewable generation, price, supply).

• Second-stage (recourse/scheduling):

$$Q(x, \xi) = \min_{y \in Y(x, \xi)} \;\; d^\top y \quad \text{s.t.} \;\; Wy \geq h(\xi) - Tx$$

Here, $y$ are scenario-specific scheduling variables (continuous or mixed-integer), $d$ are variable/operational costs, $W$ encodes system constraints, and $T$ links the first and second stage variables (Shao et al., 13 Jul 2025, Dumas, 2023).

This paradigm generalizes to applications where slow, inflexible commitment actions (e.g., generator or plant startup, pump schedules) must be made prior to the realization of uncertainty, and fast scheduling or dispatch actions adapt within each scenario.

2. Decomposition Algorithm: Commitment (Stage 1) and Scheduling (Stage 2)

The essential feature of TDCSO is the explicit decomposition:

• Stage 1 (Commitment): Jointly optimizes all commitment variables that must be scenario-invariant across the uncertainty set. This stage solves a reduced MILP over all scenarios with constraints simplified (e.g., no mixing in desalination, only the most inflexible units in UC), ensuring that scenario-coupled variables (binary decisions) are identical across all scenarios. The output is the optimal scenario-invariant commitment schedule.
• Stage 2 (Scheduling/Dispatch): For each scenario, solves an independent subproblem, optimizing continuous (and possibly binary) recourse variables with the commitment fixed at the solution from Stage 1. In the case of hybrid RO plants, this stage enables flexibility mechanisms (e.g., tank mixing, fast ramping), previously disabled in Stage 1, and omits commitment-linked constraints.

This two-step structure reduces the computational burden compared to monolithic large-scale stochastic MILPs, leverages parallelism for scenario-specific subproblems, and enables advanced flexibility exploitation in the recourse step (Hu et al., 12 Jan 2026, Dumas, 2023, Shao et al., 14 Aug 2025).

3. Scenario Generation, Reduction, and Uncertainty Modeling

Accurate modeling of uncertainty and efficient scenario management are integral to TDCSO implementations:

• Scenario Generation: Large scenario sets (order $10^3$–$10^4$) are constructed by sampling from probabilistic models of exogenous uncertainties (e.g., PV, price, load). In (Hu et al., 12 Jan 2026), a Gaussian copula is fit to historical forecast errors for power and prices to capture joint distributions.
• Scenario Reduction: High cardinality scenario sets are reduced to a tractable subset via clustering (e.g., $k$-medoids, selection of representative/medoid scenarios), assigning probabilities as cluster weights. In multi-stage rolling-horizon settings, scenario generation is updated at each stage using real-time forecast errors, with explicit control of forecast error growth (Dumas, 2023).
• Scenario Coupling: In Stage 1, scenario coupling is enforced through equal commitment variables; all other variables are scenario-indexed. In Stage 2, subproblems are fully decoupled across scenarios, greatly enhancing computational tractability (Hu et al., 12 Jan 2026, Dumas, 2023).

4. Computational Strategies and MILP Formulation

TDCSO frameworks are grounded in polyhedral modeling and linearization techniques for mixed-integer tractability:

• MILP Reformulation: All nonlinearities (e.g., pump curves, power flows, RO recovery, TDS mixing) are linearized using piecewise-linear or SOS3 (triangular domain) constructs, enabling efficient MILP modeling (Hu et al., 12 Jan 2026).
• Scenario Aggregation for Neural Surrogates: Recent work embeds deep neural network surrogates for the second-stage cost function within the first-stage MILP (via big-M ReLU linearizations), leading to problem sizes independent of the scenario count and massive speedups for high-dimensional stochastic UC, at the expense of modest optimality gap ($<1\%$) (Shao et al., 13 Jul 2025).
• Parallelization: Stage 2 subproblems are solved independently for each scenario; parallel computing environments realize substantial wall-clock savings (Hu et al., 12 Jan 2026, Dumas, 2023).
• Rolling-Horizon MPC: In operational settings with long horizons and nested uncertainty, a rolling-horizon approach decomposes commitment decisions temporally—solving small two-stage problems at each last-time-to-decide update and aggregating the commitment trajectory—while refreshing scenario sets adaptively (Dumas, 2023).

5. Theoretical Guarantees and Complexity

Rigorous complexity and optimality analyses for TDCSO methods include:

• Optimality Bounds: For dynamically monotone two-time-scale stochastic optimization, primal (resource) and dual (price) decompositions yield respectively upper and lower bounds on the optimal value function by solving instantiations of fast-scale (scheduling) Bellman equations parameterized by commitment (slow) decisions. Equality is achieved under convexity and monotonicity conditions (Rigaut et al., 2023).
• Finite Termination: Finite convergence of decomposition-based TDCSO algorithms (including column-and-constraint generation and neural surrogate methods) is guaranteed due to the finiteness of the first-stage variable domain and the iterative improvement of scenario cuts or commitment vectors (Shao et al., 14 Aug 2025).
• Polyhedral Complexity: The simplified MILP models in TDCSO yield problem sizes linear in state/action space (and often independent of scenario count via surrogates), in stark contrast to the exponential/hyperlinear scaling inherent in classical multi-stage stochastic programming (Shao et al., 13 Jul 2025, Shao et al., 14 Aug 2025).

6. Applications and Empirical Outcomes

TDCSO has been validated on both infrastructure and benchmark energy system models:

• Hybrid RO Desalination Plants: Implemented with detailed pump, tank, and water quality modeling, TDCSO achieves up to $6\%$ cost savings relative to deterministic or intractable monolithic stochastic optimization, with wall-clock times under 14 min for 10 scenarios (Hu et al., 12 Jan 2026). Key flexibility mechanisms—variable-speed pumps, mixing, preemptive storage—are activated in the second step for maximal dispatch efficiency.
• Stochastic Unit Commitment: In test cases on IEEE 5-, 30-, and 118-bus systems, neural TDCSO achieves near-optimal solutions ($<$1% gap) in computational times orders of magnitude faster than classical MILP or CCG baselines (e.g., 18.6 s vs. 4,340 s for 118-bus, 100-scenario test), and with solution quality robust to scenario scaling up to 1,000 (Shao et al., 13 Jul 2025, Shao et al., 14 Aug 2025). In operational security-of-supply deployments, TDCSO yields 10–30% reductions in expected lost-load versus deterministic benchmarks, with modest additional cost and compatible operational response times (Dumas, 2023).

7. Extensions, Limitations, and Future Directions

TDCSO admits generalization and integration with other decomposition or learning-based schemes:

• Mixed-integer Recourse: Support for fast-start unit commitment (mixed-integer recourse) through corresponding neural or MILP surrogates (Shao et al., 13 Jul 2025).
• Advanced Scenario Embedding: Neural scenario-embedding networks afford scenario set dimensionality reduction and flexible aggregation of uncertainty features (Shao et al., 13 Jul 2025).
• Integration with Benders/Cut-based Tightening: Embedding neural surrogates or resource/price decomposition within Benders or Fenchel duality frameworks may yield further computational gains and tighter bounds (Rigaut et al., 2023, Shao et al., 14 Aug 2025).
• Scalability Constraints: As the number of commitment variables increases ($\gg10^4$), MILP embedding size and solver memory footprints may become limiting; dimensionality reduction and tailored solvers are active areas of investigation (Shao et al., 13 Jul 2025).
• Contextual Adaptation: TDCSO's decomposition paradigm aligns with dynamic programming in slow/fast time-scale operations, sample average approximation approaches in scenario-based stochastic programming, and real-time decision support architectures for critical infrastructure scheduling under deep uncertainty.

TDCSO provides a systematic, theoretically principled, and computationally viable framework for large-scale stochastic optimization of systems structured by commitment and scheduling actions, with empirical support for both improved economic outcomes and tractable solve times under realistic uncertainty models (Hu et al., 12 Jan 2026, Shao et al., 13 Jul 2025, Dumas, 2023, Shao et al., 14 Aug 2025, Rigaut et al., 2023).