Multistage Stochastic Programming Overview

• Multistage stochastic programming is a modeling framework for optimal sequential decisions that incorporates uncertainty and nonanticipativity constraints.
• It employs techniques like SDDP, adaptive sampling, and rolling-horizon policies to efficiently address high-dimensional and long-horizon problems.
• The approach is widely applied in power systems, logistics, finance, and dynamic control, enhancing cost-efficiency and risk management.

Multistage stochastic programming (MSP) encompasses a core class of mathematical programming models designed to determine optimal sequential decision policies under uncertainty, where the unfolding random process is revealed progressively over multiple stages. At every stage, decisions must be made based on observed history (nonanticipativity), balancing current costs with expected future outcomes. MSP frameworks subsume both linear and integer programs with broad applications ranging from power systems, logistics, finance, and operations planning to control of stochastic dynamic systems.

1. Foundations and Mathematical Structure

A canonical multistage stochastic program is structured around a sequence of stages $t=1,\dots,T$, each associated with a state $x_t$ and a decision $u_t$, subject to uncertain data $\xi_t$ (often modeled by a probability space with an adapted filtration). The general MSP seeks a nonanticipative policy minimizing the expected sum of stagewise costs,

$$\min_{U \in \mathcal{U}} \mathbb{E} \left[ \sum_{t=0}^{T-1} c_t(X_t, U_t, X_{t+1}) + c_T(X_T) \right],$$

with stagewise constraints (possibly recourse, integrality, or network structure), and state/control processes adapted to the filtration generated by $(\xi_1, \dots, \xi_T)$ (Dommel et al., 2021). The dynamic programming principle yields recursive cost-to-go functions $V_t$, underpinned by measurable-selection and continuity theorems, establishing well-posedness in both discrete and continuous spaces.

2. Nonanticipativity, Scenario Trees, and Policy Representations

The nonanticipativity constraint is central: stage-$t$ decisions depend only on information observable up to $t$. In scenario-tree representations, nodes correspond to information sets (histories), and feasible decisions are forced to coincide at nodes sharing histories. For problems with finitely supported uncertainty, the deterministic equivalent is a large (often intractable) extensive-form program over the scenario tree, with nonanticipativity enforced by node-linking constraints. For infinite or high-dimensional uncertainty, scenario-sampling, tree reduction, and approximate dynamic programming become essential (Defourny et al., 2011, Siddig et al., 2019).

Policies can be represented functionally (decision rules mapping histories to actions), via scenario-trees (with stagewise values at each node), or as parameterized lookahead/planning models (see Section 4).

3. Algorithmic Methods: Decomposition, SDDP, and Scenario Handling

The exponential scaling with stages and scenario branching (curse of dimensionality) motivates decomposition:

• Stochastic Dual Dynamic Programming (SDDP): For problems with (convex) recourse and stagewise independence, SDDP constructs piecewise-affine lower bounds on cost-to-go functions via Benders cuts added at each iteration; forward and backward sweeps yield lower and upper bounds, with convergence under mild regularity (Siddig et al., 2019, Gangammanavar et al., 2020).
• Adaptive Strategies: Adaptive partition-based SDDP accelerates early-stage iterations by aggregating similar scenarios for cut generation, refining only when the discrepancy exceeds a tolerance, to manage large scenario-trees efficiently (Siddig et al., 2019).
• Sequential Sampling (SDLP): Alternative to SAA or tree-based MSP, SDLP uses sample-path-wise updates with quadratic regularization and basic feasible policies, converging almost surely to the true cost-to-go (Gangammanavar et al., 2020).
• Sample Complexity: For Markovian uncertainty, the Markov Recombining Scenario Tree (MRST) method leverages recombination and kernel regression on a small set of trajectories, yielding polynomial sample complexity in horizon length, sharply contrasting with the exponential growth in classic SAA scenario trees (Park et al., 2024).

Algorithm	Principle	Scalability
SDDP	Benders cuts	Effective for convex MSP; scenario-tree size limits for nonconvex/integer
Adaptive SDDP	Cut aggregation	Improved early-stage wall-clock, adaptive refinement
SDLP	Sample-path, regularization	Scenario-tree free, lower per-iteration cost
MRST (Markov)	Recombination, kernel regression	Polynomial sample growth in $T$

4. Policy Approximation: Parametric and Rolling-Horizon Methods

Due to computational intractability in long-horizon or high-dimensional MSPs, several approximative and hybrid modeling paradigms have emerged:

• Parametric Cost Function Approximation (CFA): State-of-practice in industry but newly formalized as a pragmatic alternative to scenario-tree or DP methods. A tractable deterministic lookahead model is parameterized (e.g., via forecast discount factors, safety stocks), and the parameters are tuned offline to minimize true expected cost by simulation. The CFA framework offers computational and data advantages for complex, high-dimensional state spaces, but success hinges on judicious parameterization and credible simulation data (Powell et al., 2022).
• Rolling-Horizon Policies: Rather than solving the full-horizon MSP, a sequence of truncated-horizon problems (of horizon $H$) is solved at each stage. ADP-based heuristics and explicit error bounds for infinite-horizon discounted problems provide guidance for horizon selection. Empirically, small $H$ (8–16) is often sufficient to attain near-optimal cost in resource planning, with massive savings in computational time (Siddig et al., 2021). State-adaptive lookahead lengths (learned by regression) further improve efficiency.
• Incorporation of Forecasting: Advanced, probabilistic time-series models (e.g., DeepAR) integrated into rolling-horizon or scenario-generation steps materially improve realized policy performance over classical AR(1)/ARMA models, especially for risk-averse or worst-case robust variants (Wang et al., 2020).

5. Extensions: Integer and Adaptive Models, Value of Flexibility, K-Revision, and Multi-Objective

MSP frameworks extend naturally to integer (binary) and mixed-integer domains, though these