Constrained Stochastic Shortest Paths

• CSSPs are optimization models that minimize a primary path cost while enforcing hard constraints on secondary costs, risk levels, and success probabilities.
• They integrate methodologies from Markov decision processes and network path analyses, employing techniques like dynamic programming, linear programming, and heuristic search.
• Applications span risk-sensitive planning, intelligent intersection management, and random-graph models, highlighting trade-offs between cost minimization and strict feasibility requirements.

Constrained stochastic shortest path problems (CSSPs) augment the classical stochastic shortest path problem by optimizing a primary path or policy cost subject to hard constraints on secondary costs, probabilities of failure, or risk functionals. In the literature, the term covers at least two closely related settings: network path problems with random arc attributes, and Markov decision process formulations with absorbing goal states, stochastic transitions, and additive stage costs. The common feature is that feasibility is no longer determined by reachability alone: it is defined by budgets, chance constraints, CVaR bounds, percentile constraints, or hard task specifications, and the corresponding solution methods range from algebraic path algorithms and LP/ILP formulations to stochastic approximation, heuristic search, and asymptotic probabilistic analysis (Parmentier, 2015, Khonji, 2022, Alyassi et al., 2023).

1. Formal models and problem classes

A standard stochastic shortest path problem is typically formulated on an MDP $M = (S, A, P, c, s_0, G)$ with finite state space $S$, admissible actions $A(s)$, transition probabilities $P(s' \mid s,a)$, nonnegative one-step costs $c(s,a)$, and a terminal or goal set $G \subset S$. For proper policies, the canonical Bellman equation for the cumulative-sum objective is

$$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$

Under the standard SSP assumptions—existence of at least one proper policy and infinite expected cost for all improper policies—value iteration converges monotonically to the unique fixed point $V$ (Zhang et al., 14 Dec 2025).

In constrained variants, the objective is paired with secondary cost limits. A fixed-horizon constrained MDP formulation, which directly captures a class of CSSPs, minimizes expected cumulative primary cost subject to bounds on secondary cumulative costs or on the probability of violating a safety condition. In the dead-end interpretation of SSPs, a typical formulation is

$$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$

which exhibits the cost-to-goal versus probability-to-goal trade-off explicitly (Khonji, 2022).

A closely related undiscounted CSSP formulation uses absorbing goals and multiple stage-cost components. With initial state $\bar{s}$, goal set $S$0, primary cost $S$1, secondary costs $S$2, and thresholds $S$3, one writes

$S$4

and solves

$S$5

This form is central in hierarchical constrained SSP planning (Hong et al., 2022).

The network-path viewpoint is structurally different. There, one chooses an $S$6–$S$7 path offline in a digraph or complete graph, and resources accumulate along arcs. In the lattice-ordered monoid framework, each arc $S$8 carries a resource $S$9, the path resource is

$A(s)$0

and the generalized problem is

$A(s)$1

This formulation covers deterministic, non-linear, and stochastic constraints and objectives, including chance constraints, CVaR, and scenario-based random resources (Parmentier, 2015). It contrasts with MDP-based SSPs, where one optimizes over policies under controlled transitions rather than over fixed paths.

2. Constraint semantics and risk models

CSSPs differ most sharply in the semantics of the constraint. Expected-value constraints are the closest analogue to classical constrained MDPs, but much of the literature focuses on distributional or pathwise requirements. Chance constraints are standard: $A(s)$2 for cumulative path cost $A(s)$3, threshold $A(s)$4, and risk level $A(s)$5 (Enwerem et al., 2024). In fixed-horizon SSPs with dead-ends, the same idea appears as a failure-probability constraint: policies are restricted to

$A(s)$6

so that the probability of never reaching the terminal state remains below a prescribed tolerance (Otsubo, 2024).

Percentile constraints constitute another important class. For truncated-sum cost $A(s)$7 until reaching a target set $A(s)$8, a single percentile constraint is

$A(s)$9

and the multi-constraint, multi-dimensional extension requires

$P(s' \mid s,a)$0

where each $P(s' \mid s,a)$1 may refer to a different dimension and target set (Randour et al., 2014). This formulation directly controls quantiles of path-length distributions rather than expectations.

CVaR-based CSSPs replace hard probability thresholds by tail-expectation control. With confidence level $P(s' \mid s,a)$2,

$P(s' \mid s,a)$3

and

$P(s' \mid s,a)$4

CSSPs then take forms such as $P(s' \mid s,a)$5, or optimize $P(s' \mid s,a)$6 directly (Enwerem et al., 2024, A., 2014). In network models with independent random arc times, the same risk functional is used for path travel time $P(s' \mid s,a)$7, yielding problems of the form

$P(s' \mid s,a)$8

for any risk measure $P(s' \mid s,a)$9 consistent with the usual stochastic order (Parmentier et al., 2014).

Several works broaden CSSP semantics further. Beyond-worst-case formulations combine a worst-case upper bound with an expected-value bound on the same truncated sum (Randour et al., 2014). Integer-weighted MDP formulations study weight-bounded reachability conditions such as

$c(s,a)$0

and ask whether such properties hold almost surely or with positive probability under some scheduler (Baier et al., 2018). More recent formulations incorporate hard task specifications through finite-horizon linear temporal logic, product MDPs, and strict lexicographic priorities among mixed max- and sum-aggregated objectives, thereby turning safety or sequencing requirements into hard constraints rather than soft penalties (Zhang et al., 14 Dec 2025).

A persistent misconception is that all CSSPs are simply expected-cost CMDPs. The literature does not support that view. Chance constraints, percentile constraints, CVaR, weight-bounded reachability, dead-end avoidance, and lexicographic hard priorities are all treated as genuinely distinct constraint mechanisms, with different Bellman structures, policy classes, and computational consequences (Randour et al., 2014, Baier et al., 2018).

3. Solution methods and algorithmic frameworks

One major algorithmic line treats CSSPs as generalized path problems. In the lattice-ordered monoid approach, strong pruning comes from lower bounds on suffix resources. For each node $c(s,a)$1, the best-possible suffix bound is

$c(s,a)$2

and partial path $c(s,a)$3 ending at $c(s,a)$4 can be discarded whenever either

$c(s,a)$5

This yields generalized A$c(s,a)$6, label-correcting, and label-dominance algorithms. Exact computation of $c(s,a)$7 is NP-hard in general, but extended Ford–Bellman and generalized Dijkstra procedures provide polynomially computable lower bounds that are effective in practice (Parmentier, 2015).

In stochastic network models with random arc lengths, pruning is strengthened by stochastic lower bounds. The key object is the Stochastic On-Time Arrival Problem, whose dynamic programming recursion is

$c(s,a)$8

The resulting random variables $c(s,a)$9 satisfy

$G \subset S$0

and allow label-setting algorithms for both risk minimization and deterministic-cost minimization under a stochastic risk constraint. The framework applies to risk measures consistent with the usual stochastic order, including CVaR, VaR, and the probability of being late (Parmentier et al., 2014).

Another standard technique is state augmentation. Percentile constraints on truncated sums are handled by constructing a product MDP that tracks cumulative cost up to a threshold $G \subset S$1, so that the original percentile query reduces to a reachability problem on the product state space. The same principle supports multi-percentile constraints, beyond-worst-case synthesis, and multi-environment guarantees, although the required memory and randomization can grow substantially (Randour et al., 2014). For weight-bounded properties in integer-weighted MDPs, end-component classification, spider constructions, and reductions to mean-payoff games yield pseudo-polynomial or polynomial algorithms depending on the quantification pattern and the property considered (Baier et al., 2018).

Linear programming and integer programming dominate exact constrained MDP formulations. The standard occupancy-measure LP handles expected-value constraints, while exact chance-constrained SSP formulations require additional structure. In fixed-horizon layered models, chance constraints can be linearized through execution-risk recursions. This leads to exact ILP formulations with flow variables and binary policy-selection variables, as well as LP relaxations with randomized rounding. The same line of work shows that multi-step global chance constraints can be reduced to local chance constraints through augmented states, with a polynomial-sized approximation via resource augmentation (Alyassi et al., 2023). Under local transition structure, a different approximation route is available: a fully polynomial-time approximation scheme computes near-optimal deterministic policies by discretizing objective-to-go values and solving structured knapsack subproblems inside a layered dynamic program (Khonji, 2022).

Risk-sensitive optimization has also been attacked by gradient methods. For CVaR-constrained SSPs, the Lagrangian

$G \subset S$2

is optimized with respect to policy parameters $G \subset S$3 and multiplier $G \subset S$4. The crucial ingredients are likelihood-ratio policy-gradient estimators, Rockafellar–Uryasev CVaR estimation, multi-timescale stochastic approximation or mini-batches, and importance sampling for rare-event tail estimation. The resulting algorithms converge asymptotically to local saddle points of the Lagrangian (A., 2014).

4. Mean-field random CSSPs and asymptotic analysis

A distinctive strand of the CSSP literature studies random complete graphs. On $G \subset S$5 with fixed source and target vertices, each edge $G \subset S$6 carries independent random length $G \subset S$7 and cost $G \subset S$8, and the constrained objective is

$G \subset S$9

In the base case $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$0 are independent Uniform$$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$1, while the generalization takes $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$2 to be independent copies of $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$3 with $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$4 (Frieze et al., 2020).

The central asymptotic statement in the uniform case is

$$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$5

for budgets $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$6 in a wide admissible window. The hop-count $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$7 of the optimal constrained path satisfies $$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$8. A key lower bound is the universal product inequality

$$V(s) = \min_{a \in A(s)} \left( c(s,a) + \sum_{s' \in S} P(s'\mid s,a)\, V(s') \right), \quad s \notin G;\qquad V(s)=0,\ s\in G.$$9

which implies

$V$0

The matching upper bound is obtained by Lagrangian relaxation with modified edge weights $V$1 and an optimally chosen multiplier $V$2 (Frieze et al., 2020).

The dual analysis links the constrained problem to mean-field first-passage percolation. Near zero, $V$3 behaves like a square-root tail, so the unconstrained shortest path under $V$4 is asymptotically governed by the $V$5 first-passage model. This yields

$V$6

and hence a feasible path with asymptotically optimal constrained length (Frieze et al., 2020).

For $V$7 edge weights, the paper proves

$V$8

with an explicit $V$9-dependent constant expressible using $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$0. The small-$$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$1 tail $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$2 is therefore the governing parameter. Heavier near-zero tails yield shorter unconstrained paths and tighter constrained bounds, while lighter tails lengthen both (Frieze et al., 2020).

This random-graph analysis makes the budget penalty explicit. In the uniform case, the unconstrained shortest path is of order $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$3, whereas the constrained optimum is

$$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$4

Thus, for $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$5, the constraint is strongly binding; for $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$6, the unconstrained shortest path is within budget with high probability (Frieze et al., 2020). The same paper also notes an expected polynomial-time route in this random setting, combining Lagrangian relaxation with dynamic programming over hop counts and random-knapsack techniques, leading to an $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$7 expected-time algorithm.

5. Structured extensions: hierarchy, multi-agent coordination, sampling, and specifications

Hierarchical constrained SSPs arise when high-level actions must themselves be planned as lower-level SSPs. In this setting, events act as macro-level states, activities act as macro-actions, and global secondary-cost constraints couple multiple lower-level problems. A hierarchical solution is a pair $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$8, where $$\min_{\pi} \ \mathbb{E}_{\pi}\Big[\sum_{t=0}^{H-1} c_t(S_t,\pi(S_t))\Big] \quad \text{s.t.}\quad \mathbb{P}_{\pi}(S_H = g) \ge 1-\delta,$$9 is a procedural policy over events and $\bar{s}$0 collects lower-level policies $\bar{s}$1 for all activated activities. The global constraint has the form

$\bar{s}$2

and an anytime branch-and-bound algorithm searches over allocations of secondary-cost budgets to activities. Lower bounds come from optimistic activity costs and conservative secondary costs; upper bounds come from feasible incumbent activity policies. Under exact activity lower bounds, the branch-and-bound gap converges uniformly to zero as partitions shrink (Hong et al., 2022).

Multi-agent chance-constrained SSPs appear prominently in intelligent intersection management. There the system is decomposed into localized interaction points $\bar{s}$3, each coupling only a small subset of vehicles $\bar{s}$4. The crucial execution-risk recursion at interaction point $\bar{s}$5 is

$\bar{s}$6

with

$\bar{s}$7

This recursion enables an exact ILP in dual-flow variables. When the number of agents per interaction point is bounded, the formulation has a polynomial number of variables and constraints in the number of agents. Simulations in CARLA report up to $\bar{s}$8 efficiency improvement while conforming to tunable risk thresholds (Khonji et al., 2022).

Sampling-based planners provide another structured extension. In risk-sensitive incremental sampling, RA-RRT$\bar{s}$9 replaces expected-length cost-to-come by a CVaR cost-to-come. Under additive costs, independent segment uncertainties, fixed $S$00, and standard RRT$S$01 rewiring schedules, the planner uses

$S$02

and selects expansions that minimize cumulative segment-wise CVaR. In the reported grid-world experiments, query time and space remain linear in $S$03, processing time becomes $S$04, and the planner exhibits lower noise sensitivity and lower failure rates than baseline RRT$S$05 (Enwerem et al., 2024).

Dead-end-aware formulations treat failure probability itself as the constrained quantity. By introducing discounted surrogates for the failure event and coupling an objective MDP with a constraint MDP in a two-person zero-sum game, the constrained problem becomes a Bayesian MDP over an augmented belief state. Value iteration on the operator

$S$06

computes the game value and yields near-optimal mixed policies expressible as distributions over deterministic policies (Otsubo, 2024).

Hard specifications can also be internalized. Finite-horizon LTL is translated into a deterministic finite automaton, combined with the MDP into a product MDP, and then optimized lexicographically under mixed max- and sum-aggregated costs. Because max aggregation breaks the standard Bellman recursion on the original state space, the model augments the state with the running maximum cost $S$07. The resulting finite-horizon lexicographic value iteration handles bottleneck-style safety objectives together with cumulative efficiency objectives and resolves zero-marginal-cost cycles that otherwise obstruct proper termination (Zhang et al., 14 Dec 2025).

6. Complexity, policy structure, and open directions

Complexity results across the CSSP literature are uniformly stringent. Deterministic CSSP is NP-hard in general (Frieze et al., 2020). Exact computation of best suffix resource bounds $S$08 is NP-hard in the monoid framework (Parmentier, 2015). Even under local transition structure, deterministic-policy constrained MDPs are NP-hard for horizon $S$09 (Khonji, 2022). Percentile SSPs are PSPACE-hard, beyond-worst-case synthesis is NP-hard, and general multi-percentile queries require exponential time (Randour et al., 2014). For weight-bounded reachability in integer-weighted MDPs, several decision problems lie in $S$10 and are at least as hard as mean-payoff games (Baier et al., 2018).

Policy structure is equally nuanced. In classical expectation-minimizing SSPs, pure memoryless optimal policies exist under standard assumptions (Zhang et al., 14 Dec 2025). That simplicity does not survive most constrained extensions. For single percentile constraints and beyond-worst-case SSPs, finite memory is sufficient and sometimes necessary; for multi-percentile constraints, randomized exponential-memory strategies are sufficient and in general necessary (Randour et al., 2014). By contrast, several applied formulations deliberately restrict attention to deterministic policies for explainability, reliability, or solver tractability, including hierarchical constrained SSP planning and exact chance-constrained SSP ILPs (Hong et al., 2022, Alyassi et al., 2023).

Most tractable formulations also rely on strong structural assumptions. Independence is central in random-graph asymptotics, stochastic lower bounds via convolution, and CVaR decomposition across path segments (Frieze et al., 2020, Parmentier et al., 2014, Enwerem et al., 2024). Gaussian segment noise, bounded numbers of interacting agents, local transition overlap, finite supports on integer grids, or fixed horizons each play an enabling role in specific methods (Khonji et al., 2022, Khonji, 2022). When these assumptions fail—because of correlation, non-additive path costs, dense interaction graphs, or unbounded continuous resources—the existing theory becomes much thinner.

Several open directions are identified explicitly. In the random complete-graph model, correlated edge lengths and costs remain untreated, as do sparse random-graph analogues with asymptotically sharp constants (Frieze et al., 2020). In CVaR-based sampling planners, correlated or non-Gaussian uncertainties, distributionally robust CVaR, and non-additive cost structures require new constructions (Enwerem et al., 2024). In integer-weighted MDPs, the literature emphasizes that quantitative thresholds $S$11 for weight-bounded properties remain hard and open (Baier et al., 2018). More broadly, the coexistence of exact ILP methods, pseudo-polynomial dynamic programs, approximation schemes, and asymptotic probabilistic analyses suggests that “CSSP” is best understood not as a single problem, but as a family of constrained stochastic path-planning models whose mathematical character is determined by the interaction of constraint semantics, policy class, and ambient stochastic structure.