Neural Surrogate Modeling of Recourse Functions

• Neural surrogate modeling of recourse functions is a method that uses neural networks to approximate expensive recourse evaluations in multi-stage decision problems.
• It employs architectures such as feed-forward ReLU networks and encoder–decoder Transformers to generate differentiable, efficient approximations that facilitate optimization.
• Empirical studies show high accuracy (e.g., <2.5% MAPE) and significant computational speed-ups, underscoring its potential in robust algorithmic recourse and stochastic programming.

Neural surrogate modeling of recourse functions refers to the use of neural networks as data-driven approximators for functions characterizing optimal or feasible responses in optimization, decision-making, or algorithmic recourse contexts, where explicit computation of such functions is expensive or infeasible. These surrogate models enable efficient, tractable, and often differentiable representations of operational subproblems, counterfactual mappings, or gradient-based interventions, making them central to robust algorithmic recourse and stochastic programming.

1. Mathematical Foundations and Problem Classes

Neural surrogate models for recourse are employed in problems exhibiting two-stage or multi-stage decision structures, where a first-stage “strategic” decision $x$ is followed by a second-stage or multi-horizon recourse action $(y)$ in response to random data $\xi$ or an automated classifier outcome. The canonical form in stochastic programming is

$$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$

where $Q(x)$ is the expected recourse cost, itself defined as the optimum over operational or corrective actions in each scenario (Zhang et al., 2 Dec 2025).

In algorithmic recourse, the goal is to map an unfavorable instance $x$ to a minimally-perturbed $x'$ that achieves a desired model outcome, with competing criteria for proximity (cost), plausibility (density), and validity (outcome) formalized as

$$x^*(x) = \arg\min_{x'} \lambda C(x, x') - \log P(x'|y^+) \quad \text{subject to} \quad P(y^+|x') > 0.5,$$

where $C(x, x')$ is a cost metric, $P(x'|y^+)$ is a class-conditional density, and $(y)$0 encodes validity (Garg et al., 12 May 2025).

2. Neural Surrogate Model Construction

Neural surrogate modeling is predicated on the empirical approximation of recourse functions via a neural network $(y)$1 or, in counterfactual recourse, an autoregressive conditional generator $(y)$2.

In stochastic programming (Zhang et al., 2 Dec 2025):

• Feed-forward, fully connected ReLU networks are trained on $(y)$3 data, where $(y)$4 is evaluated offline via exact solution of the recourse subproblem for sampled $(y)$5.
• Typical architectures use 2–3 hidden layers (e.g., 16–8–4, 32–16–8, or 64–32–16 neurons).
• Training employs mean squared error loss, $(y)$6, with stochastic gradient descent and $(y)$7 regularization.

In generative algorithmic recourse (Garg et al., 12 May 2025):

• GenRe constructs an encoder–decoder Transformer with causal self-attention for autoregressive modeling of $(y)$8.
• Output features are mixtures of RBF kernels on quantile-binned bins, enabling density modeling of $(y)$9.
• Training circumvents the absence of true $\xi$0 recourse supervision by using a “soft nearest neighbors” proxy $\xi$1, constructed over valid positive class instances.

3. Embedding Surrogates in Optimization and Inference

The neural surrogate can be integrated into the main optimization via explicit model linearization or efficient sampling:

• For recourse in MHSPs, surrogate networks are encoded as a system of linear and binary (“big-M”) constraints. Each ReLU neuron is represented by the introduction of auxiliary variables and indicators:

$\xi$2

This allows the composite problem (first-stage constraints plus neural surrogate) to be solved as a single MILP (Zhang et al., 2 Dec 2025).

• For generative recourse, inference is performed by forward sampling: for each $\xi$3, the model samples $\xi$4 candidate $\xi$5 by decoding one feature at a time using softmax-sampled bins and Gaussian noise, keeping the candidate with minimum cost under the validity constraint $\xi$6 (Garg et al., 12 May 2025).

4. Training Regimes and Theoretical Guarantees

In the absence of direct supervision for recourse mappings, synthetic supervision is generated using proxy distributions or importance-weighted sampling:

• For GenRe, $\xi$7 is constructed from “valid” positive instances in training data, weighted by recourse cost. The loss is the expected negative log-likelihood under $\xi$8, ensuring that the encoder–decoder learns to generate plausible, valid, and low-cost recourses.
• Theoretical guarantees (Theorem 3.1 of (Garg et al., 12 May 2025)) provide statistical consistency: for any test function $\xi$9, the difference $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$0 vanishes as the number of positive data points grows, provided classifier and density match on the support.

For stochastic programs, the surrogate network's generalization is managed via regularization, cross-validation, and embedding constraints to prevent overfitting. The network size (number of neurons/layers) directly influences both approximation quality (e.g., $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$1 and $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$2 MAPE in the UK power system case) and the computational burden of the MILP (Zhang et al., 2 Dec 2025).

5. Performance, Efficiency, and Trade-Offs

Key empirical results demonstrate that neural surrogates deliver substantial practical benefits:

Application	Surrogate Architecture	Approximation Quality	Computation Speed-Up	Robustness (out-of-sample)
Multi-horizon SP	32–16–8 ReLU FFN	$$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$31.7% MAPE	%%%%40$x$40%%%%511 (50 scenarios)	Comparable or improved vs. exact
Algorithmic Recourse	Transformer, RBF bins	Score $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$61.9/2; validity $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$7	Milliseconds/inference	Stable across $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$8 trade-off

Larger neural nets give finer approximation but increase binary variables in MILP, slowing optimization. A “sweet spot” exists (e.g., 32–16–8 network with $$\min_{x \in X} \; c^\top x + Q(x), \quad Q(x) = \mathbb{E}_\xi [q(x, \xi)],$$9670 binaries and low MAPE). For GenRe, sampling avoids online gradient or combinatorial search, making inference nearly instantaneous compared to search-based or robust baselines (Garg et al., 12 May 2025), and recourse recommendations are statistically consistent, plausible (high density), and cost-effective.

6. Generalizations and Extensions

The neural surrogate modeling paradigm generalizes across domains and objective classes:

• Stochastic Programs: The method applies to any two- or multi-stage stochastic program with complicated recourse; it suffices to construct $Q(x)$0 datasets and train a predictive neural net, which is then linearized and embedded as above (Zhang et al., 2 Dec 2025).
• Recourse in ML Systems: Generative neural models can encode cost, plausibility, and validity in counterfactual generation for any black-box classifier whose decision boundary and class-conditional densities are available or learnable (Garg et al., 12 May 2025).
• Extensions include approximating risk measures (e.g., CVaR surrogates), integrating scenario embeddings, and employing active learning to iteratively refine the surrogate by targeting uncertain regions.

A plausible implication is that as surrogate models become more expressive and easier to embed in optimization, the approach is likely to subsume classical explicit recourse evaluation in large-scale, uncertain, or data-centric domains.

7. Limitations and Practical Considerations

Accuracy-efficiency trade-off is inherent: over-parameterized surrogates risk overfitting and computational slowdown, while under-parameterized networks may lead to significant bias, especially in the tails of recourse distributions. In stochastic programming, the offline data generation phase (solving subproblems for many $Q(x)$1) can be computationally intense, but this cost pays dividends in dramatically reduced online solve time (up to $Q(x)$234.7 speed-up) and tractable embedding for large scenario sets (Zhang et al., 2 Dec 2025). In recourse, the lack of true counterfactual supervision demands robust synthetic proxy construction and careful evaluation of plausibility and validity metrics.

Objective evaluation on standardized metrics—cost, validity (fraction of favorable recourse), and plausibility (density/inlierness)—is essential for meaningful benchmark comparisons, as varying focus on these axes can dramatically influence qualitative behavior (Garg et al., 12 May 2025).

Neural surrogate modeling of recourse functions thus represents a unifying methodological advance at the intersection of statistical learning, discrete optimization, and operational research, with proven benefits for both robust individual recourse and large-scale, uncertain systems optimization.