Sampler-Robust Optimization under Generative Models

Abstract: Modern stochastic optimization pipelines increasingly rely on learned generative models to represent uncertainty, while downstream decisions are evaluated almost entirely through Monte Carlo scenarios. This shifts the operational object of uncertainty from an explicit probability law to the sampler induced by the learned generator. Reliability therefore depends on two errors: sampler misspecification and finite-simulation error. We propose Sampler-Robust Optimization (SRO), which optimizes decisions against the worst-case sampler induced by perturbing the learned generator. This sampler-first formulation aligns with simulation-based decision pipelines and admits a sharpness-aware interpretation: it favors decisions whose performance is stable under generator perturbations, rather than merely under the nominal sampler. Under a coverage assumption, we show that the empirical worst-case objective provides a high-probability upper certificate for the true population objective, with finite-simulation error partially absorbed by the robustification used to guard against sampler misspecification. The framework accommodates generative models with or without explicit densities and admits efficient minimax procedures. Portfolio-optimization experiments show that SRO produces more stable decisions and improves out-of-sample performance under distribution shift.

• The paper proposes SRO, a robust optimization technique that mitigates generator misspecification and finite-simulation error by optimizing directly over perturbations in the sampler.
• It introduces two efficient minimax solution procedures—a first-order local approximation for small perturbations and a projected alternating gradient method for larger radii—to stabilize high-dimensional decision problems.
• Empirical evaluations demonstrate that SRO reduces over-optimism and improves risk metrics such as the Sharpe ratio and CVaR, providing more reliable out-of-sample performance.

Sampler-Robust Optimization under Generative Models: A Technical Review

Motivation and Problem Setting

Modern stochastic optimization pipelines increasingly integrate learned generative models as the principal mechanism for simulating uncertainty, particularly in settings such as portfolio optimization, supply chain management, and energy systems. While classical methods often operate on explicit probability distributions, practical decision-making workflows typically interact with generative models solely through their induced samplers—using Monte Carlo simulation to evaluate candidate actions. This poses two fundamental reliability challenges:

• Sampler Misspecification: The generator parameter $\hat\theta$, learned from finite data, may diverge from the true (often unobservable) data-generating parameter $\theta^\star$.
• Finite-Simulation Error: Decision quality is estimated from finitely many samples, resulting in statistical uncertainty.

Traditional distributionally robust optimization (DRO) approaches construct ambiguity sets over probability laws and impose robustness in the likelihood space. However, for highly expressive or implicit generative models (notably GAN architectures without tractable densities), this perspective becomes less operationally meaningful. In such applications, robustness should be imposed directly at the level of the sampler. The Sampler-Robust Optimization (SRO) framework developed in the paper addresses this gap.

Sampler-Robust Optimization Formulation

SRO recasts the robust optimization problem by focusing on worst-case performance across a neighborhood of generators in parameter space rather than over an ambiguity set of distributions. The critical objects are:

1. Generator Model: $y = G_\theta(z, x)$, where $z$ is latent noise and $x$ is context.
2. Nominal Population Objective: $$J(\omega; x) = \mathbb{E}_{z \sim \nu}[f(\omega, G_{\hat\theta}(z, x))]$$.
3. Worst-Case Sampler Neighborhood: $$\Theta_\rho(\hat\theta) = \{ \theta : \| \theta - \hat\theta \| \leq \rho \}$$, a norm-ball around the learned generator.

The SRO rule seeks

$$\hat{\omega}(x) \in \arg\min_{\omega \in \mathcal{W}} \sup_{\theta \in \Theta_\rho(\hat\theta)} \frac{1}{N} \sum_{i=1}^N f\left(\omega, G_\theta(z_i, x)\right),$$

directly mirroring the simulation-based decision pipeline. Robustification corresponds to penalizing for "sampler-sharpness": decisions are discouraged if their performance deteriorates sharply for small generator perturbations.

Theoretical Guarantees and Regimes

A central result is a uniform finite-sample bound: under a natural "coverage" assumption (the oracle sampler is within the neighborhood), the maximized empirical utility is shown to constitute a high-probability upper certificate for the true expected performance. The correction for simulation error is partially offset by the slack introduced through robustification, and crucially, the error term does not scale with the dimension of the generator parameter space.

Three operating regimes for the perturbation radius $\rho$ are distinguished:

1. Small $\rho$: Minimal robustness; standard Monte Carlo error dominates.
2. Moderate $\theta^\star$0: Robustification slack partially absorbs simulation error; balanced trade-off.
3. Large $\theta^\star$1: Conservative decisions; robustification slack saturates the simulation error, but further increases can overly sacrifice nominal performance.

Efficient Minimax Solution Procedures

Direct minimax optimization over high-dimensional generator and decision spaces incurs computational complexity. Two tractable refinement regimes are proposed:

• First-Order (Local) Approximation: For small $\theta^\star$2, the inner maximization over generator parameters is linearized, enabling analytic adversarial updates via dual norm calculations. This approach is computationally efficient for slight robustness.
• Projected Alternating Minimax Optimization: For larger $\theta^\star$3, local approximation breaks down. The authors employ two-timescale alternating gradient steps: adversarial generator updates use a small step (to stay within the norm-ball), while the portfolio decision updates proceed at faster pace. This strategy stabilizes the minimax dynamics in high-dimensional, non-convex settings.

Architectural Design and Implementation

The SRO framework is instantiated over a conditional LSTM-based GAN for the underlying generative model, with separate architectures for the generator and discriminator.

Figure 1: Conditional LSTM-based generator architecture, mapping a rolling window of historical log-returns and latent noise to one-step-ahead simulated returns.

Figure 2: Conditional LSTM-based discriminator (critic) used in adversarial training, distinguishing real from generated returns conditioned on historical input.

Calibration is by least-squares GAN objectives, which improves stability over traditional cross-entropy losses in financial time series contexts.

Empirical Evaluation

Controlled Generator-to-Generator Experiment

Here, the nominal generator is purposely misspecified relative to an "oracle" generator, which remains observable for diagnostic purposes. The evaluation examines:

• Empirical, oracle, and empirical-to-oracle gaps in utility
• Risk metrics: Sharpe ratio, Conditional Value-at-Risk (CVaR), Maximum Drawdown

  Figure 3: Utility-based diagnostics demonstrate the empirical-to-oracle gap closing effect of SRO versus nominal optimization.

  

  Figure 4: Out-of-sample risk metrics show SRO yields consistently higher Sharpe ratios, reduced drawdowns, and improved CVaR compared to nominal optimization.

Strong findings include:

• The empirical criterion of the nominal method overstates utility relative to oracle performance. SRO drastically reduces this over-optimism, yielding a nearly zero (occasionally slightly negative) empirical-to-oracle gap.
• Risk metrics are markedly improved under SRO relative to nominal: Sharpe ratio, CVaR(5%), and maximum drawdown all reflect stricter downside control and more reliable out-of-sample performance.

Real-Data Portfolio Backtest

Randomized asset pools from a broad S&P 500 sample are used to test generalization and robustness under genuine regime shifts. SRO consistently outperforms the nominal benchmark in:

• Mean and risk-adjusted returns (Sharpe ratio)
• Downside risk measures (CVaR, maximum drawdown)

Implications, Limitations, and Future Directions

The theoretical and empirical results demonstrate that robustification at the sampler level, rather than the likelihood or probability-law level, is principled and operationally relevant for simulation-driven, high-parametric generative models. The SRO approach is notable for producing decisions that are demonstrably more stable under model misspecification and subject to distributional regime change.

The framework is especially potent where explicit density estimation is intractable, e.g., in adversarially trained GANs for time series or scenarios where only sampling-based uncertainty representation is feasible.

Potential limitations and open directions:

• Scalability: For extremely large generator models (e.g., transformer-based simulators), the efficiency and accuracy of the projected minimax procedure merit further study.
• Distributional Coverage: The performance of SRO hinges on the oracle parameter being in the robustification set. Techniques for adaptive radius selection and coverage diagnostics are critical for deployment in non-parametric and online settings.
• Extension to Constraints: Generalizing SRO to constraints-driven problems—where constraints are themselves evaluated via simulation—may further broaden applicability.

Conclusion

Sampler-Robust Optimization (SRO) offers a robust, theoretically grounded alternative for decision-making under learned generative models. By imposing worst-case robustness directly at the level of the decision pipeline's operational object—the generator-induced sampler—the framework harmonizes with the realities of contemporary simulation-based optimization and highlights a nuanced trade-off between conservatism and statistical reliability. The results strongly support the SRO paradigm as a compelling next step for robust AI-driven stochastic optimization under model uncertainty in high-dimensional practical scenarios (2604.27447).