Predict-Then-Optimize in Decision-Making

• Predict-then-optimize is a data-driven decision-making paradigm that estimates unknown parameters from features to drive optimal decisions in various optimization settings.
• It employs techniques such as surrogate losses, end-to-end differentiation, and solver-free methods to address the misalignment between prediction accuracy and decision performance.
• Applications span resource allocation, scheduling, and portfolio optimization, highlighting trade-offs between computational efficiency and decision quality.

Predict-then-optimize is a data-driven decision-making paradigm in which a predictive model estimates unknown parameters of an optimization problem from observable features, and the resulting predictions are then passed to a downstream solver to produce a decision. In its standard form, one observes contextual features and predicts uncertain objective coefficients before solving a linear, convex, combinatorial, or mixed-integer optimization problem; the central methodological issue is that predictive accuracy and downstream decision quality need not align (Elmachtoub et al., 2017). Across the literature, the framework is formulated either as prediction of latent optimization parameters followed by exact optimization, or more broadly as learning decisions from features under an optimization-induced regret objective (Kotary et al., 2024).

1. Formal setting and decision-theoretic objective

A canonical formulation considers a feature vector $x \in \mathbb{R}^p$, an unknown cost vector $y \in \mathbb{R}^d$, a feasible region $\mathcal S \subseteq \mathbb{R}^d$, and the downstream problem

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$

A predictor $f:\mathbb{R}^p\to\mathbb{R}^d$ produces $\hat y=f(x)$, after which the decision oracle returns

$$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$

The learning objective is then to choose $f$ so that the induced decision minimizes expected decision regret rather than ordinary prediction error (Wan et al., 17 Jun 2026).

In the linear-objective formulation developed in Smart Predict-then-Optimize, the nominal optimization problem is

$P(c):\quad z^\ast(c):=\min_{w\in S} c^\top w,$

with $S$ a known feasible region and $y \in \mathbb{R}^d$0 an unknown cost vector. The decisive point is that what matters is not numerical proximity of $y \in \mathbb{R}^d$1 to $y \in \mathbb{R}^d$2, but whether $y \in \mathbb{R}^d$3 is good under the true $y \in \mathbb{R}^d$4 (Elmachtoub et al., 2017). The same logic appears in multi-task PTO, where several related optimization problems

$y \in \mathbb{R}^d$5

are solved jointly from shared contextual information (Tang et al., 2022).

A broader generic formulation writes the optimization problem as

$y \in \mathbb{R}^d$6

where $y \in \mathbb{R}^d$7 denotes unknown problem parameters and $y \in \mathbb{R}^d$8 observable features. Standard two-stage PTO learns a predictor $y \in \mathbb{R}^d$9, forms $\mathcal S \subseteq \mathbb{R}^d$0, and computes $\mathcal S \subseteq \mathbb{R}^d$1; the true objective is to minimize downstream objective value or regret under the true parameters, not merely parameter-estimation loss (Kotary et al., 2024).

The literature repeatedly emphasizes an objective mismatch. Mean squared error or related predictive losses may be small even when the induced decision is poor, because small parameter errors can be amplified by the optimization step, because two predictions with similar $\mathcal S \subseteq \mathbb{R}^d$2 error can lie on opposite sides of a decision boundary, or because standard regression can spend model capacity fitting outcome levels that do not affect the action ranking (Elmachtoub et al., 2017). In black-box binary-decision PTO, this misalignment is particularly explicit: shifting the reward function by a context-dependent constant changes MSE but leaves the optimal action and regret unchanged (Tan et al., 2024).

2. Regret, SPO, and surrogate losses

The foundational decision loss for linear-objective PTO is the Smart Predict-then-Optimize loss

$\mathcal S \subseteq \mathbb{R}^d$3

or, in its unambiguous version,

$\mathcal S \subseteq \mathbb{R}^d$4

This loss measures the suboptimality of the decision induced by $\mathcal S \subseteq \mathbb{R}^d$5 under the true cost vector $\mathcal S \subseteq \mathbb{R}^d$6 (Elmachtoub et al., 2017). It is nonconvex and can be discontinuous, because the optimizer changes discontinuously when the predicted cost crosses decision boundaries (Elmachtoub et al., 2017).

To obtain a tractable surrogate, Smart Predict-then-Optimize derives the convex SPO+ loss

$\mathcal S \subseteq \mathbb{R}^d$7

which upper-bounds SPO, is convex in $\mathcal S \subseteq \mathbb{R}^d$8, and admits the subgradient

$\mathcal S \subseteq \mathbb{R}^d$9

In a binary special case, SPO reduces to $$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$0–$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$1 loss and SPO+ reduces to hinge loss up to scaling (Elmachtoub et al., 2017).

The same surrogate appears in later work as the default decision-focused loss for single-task and multi-task settings. Multi-task PTO uses SPO+ when cost labels are available and contrasts it with PFYL when only optimal solutions are observed (Tang et al., 2022). PyEPO packages SPO+ as one of its four base algorithms, alongside a differentiable black-box solver approach and two perturbation-based methods (Tang et al., 2022).

Perturbation-based surrogates define smoothed objectives by solving perturbed optimization problems. In PFYL, one defines

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$2

and then the Fenchel-Young loss

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$3

with Monte Carlo gradient approximation through perturbed argmins (Tang et al., 2022). These methods smooth the discrete decision map, but they typically require repeated solver calls during training (Tang et al., 2022).

Recent work has also introduced alternative surrogates that exploit geometric structure. The 2026 solver-free method observes two properties of decision loss for linear objectives: scale invariance in the prediction,

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$4

and linear scaling in the true cost,

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$5

From this, it derives a measure transformation and the Weight Integrated Spherical Error (WISE)

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$6

WISE focuses learning on the direction of the cost vector while weighting examples by $$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$7, thereby avoiding solver calls entirely during training (Wan et al., 17 Jun 2026).

In black-box binary PTO with only observed-action rewards, Empirical Soft Regret (ESR) replaces the non-differentiable regret indicator by a sigmoid-like soft term. In the paired-data case, the soft regret for context $$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$8 is

$$\min_{w \in \mathcal S}\ \mathbb{E}_P[Y^\top w \mid x] = \min_{w \in \mathcal S}\ \mathbb{E}_P[Y\mid x]^\top w.$$9

and the empirical loss uses nearest opposite-action neighbors when exact counterfactual pairs are unavailable (Tan et al., 2024). This extends regret-oriented training to a bandit-feedback-like supervised setting where the full reward surface is not observed.

3. Training paradigms: two-stage, end-to-end, solver-free, and proxy-based

The conventional two-stage pipeline predicts unknown parameters and then solves the nominal optimization problem with those predictions. Its main advantage is modularity, but the literature consistently treats it as potentially misaligned with regret minimization because the training loss ignores the downstream optimization map (Kotary et al., 2024).

End-to-end PTO, also described as decision-focused learning or Smart Predict-Then-Optimize, aligns the training objective with downstream decision quality by solving and differentiating the optimization problem in the learning loop (Kotary et al., 2024). In the generic parameter-prediction formulation this objective is

$f:\mathbb{R}^p\to\mathbb{R}^d$0

Its central limitation is computational and analytic: differentiating through $f:\mathbb{R}^p\to\mathbb{R}^d$1 requires problem-specific derivatives, typically from KKT conditions or implicit differentiation, and repeated solver calls can be prohibitive for discrete, mixed-integer, or nonconvex problems (Kotary et al., 2024).

Several strands of work seek to reduce or eliminate this dependence on exact differentiation through the solver. CaVE recasts decision-focused learning for binary linear programs as cone alignment: the predicted cost vector is trained to lie in the normal cone corresponding to the true optimal solution of the training instance. Its loss is the negative cosine similarity between $f:\mathbb{R}^p\to\mathbb{R}^d$2 and its projection onto the relevant normal cone or subcone,

$f:\mathbb{R}^p\to\mathbb{R}^d$3

with the projection defined via a nonnegative QP (Kotary et al., 2023). This avoids BLP solves during gradient computation and replaces them by cone-projection computations.

The 2026 WISE approach goes further by making the training loop entirely optimization-solver-free. Rather than approximate the optimizer or differentiate through it, it uses a change-of-measure argument to convert decision-focused learning into ordinary first-order training on a closed-form surrogate (Wan et al., 17 Jun 2026). Under boundedness assumptions, WISE is differentiable and convex in $f:\mathbb{R}^p\to\mathbb{R}^d$4 for fixed $f:\mathbb{R}^p\to\mathbb{R}^d$5, $f:\mathbb{R}^p\to\mathbb{R}^d$6-smooth, locally Lipschitz with constant $f:\mathbb{R}^p\to\mathbb{R}^d$7, and bounded by $f:\mathbb{R}^p\to\mathbb{R}^d$8 (Wan et al., 17 Jun 2026).

A separate research direction abandons parameter prediction altogether and learns decisions directly from features. Learning to Optimize from Features (LtOF) defines a direct feature-to-decision model

$f:\mathbb{R}^p\to\mathbb{R}^d$9

trained with an LtO objective adapted to features,

$\hat y=f(x)$0

The argument is that a pretrained optimization proxy inserted into an end-to-end predictor can fail badly because the predicted parameters encountered during training drift away from the distribution on which the proxy was trained, producing distribution shift and regret blow-ups (Kotary et al., 2024). LtOF resolves this by joint training or, more directly, by removing the intermediate parameter-prediction bottleneck entirely (Kotary et al., 2023).

Within LtOF, representative methods include Lagrangian Dual Learning, Self-Supervised Primal-Dual Learning, and Deep Constraint Completion and Correction, each of which enforces feasibility through dual penalties, augmented Lagrangian terms, or completion/correction mechanisms rather than exact solver differentiation (Kotary et al., 2024). This makes the approach applicable to nonconvex problems such as AC optimal power flow, where general end-to-end solver differentiation is not practical (Kotary et al., 2024).

4. Statistical properties, calibration, and generalization

A major theme in PTO theory is calibration: whether minimizing a surrogate loss implies minimizing the downstream decision loss. Smart Predict-then-Optimize proves that SPO+ is Fisher consistent with respect to SPO under mild conditions. In the population setting, if $\hat y=f(x)$1 is almost surely a singleton, $\hat y=f(x)$2 is centrally symmetric about its mean, $\hat y=f(x)$3 is continuous, and $\hat y=f(x)$4, then any minimizer of the SPO+ Bayes risk is almost surely equal to $\hat y=f(x)$5, which also minimizes the Bayes risk of SPO (Elmachtoub et al., 2017).

WISE establishes an analogous result through a different mechanism. Its Bayes minimizer is

$\hat y=f(x)$6

which is a scaled version of the conditional mean. Because the optimization oracle is scale-invariant,

$\hat y=f(x)$7

so minimizing WISE risk yields the same optimal downstream decision as the true conditional mean predictor (Wan et al., 17 Jun 2026). The paper states this as a Fisher consistency theorem and further proves the excess-risk bound

$\hat y=f(x)$8

This is a regret-calibration inequality: small excess WISE risk implies small excess decision regret (Wan et al., 17 Jun 2026).

Generalization theory for PTO is complicated by the fact that decision losses are nonconvex, non-Lipschitz, and sometimes discontinuous. One route, developed in the generalization-bounds literature, reduces PTO to a multiclass-style problem over extreme points of the feasible set and derives Natarajan-dimension bounds for SPO loss (Balghiti et al., 2019). Another route introduces a distance-to-degeneracy margin $\hat y=f(x)$9, a strength property for the feasible set, and a $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$0-margin SPO loss that upper-bounds SPO while becoming Lipschitz continuous (Balghiti et al., 2019). This yields multivariate Rademacher-complexity bounds that can improve dependence on feature and decision dimensions (Balghiti et al., 2019).

Dependent data require further modifications. For stationary $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$1-mixing time series, autoregressive PTO replaces the i.i.d. contextual setup by prediction from lagged observations and proves generalization bounds for SPO loss via independent-block arguments (Liu et al., 2024). The same work extends uniform calibration results to this dependent-data regime and reports that SPO+ outperforms absolute loss and least squares when cost vectors are generated by stationary dynamical systems, with normalized regret empirically related to mixing coefficients (Liu et al., 2024).

In black-box binary PTO, ESR proves an asymptotic optimality result in the paired-data setting. Under bounded reward gaps, controlled covering numbers, and a separation condition ensuring predictions are not asymptotically concentrated near the decision boundary, optimizing ESR yields asymptotically optimal regret within the model class as $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$2 for $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$3 (Tan et al., 2024).

5. Variants, extensions, and structural refinements

The PTO literature has expanded well beyond the single-task, static, linear-objective setting. Multi-task PTO generalizes the paradigm to multiple related optimization problems, possibly with different feasible regions, trained jointly through shared representations and task-specific heads (Tang et al., 2022). The framework distinguishes shared-cost and multi-cost settings, supports both learning from costs via SPO+ and learning from solutions via PFYL, and reports that multi-task learning is especially useful in the small-data regime and when the number of related tasks is large (Tang et al., 2022).

Sequential decision variants replace a single downstream optimization problem with an MDP solved by reinforcement learning. In this setting, a model predicts missing reward or transition parameters from environment features, and the predicted MDP is then solved via RL (Wang et al., 2021). Decision-focused training differentiates through MDP optimality conditions using sample-based unbiased derivative estimates and a low-rank Woodbury approximation to large policy Hessians, addressing the infeasibility of exact differentiation in large state and action spaces (Wang et al., 2021).

Several papers refine PTO by modifying the surrogate-loss-learning process itself. “Leaving the Nest” argues that prior learned-loss approaches are sample inefficient because they learn losses instance-by-instance and rely on a localness assumption that model predictions stay near the truth (Shah et al., 2023). It proposes Efficient Global Losses, combining Feature-based Parameterization, which maps features to loss parameters, with Model-based Sampling, which uses predictions from intermediate models rather than Gaussian perturbations of labels. The paper states that prior local learned losses are not Fisher consistent even in simple linear-objective PTO problems, whereas weighted MSE becomes Fisher consistent when the weights depend only on features (Shah et al., 2023).

Another refinement preserves the modularity of two-stage PTO while correcting for downstream asymmetry. Post-estimation adjustment for pricing studies objectives $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$4 that are asymmetric in estimation error and shows that a multiplicative correction

$$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$5

can uniformly and asymptotically improve standard PTO when a derivative-ratio condition

$$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$6

holds (Albert et al., 28 Jul 2025). The method is developed for linear, log-linear, and power-law demand models and produces $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$7 gains in expected reward under the stated assumptions (Albert et al., 28 Jul 2025). This suggests that PTO need not always be replaced by end-to-end training; in some structured problems, second-order post-estimation corrections can already exploit decision asymmetry.

The framework also extends to settings where the predictive object is not a cost vector in a standard optimization model. In continuous-treatment uplift modeling, the prediction stage estimates conditional average dose responses and conditional average dose effects, and the optimization stage solves an ILP that allocates doses under a budget, with optional fairness and cost-sensitive constraints (Vos et al., 2024). In black-box binary PTO, the prediction target is a reward function observed only for chosen actions rather than all counterfactual actions (Tan et al., 2024). These cases preserve the same architectural principle—predict uncertain quantities, then optimize a constrained decision—but alter the statistical and optimization interfaces substantially.

6. Applications, software, and empirical patterns

PTO has been applied to a wide range of domains. Canonical academic benchmarks include shortest path, knapsack, matching, traveling salesperson, portfolio optimization, and other linear or combinatorial problems (Wan et al., 17 Jun 2026). Multi-task PTO focuses on routing benchmarks such as multiple TSPs, multiple shortest paths, graph routing, and Warcraft pathfinding (Tang et al., 2022). Proxy-based and solver-free approaches are evaluated on portfolio optimization, a nonconvex QP with a sinusoidal term, and AC optimal power flow (Kotary et al., 2024). Sequential PTO is studied on gridworld, snare-finding, and tuberculosis treatment MDPs (Wang et al., 2021).

Industrial and domain-specific deployments emphasize the modularity of PTO. In online fund recommendation, PTOFA predicts expected revenue for each customer–fund pair and then solves a constrained allocation problem with customer top-$$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$8, fund exposure, and risk constraints (Tang et al., 5 Mar 2025). Its optimization heuristic reaches $$w^*(\hat y)\in\arg\min_{w\in\mathcal S}\hat y^\top w.$$9 of IP’s objective score on 200,000 users and yields up to $f$0 CPME lift and $f$1 RPME lift in online A/B tests (Tang et al., 5 Mar 2025). In clinician scheduling, a logistic-regression availability predictor augmented by FLAN-T5-base note extraction feeds a lexicographic multi-objective MIP balancing cFTE compliance, equitable shift-type proportions, predicted availability, and schedule consistency (Jha et al., 2 Oct 2025). PTO-CS achieves $f$2 coverage across six evaluated months while reducing cFTE deviation relative to historical schedules (Jha et al., 2 Oct 2025).

Other applications use prediction to parameterize more elaborate optimization or control layers. Wildfire drone-swarm planning predicts fire spread with Convex-NN models and then solves a MIP combined with dynamic programming and chance-constrained robust optimization, with Benders Decomposition and Branch-and-Cut used for computation (Pan et al., 2024). Equitable post-disaster power restoration uses equity-conformalized quantile regression for repair-duration prediction and a constrained RL decision layer that optimizes efficiency subject to a Wasserstein inequity constraint; EPOPR reduces average outage duration by $f$3 and inequity by $f$4 relative to state-of-the-art baselines (Jiang et al., 6 Aug 2025).

A substantial practical contribution is PyEPO, a PyTorch-based library for end-to-end PTO with linear and integer programming and predicted objective coefficients (Tang et al., 2022). PyEPO provides implementations of SPO+, a differentiable black-box solver approach, and two perturbation-based methods, together with interfaces for defining optimization models and comparing end-to-end and two-stage approaches (Tang et al., 2022).

Several empirical patterns recur across the literature. Lower prediction error does not reliably imply better decision quality: this conclusion appears in SPO, PyEPO, black-box ESR, simulation-based PTO evaluation for classification, and continuous-treatment uplift (Elmachtoub et al., 2017). Multi-task sharing, adaptive weighting, and MSE regularization can improve regret in cost-labeled settings, especially with limited data (Tang et al., 2022). Solver-free or proxy-based methods often trade slight approximation for large runtime gains: WISE is reported to be roughly 4 to 100 times faster than decision-focused learning methods that differentiate through the solver on shortest path (Wan et al., 17 Jun 2026), while CaVE variants make previously impractical large CVRP instances trainable by replacing BLP solves with cheaper cone projections (Kotary et al., 2023).

7. Recurring misconceptions, limitations, and unresolved tensions

A common misconception is that PTO is justified whenever prediction is accurate enough. Multiple studies instead show that the relationship between predictive metrics and solution quality is non-trivial. Simulated-classifier experiments on scheduling explicitly report that better classification metrics do not always imply better optimization outcomes and that FPR can matter more than TPR for downstream objective value (Smet, 2 Sep 2025). Continuous-treatment uplift reports that better CADR prediction error does not always imply better policy value (Vos et al., 2024). ESR argues that accuracy-based metrics such as MSE can be distracted by aspects of the response surface irrelevant to action selection (Tan et al., 2024).

Another recurring tension concerns the role of exact solver differentiation. End-to-end PTO aligns training with the evaluation objective, but it often requires handcrafted gradients and repeated solver calls (Kotary et al., 2024). Solver-free WISE, cone-aligned CaVE, and LtOF-by-proxy all respond to this limitation, but they do so by changing what is learned: a smooth decision-calibrated surrogate, a geometric cone-alignment objective, or a direct feature-to-decision map (Wan et al., 17 Jun 2026). A plausible implication is that the field no longer treats differentiation through the exact optimizer as the unique route to decision-aware learning.

Proxy methods introduce their own failure mode: distribution shift. Both proxy-based PtO papers report that a pretrained optimization proxy used inside an end-to-end predictor can degrade badly when predictor updates drive inputs outside the proxy’s training distribution (Kotary et al., 2024). Learned-loss methods face an analogous issue when local perturbation sampling fails to reflect the actual predictions produced by a misspecified model (Shah et al., 2023).

Theoretical guarantees also remain regime-dependent. SPO+ consistency requires symmetry and uniqueness conditions (Elmachtoub et al., 2017). WISE assumes bounded predictor outputs, feasible decisions, and costs, together with $f$5 almost surely (Wan et al., 17 Jun 2026). Post-estimation adjustment is specific to objective asymmetry governed by a derivative-ratio condition and is not presented as a universal PTO fix (Albert et al., 28 Jul 2025). Dependent-data calibration weakens as mixing slows, especially near marginally stable dynamics (Liu et al., 2024). These results do not undermine PTO, but they indicate that statistical validity is tightly coupled to optimization geometry and data-generating structure.

The most stable conclusion across the surveyed work is narrower and more technical than the broad slogan “predict better to decide better.” PTO research instead shows that decision-making systems benefit when the prediction stage is designed, trained, regularized, calibrated, or post-corrected with explicit reference to the downstream optimization map. That claim is shared by Smart Predict-then-Optimize, solver-free WISE, proxy-based LtOF, ESR, multi-task PTO, and later application papers, even though they implement it through markedly different statistical and computational mechanisms (Elmachtoub et al., 2017).