Decision-Focused Learning: When and Why Traditional Prediction Models Fail

Abstract: Plugging predictions of unknown parameters into downstream optimization problems, often referred to as the ``predict-then-optimize'' paradigm, has long been a standard approach in decision-making under uncertainty. However, improved predictive accuracy does not, in general, translate into improved decision quality. This disconnect has motivated growing interest in decision-focused learning (DFL) within the operations research community. This tutorial reviews recent developments in DFL and highlights key methodological insights, with a particular focus on stochastic linear programming as the downstream decision-making problem. We discuss why several widely used tools in traditional statistical learning are not directly suited to decision-focused settings and must be rethought, including (i) data collection strategies driven purely by predictive uncertainty and (ii) distributional distance measures such as the Wasserstein distance. We summarize properties of DFL that distinguish it from conventional predictive modeling and provide insights into the development of new decision-focused tools.

• The paper shows that traditional prediction minimizes error but can fail to yield optimal decisions due to misaligned loss functions.
• It introduces decision-focused training objectives, such as the SPO+ loss, to emphasize errors near decision boundaries and improve decision quality.
• The work outlines decision-aware data acquisition and distribution metrics that enhance downstream decision making in complex operations research settings.

Decision-Focused Learning: Revisiting Statistical Learning for Operations Research

Introduction

The paper "Decision-Focused Learning: When and Why Traditional Prediction Models Fail" (2606.21773) provides an authoritative survey of methodologies and theoretical underpinnings of Decision-Focused Learning (DFL) in the context of decision-making under uncertainty, particularly for Operations Research (OR). The central theme is the disconnect between predictive accuracy and decision quality in the canonical "predict-then-optimize" paradigm: minimizing prediction error does not guarantee improved downstream decisions. This mismatch motivates a shift to decision-focused training objectives, requiring revisiting model selection, data collection, and distributional comparison through a decision-aware lens.

Classical Prediction versus Decision-Focused Approaches

Traditional statistical learning pipelines involve fitting predictive models (e.g., minimizing $\ell_2$-loss) for uncertain parameters and subsequently plugging these estimates into optimization tasks. The theoretical justification for this approach holds only under linear objective uncertainty, where the conditional mean prediction suffices for optimality. However, this fails for most real-world settings, including nonlinear objectives and stochastic constraints, as demonstrated in several canonical examples (e.g., newsvendor, portfolio optimization).

Figure 1: A shortest-path toy example illustrating that superior predictive fit does not necessarily yield optimal decisions—decision boundaries hinge on context-sensitive thresholds, not global prediction error.

The paper decomposes these failures into three classes:

• Mismatch of prediction error and decision quality: Regions of prediction error may be decision-irrelevant; errors near decision boundaries are disproportionately impactful.
• Inefficiency of uniform data acquisition: Data collection should be concentrated near regions where uncertainty affects decisions, not uniformly across the feature space.
• Inadequacy of standard distributional metrics: Classical metrics (e.g., Wasserstein, KL) may overstate distributional discrepancy when decision boundaries are invariant.

Decision-Focused Learning Formulations

DFL formalizes model selection via minimizing the decision loss, most notably the Smart Predict-then-Optimize (SPO) loss:

$$\ell_{\mathrm{decision}}(h(x),c) := c^\top w^*(h(x)) - c^\top w^*(c)$$

Given a data-generating distribution $D$, the population risk aligns with expected decision loss. Computationally, training DFL models faces challenges arising from nonconvexity, nondifferentiability, and intermittently unobservable cost vectors.

Computational Methods for DFL

DFL training regimes encompass:

• Direct minimization of decision loss: Computationally intractable due to discontinuity and zero gradients at cone interiors.
• Surrogate loss methods: SPO+ [elmachtoub2022smart], perturbation-gradient [huang2024directional], LAVA [berden2025solver], WISE [wan2026solverfree], and PEAR [lee2026decisionfocused] losses facilitate tractable optimization.
• End-to-end prediction of actions: Models directly infer $w$ from context, bypassing explicit optimization, but risk alignment with true decision loss is contingent.
• Prediction of loss functions and optimization oracles: Learning local or global surrogates or training neural meta-solvers that approximate optimization oracles enables scalable deployments.

Geometry and Statistical Properties in Stochastic Linear Programming

For problems with linear objective uncertainty and deterministic feasible regions ($Aw \leq b$), stochastic LPs enjoy uniquely tractable geometry:

Figure 2: The geometric map from cost vectors to optimal extreme points, partitioning the cost space into normal cones—decision changes only at cone boundaries.

The decision loss exhibits several salient properties:

• Zero loss for predictions within the same normal cone (piecewise constant).
• Decision loss is scale-invariant in the prediction but positively homogeneous in the true cost.
• Gradient is zero almost everywhere (except at cone boundaries).
• Decision errors manifest only when predictions cross cone boundaries.

Expected decision loss remains nonnegative and decomposes into excess loss plus the Jensen gap between plug-in and true optimal value.

Data Acquisition: Margin-Based and Directional Methods

DFL motivates novel active learning schemes:

• Margin-Based Acquisition: Quantifies the distance of predicted cost vectors to cone boundaries; data is collected when confidence regions intersect decision boundaries.

  Figure 3: Margin-based uncertainty: Only predictions near cone boundaries (degeneracy) are decision-uncertain, guiding active data acquisition.

• Direction-Based Acquisition: Emphasizes angular disagreement due to scale-invariance of decision loss; directional metrics outperform Euclidean-based uncertainty in identifying decision-relevant samples [wan2026directional].

Figure 4: Standard Euclidean disagreement can be decision-blind, missing critical cone boundary crossings.

Decision-Focused Metrics for Comparing Distributions

Traditional distributional metrics (e.g., Wasserstein) are decision-blind and fail in scenarios where geometric closeness does not imply decision equivalence.

Figure 5: Geometric similarity is insufficient—a group of cost vectors within the same normal cone (decision-equivalent) contrasts with another group across the boundary (decision-different).

The paper introduces the decision-focused optimistic divergence as an optimal transport-based metric:

$$(\mu, \nu) := \inf_{\gamma \in \Gamma(\mu, \nu)} \int (x, y) \, d\gamma(x, y)$$

This metric achieves dimension-independent rates, computational efficiency, and direct control over downstream risk, outperforming classical metrics for clustering, interpolation, and transfer tasks.

Practical Applications: Clustering and Interpolation

DFL-inspired distance metrics enable operationally meaningful clustering and interpolation:

• Decision-aware clustering: Groups units/distributions that induce the same decision, regardless of geometric similarity.
• Decision-aware interpolation: Yields distribution forecasts and barycenters that respect decision boundaries, circumventing artifacts (e.g., multimodal averages).

  Figure 6: Distinct demand distributions yielding identical optimal order quantities highlight the necessity for decision-aware clustering.

  

Figure 7: Naive averaging yields bimodality, whereas transport- and decision-aware interpolation produces more realistic, operationally meaningful intermediates.

Theoretical and Practical Implications, Future Directions

DFL reframes questions of sample complexity, generalization, uncertainty quantification, and robust optimization with respect to decision quality. It challenges the classical paradigm in settings with nonlinear objectives, stochastic constraints, bandit feedback, sequential design, and hierarchical models. Research directions include:

• Statistical learning rates and margin conditions for DFL.
• Extension to multi-period, non-linear, and dynamic settings—interfaces with reinforcement learning.
• Design of computationally tractable decision-aware statistical procedures.
• Feature selection, denoising, and offline policy evaluation under decision focus.
• Broader deployment in healthcare, logistics, pricing, and dynamic service systems.

Conclusion

The DFL framework necessitates reassessing statistical learning pipelines in operations-centric contexts. It couples geometric and statistical intuition to reveal why traditional models fail and provides principled methods to mitigate these failures. DFL demonstrates that achieving superior downstream decisions requires reengineering not only loss functions but also data acquisition protocols and distributional comparison metrics. Future work will extend these foundations to broader classes of uncertainties, objectives, and sequential decision-making environments.