Asymptotically Optimal Learning for Parametric Prophet Inequalities

Abstract: We study learning in prophet inequalities with i.i.d. rewards drawn from an exponential-type parametric family with an unknown parameter $θ$, a class that includes exponential, Pareto, and bounded-support power-family distributions. We first characterize the optimal full-information asymptotic competitive ratio for this family. In the unbounded-support case, the limit is $${\left(θ/({θ-c_+})\right)^{c_+/θ}}/ {Γ(1-c_+/θ)},$$ while in the bounded-support case, the limit is $1$. We then propose a confidence-based dynamic-programming policy for online learning. By exploiting the explicit parametric structure, the policy achieves the same optimal asymptotic competitive ratio using only online observations, without external offline samples. We further derive distribution-specific convergence rates for canonical examples. Finally, numerical experiments on synthetic instances illustrate the performance of our algorithm.

• The paper introduces a confidence-based dynamic programming policy that learns the unknown parameter online to attain full-information benchmarks.
• The methodology provides explicit convergence rates for exponential, Pareto, and bounded-support models, significantly outperforming nonparametric rules.
• The study unifies optimal stopping theory with extreme-value analysis, establishing asymptotically optimal performance guarantees under parametric assumptions.

Asymptotically Optimal Learning for Parametric Prophet Inequalities

Overview

The paper "Asymptotically Optimal Learning for Parametric Prophet Inequalities" (2606.26893) provides a comprehensive study of online learning in prophet inequalities when underlying i.i.d. rewards are drawn from a parametric exponential-type family with an unknown parameter. The study advances the analysis of optimal stopping problems by characterizing the full-information asymptotic competitive ratio under parametric models, introducing a confidence-based dynamic programming policy that learns the unknown parameter online, and providing explicit convergence rates in canonical distributional settings. The implications extend to domains where classic prophet inequality guarantees are insufficient due to a lack of distributional knowledge but where structural assumptions are justified.

Model and Problem Formulation

In the considered finite-horizon online stopping problem, at each round $i \in [n]$, the agent observes $X_i \sim F_\theta$, where $F_\theta$ is a member of a one-parameter exponential-type family: $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$ for a fixed, known strictly increasing function $\phi$ and unknown $\theta > 0$, with either unbounded ($x_F = \infty$) or bounded ($x_F < \infty$) support.

The goal is to design an online policy $\tau$, using only the sequentially observed rewards (no offline samples), that maximizes the expected competitive ratio

$$\mathrm{CR}_n(\tau; \theta) = \frac{\mathbb{E}_\theta[X_\tau]}{\mathbb{E}_\theta[\max_{i \in [n]} X_i]}$$

compared to the performance of a "prophet" who knows all $X_i \sim F_\theta$0 in advance.

Main Theoretical Contributions

Full-Information Asymptotic Benchmarks

A core result is the precise asymptotic characterization of the optimal competitive ratio in the parametric setting:

• Unbounded support ($X_i \sim F_\theta$1): The limit depends on a tail parameter $X_i \sim F_\theta$2 (derived from the endpoint growth rate of $X_i \sim F_\theta$3),

$X_i \sim F_\theta$4

with $X_i \sim F_\theta$5, $X_i \sim F_\theta$6, and the increment structure capturing extreme-value theoretic properties (Gumbel- and Fréchet-type behavior).

• Bounded support ($X_i \sim F_\theta$7): Under endpoint regularity, the optimal competitive ratio converges to $X_i \sim F_\theta$8 as $X_i \sim F_\theta$9.

(Canonical cases include exponential ($F_\theta$0, $F_\theta$1), Pareto ($F_\theta$2, $F_\theta$3), and bounded power-law families.)

Online Learning Algorithm: Confidence-Based Dynamic Programming

An exploration-exploitation scheme is constructed:

1. Exploration: Observe and reject the first $F_\theta$4 samples; use a maximum-likelihood estimator (MLE) for $F_\theta$5 based on a linearized exponential transformation ($F_\theta$6).
2. Plug-in DP with Confidence: Construct an upper-confidence bound $F_\theta$7 for the parameter to counteract censoring effects; use backward dynamic programming to calculate thresholds recursively for subsequent rounds, treating $F_\theta$8 as the model parameter.

Theoretical guarantee: With exploration length $F_\theta$9 scaling appropriately in $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$0, the learned policy attains the full-information asymptotic competitive ratio for the ground-truth $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$1, matching the unattainable performance of classic nonparametric policies in any heavy-tail regime.

$$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$2

where $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$3 is the failure probability parameter used in the confidence bound.

Distribution-Specific and Rate-Optimality Results

• Exponential: Convergence to $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$4, with the rate matching full-information DP, i.e., $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$5.
• Pareto: Attainment of the Fréchet-optimal limit $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$6, with convergence rate $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$7 dominated by intrinsic finite-horizon effects rather than estimation error.
• Bounded-Support Power Family: Convergence rate $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$8, faster than rank-based rules.

These rates always strictly improve upon nonparametric policies (based on relative ranks alone), especially in Fréchet-type heavy tail settings, where nonparametric rules are provably sub-optimal.

Experimental Results

Figure 1 demonstrates empirical competitive-ratio trajectories on synthetic exponential, Pareto, and bounded-support reward sequences. The proposed algorithm consistently approaches the theoretical full-information limits for all models, outperforming both classic secretary/prophet-style rules and nonparametric rank-based policies, with pronounced separation in the Pareto regime.

Figure 1: Competitive-ratio curves for exponential, Pareto, and bounded-support rewards, confirming near-optimality across all parametric cases.

Theoretical and Practical Implications

These results provide a sharp separation between:

• Nonparametric prophet learning, where severe sample complexity barriers are provable ([correa2019prophet]), and parametric structure must be assumed for learning to be feasible;
• Parametric prophet learning, where even minimal online observations suffice for asymptotic optimality, provided the structural family is correctly specified.

The analysis unifies optimal stopping theory with extreme-value theory, clarifies when relative-rank rules fail to be optimal, and demonstrates that learning under smoothly parameterized model classes recovers the benefits of known-distribution DP policies.

The confidence-based plug-in DP framework introduces a robust and general approach for parametric optimal stopping, potentially extensible to higher-dimensional, contextual, or structured reward models.

Methodological Highlights

• Reduction of the parametric estimation problem to exponential MLE via sufficient statistics for online learning.
• Construction of tight confidence intervals for $$F_\theta(x) = \begin{cases} 0, & x < x_0 \ 1 - \exp(-\theta \phi(x)), & x_0 \le x < x_F \ 1, & x \ge x_F, \end{cases}$$9 based on exponential concentration inequalities.
• Recursion/DP structure is fully explicit in all canonical cases, allowing for analytical and computational tractability.
• Analytical derivation of distribution-specific rates and asymptotic constants through coupling with regular variation / extreme-value analysis.

Limitations and Future Directions

The framework presumes a correctly specified one-parameter exponential-type family; misspecified or multi-parameter settings, as well as models where the structure is not fully known, remain open for further research. Extension to contextual, high-dimensional, or adversarially perturbed settings poses significant new challenges, e.g., in combining regret minimization and competitive analysis with online statistical estimation.

Conclusion

This work sharply characterizes and realizes the optimal approach to online learning in parametric prophet inequalities, both theoretically and algorithmically. For a broad class of exponential-type reward families, it establishes that online learning with only sequential observations suffices to match powerful full-information prophet benchmarks. The developed methodology sets a new standard for structured online optimal stopping under both light-tailed and heavy-tailed regimes, with concrete performance gains over nonparametric techniques and explicit guidance for practical implementation.

Reference:

"Asymptotically Optimal Learning for Parametric Prophet Inequalities" (2606.26893).