Learning in Prophet Inequalities with Noisy Observations

Abstract: We study the prophet inequality, a fundamental problem in online decision-making and optimal stopping, in a practical setting where rewards are observed only through noisy realizations and reward distributions are unknown. At each stage, the decision-maker receives a noisy reward whose true value follows a linear model with an unknown latent parameter, and observes a feature vector drawn from a distribution. To address this challenge, we propose algorithms that integrate learning and decision-making via lower-confidence-bound (LCB) thresholding. In the i.i.d.\ setting, we establish that both an Explore-then-Decide strategy and an $\varepsilon$-Greedy variant achieve the sharp competitive ratio of $1 - 1/e$, under a mild condition on the optimal value. For non-identical distributions, we show that a competitive ratio of $1/2$ can be guaranteed against a relaxed benchmark. Moreover, with limited window access to past rewards, the tight ratio of $1/2$ against the optimal benchmark is achieved.

• The paper demonstrates that leveraging a linear structure with contextual features enables achieving sharp competitive ratios of 1-1/e (i.i.d.) and 1/2 (non-identical) despite noisy observations.
• It introduces explore-then-decide and ε-greedy methods using lower confidence bound thresholds to effectively balance exploration and exploitation in sequential decision-making.
• Empirical validations on synthetic data confirm that the proposed algorithms outperform classical benchmarks, offering practical insights for online optimal stopping applications.

Learning in Prophet Inequalities with Noisy Observations: A Formal Essay

Introduction

The paper "Learning in Prophet Inequalities with Noisy Observations" (2604.01789) undertakes a rigorous study of the classical prophet inequality problem under realistic, informational constraints: reward distributions are unknown and rewards are observable only through noisy signals. Exploiting the structural assumption that latent rewards are linearly parameterized by observable contextual features, the authors develop learning-based thresholding algorithms that integrate statistical estimation and optimal stopping. The resulting competitive ratio guarantees demonstrate that sharp prophet bounds ($1 - 1/e$ for i.i.d. and $1/2$ for non-identical distributions) are achievable under mild regularity conditions, thus overcoming key impossibility results in the literature.

Problem Setting and Structural Assumptions

The decision-maker sequentially observes noisy reward realizations $y_i = x_i^\top \theta + \eta_i$, where each feature $x_i \in \mathbb{R}^d$ is drawn from an unknown distribution $\mathcal{D}_{x,i}$; $\theta \in \mathbb{R}^d$ is an unknown latent parameter and $\eta_i$ is i.i.d. $\sigma$-sub-Gaussian noise. The classical prophet inequality asks: how closely can an online stopping rule approximate the expected value achieved by a prophet (omniscient agent) who always selects the maximum reward? The target is the asymptotic competitive ratio

$$\liminf_{n\rightarrow\infty}\frac{\mathbb{E}[X_\tau]}{\mathbb{E}[\max_{i\in[n]}X_i]},$$

where $\tau$ is the stopping time determined by policy and $1/2$0. Key regularity assumptions include boundedness on $1/2$1 and $1/2$2, and non-degeneracy on the feature covariance's minimum eigenvalue.

Noisy Observations: Fundamental Limitations and Structural Recovery

A central result is the demonstration that, in the absence of structural knowledge, noisy reward observations alone induce a trivial competitive ratio: for some distribution classes, any policy must satisfy

$1/2$3

(Proposition 1 of the paper). The critical shift is leveraging feature-dependent linearity: this enables statistical learning of $1/2$4 from noisy observations, thereby permitting nontrivial prophet bounds even when the underlying reward distributions are unknown.

Learning-Based Algorithms and Competitive Ratio Guarantees

Two principal algorithmic frameworks are developed for the i.i.d. case:

Explore-then-Decide with Lower Confidence Bound (LCB) Thresholding

This method employs an initial forced exploration phase of size $1/2$5 to collect pairs $1/2$6. An estimate $1/2$7 is formed via regularized least squares, yielding a lower-confidence bound (LCB) for each $1/2$8 given by

$1/2$9

where $y_i = x_i^\top \theta + \eta_i$0 controls the confidence radius. The threshold $y_i = x_i^\top \theta + \eta_i$1 is set as the $y_i = x_i^\top \theta + \eta_i$2 quantile using $y_i = x_i^\top \theta + \eta_i$3 and the LCB estimator. Stopping occurs when $y_i = x_i^\top \theta + \eta_i$4.

$y_i = x_i^\top \theta + \eta_i$5-Greedy with LCB Thresholding

This method randomly interleaves exploration and stopping phases, where exploration rounds update $y_i = x_i^\top \theta + \eta_i$6 online and corresponding LCBs and thresholds are recalculated dynamically. This ensures that information acquisition is temporally distributed, desirable in practical online systems.

Competitive Ratios

Both methods provably achieve the sharp competitive ratio of $y_i = x_i^\top \theta + \eta_i$7 in the i.i.d. setting:

$y_i = x_i^\top \theta + \eta_i$8

contingent upon a mild growth condition on $y_i = x_i^\top \theta + \eta_i$9 relative to the exploration length and the noise level (Corollary 1, Theorem 2).

Figure 1: Empirical competitive ratios for i.i.d. distributions under $x_i \in \mathbb{R}^d$0 demonstrating performance above $x_i \in \mathbb{R}^d$1.

Non-Identical Distributions: Relaxed and Windowed Prophet Benchmarks

For general non-identical rewards, the classical prophet benchmark is unattainable in the online single-sequence setting due to the necessity of initial exploration. The authors target two relaxed benchmarks:

1. Relaxed Prophet Benchmark: Exclude pure exploration rounds, compare with $x_i \in \mathbb{R}^d$2. The competitive ratio is provably at least $x_i \in \mathbb{R}^d$3 (Theorem 4).
2. Window Access: By allowing retrospective selection from a window of indices after the exploration phase (of size $x_i \in \mathbb{R}^d$4), the method achieved the optimal $x_i \in \mathbb{R}^d$5 competitive ratio against the true prophet benchmark (Theorem 6).

Both approaches exploit the linear structure to minimize offline sample requirements or window access size: $x_i \in \mathbb{R}^d$6 suffices for polynomial $x_i \in \mathbb{R}^d$7.

Figure 2: Competitive ratios for non-identical distributions, with $x_i \in \mathbb{R}^d$8, confirming the empirical attainment of the $x_i \in \mathbb{R}^d$9 bound with window access.

Empirical Validation

Monte Carlo experiments on synthetic data (linear rewards with Gaussian noise) validate the theoretical predictions, showing competitive ratios above thresholds of $\mathcal{D}_{x,i}$0 (i.i.d.) and $\mathcal{D}_{x,i}$1 (non-i.i.d.) under increasing noise levels. Both ETD-LCBT and $\mathcal{D}_{x,i}$2-Greedy-LCBT outperform Gusein-Zade's rule ($\mathcal{D}_{x,i}$3) as noise increases.

Implications and Future Directions

The results have practical impact for sequential decision-making domains (e.g., advertising, hiring, recommendation) with contextual features and stochastic rewards. By coupling statistical estimation with optimal stopping, the approach paves the way for generalization to more expressive structural reward models beyond linearity, and for extensions to multi-armed prophet problems, correlated reward structures, or adaptivity-limited bandit settings. Future work includes addressing more general noise models and relaxing feature distribution assumptions, as well as higher-order competitive ratios via adaptive/online quantile thresholds.

Conclusion

This paper establishes that, even in the presence of unknown reward distributions and noisy observations, the combination of contextual feature-based linear structure and careful confidence-threshold policies enables sharp prophet inequality guarantees. The algorithmic techniques and theoretical bounds constructed here directly address fundamental impossibility barriers, establishing a robust foundation for realistic online optimal stopping under uncertainty.