Revisiting Active Sequential Prediction-Powered Mean Estimation

Abstract: In this work, we revisit the problem of active sequential prediction-powered mean estimation, where at each round one must decide the query probability of the ground-truth label upon observing the covariates of a sample. Furthermore, if the label is not queried, the prediction from a machine learning model is used instead. Prior work proposed an elegant scheme that determines the query probability by combining an uncertainty-based suggestion with a constant probability that encodes a soft constraint on the query probability. We explored different values of the mixing parameter and observed an intriguing empirical pattern: the smallest confidence width tends to occur when the weight on the constant probability is close to one, thereby reducing the influence of the uncertainty-based component. Motivated by this observation, we develop a non-asymptotic analysis of the estimator and establish a data-dependent bound on its confidence interval. Our analysis further suggests that when a no-regret learning approach is used to determine the query probability and control this bound, the query probability converges to the constraint of the max value of the query probability when it is chosen obliviously to the current covariates. We also conduct simulations that corroborate these theoretical findings.

• The paper presents non-asymptotic high-probability deviation bounds that provide time-uniform confidence for sequential mean estimation.
• It formalizes query allocation as an online convex optimization problem using the FTRL paradigm, achieving sublinear regret.
• Empirical results demonstrate that uniform querying under budget constraints performs as well as, or better than, uncertainty-informed policies.

Revisiting Active Sequential Prediction-Powered Mean Estimation: A Technical Analysis

Problem Formulation and Contributions

This paper undertakes a comprehensive study of the sequential active mean estimation problem where sample labels are expensive to query, but model predictions are freely available and potentially biased. The target is efficient estimation of the mean label under strict labeling budget constraints by dynamically choosing, for each example, either to query the ground-truth label or to rely on model predictions. The central question is the construction and analysis of query policies that allocate label queries to maximize estimation efficiency.

Previous work, particularly "Active Statistical Inference" [zrnic2024active], proposes a mixture policy: label querying probabilities are a convex combination of a model-uncertainty-informed rule and a uniform (budget-driven) rule. This paper makes several significant theoretical and empirical refinements:

• Establishes non-asymptotic high-probability bounds for estimator deviation, providing time-uniform confidence with explicit dependence on the sequentially accumulating conditional variances.
• Investigates, both theoretically and empirically, the role of model uncertainty in policies, revealing that uniform querying (ignoring instantaneous uncertainty) is often empirically optimal.
• Formalizes the problem of query allocation as an online convex optimization task, leveraging the FTRL (Follow-the-Regularized-Leader) paradigm, and provides regret analyses that tie the policy's long-run behavior to the uniform budget constraint.

Non-Asymptotic Analysis of the Estimator

The estimator under study is an inverse-probability-weighted sum that, at each step, fuses the model prediction with the (occasionally observed) label, using the querying probability $\pi_t(x_t)$ as a weight. The paper rigorously derives a non-asymptotic, time-uniform deviation bound for the estimator, improving on the prior literature's merely asymptotic confidence intervals.

The key technical contribution is the application of Freedman's inequality for martingales, leading to the result that for time $t$ and any $\delta \in (0,1/e)$, with high probability:

$$| w_{t+1} - \mu_y| \leq \frac{ 2 \max \left \{ 2 \sqrt{S_t} , (G+|\mu_y|) \sqrt{ \log \left( \frac{\log(T)}{\delta} \right) } \right \} \sqrt{ \log\left( \frac{\log(T)}{\delta} \right) } }{T} + \left( 1 - \frac{t}{T} \right) \left| \mu_y \right|$$

where $S_t$ is the sum of conditional variances up to $t$. This simultaneously controls estimator deviation at all times $t$, capturing the transient period before the estimator stabilizes and yielding explicit convergence rates, typically $O(1/\sqrt{t})$ once a burn-in regime is passed.

Query Policy Optimization via Online Learning

Building on the observation that the only component of estimator variance directly controlled by the label-querying policy is the term $$\frac{1}{\pi_t(x_t)} \mathbb{E}\left[ (y_t - f_t(x_t))^2 | \mathcal{F}_{t-1} \right]$$, the paper introduces an oracle-driven optimization: select query probability $p_t$ at each time by minimizing an online sequence of convex losses.

The algorithmic instantiation is FTRL, which updates $t$0 each round to optimize accumulated surrogate loss $t$1, with $t$2 serving as a proxy for the (unobservable) conditional variance. The established sublinear regret bound ensures that, asymptotically, the algorithm cannot substantially outperform simply setting $t$3 to the budget-fixed value $t$4.

The data-dependent confidence bounds now depend on the regret achievable by the online policy. As rigorously shown, when accessible oracles are bounded and policies follow FTRL, regret is $t$5, and the estimator's high-probability deviation is tightly controlled—essentially matching the theoretical lower bound available to fixed, budget-oblivious policies.

Empirical Evaluation: Uniform vs. Model-Uncertainty-Guided Policies

The empirical portion encompasses four datasets, including real-world (politeness scoring, wine review, post-election survey) and synthetic tasks. The core findings are consistent and sharply challenge conventional wisdom:

• Uniform query policies, oblivious to model uncertainty, deliver confidence intervals that are as tight or tighter than those incorporating model-based uncertainty, both in mean width and empirical coverage.
• FTRL-based adaptive policies, which quickly converge to the uniform policy, match or exceed the performance of the model-uncertainty-mixing scheme from [zrnic2024active].
• Across settings, incorporating complex model uncertainty signals into the query policy did not yield practical benefit for mean estimation under the sequential label budget regime.

These findings are illustrated in detail:

Figure 1: Politeness score task: Comparison across policies shows similar or better interval width and coverage for uniform querying.

Figure 2: Wine review task: Empirical results reproduce the pattern where uniform and FTRL-based policies match model-uncertainty-mixing performance.

Figure 3: Post-election survey: Performance profiles as a function of sampling budget reaffirm lack of gain from uncertainty-informed querying.

Figure 4: Synthetic dataset experiments confirm that adaptivity to model uncertainty does not improve statistical confidence intervals.

Theoretical and Practical Implications

The paper presents a contradictory result relative to much of the active learning and estimation literature: for this estimation problem and under limited label budgets, conditioning query probability on model uncertainty does not lead to tighter or more accurate confidence intervals compared to uniform query allocation (subject to the budget constraint).

This insight has broad consequences for the design of hybrid human-in-the-loop or LLM-powered inference pipelines. Under sequential label constraints, investing modeling effort into uncertainty quantification for the purposes of active mean estimation is not statistically justified. Instead, optimal empirical policies are simple and deterministic.

On the theoretical front, the formal online learning analysis ties active inference to regret minimization and demonstrates the power of non-asymptotic, time-uniform guarantees for sequential inference problems.

Future Directions in AI-Powered Inference

As AI systems become more integrated into hybrid data collection and inference tasks, understanding when model-derived uncertainty facilitates efficiency is essential. This paper suggests the need for carefully delineating the regimes (e.g., high-dimensional, adaptive feedback, richer structural models) in which model-based policies may have actual merit, highlighting a disconnect between classical uncertainty-driven active learning and the sequential mean estimation setting. Further work should revisit the intersection between nonparametric uncertainty quantification, adaptive allocation, and downstream inferential goals beyond mean estimation.

Conclusion

This work provides a rigorous and empirically validated reassessment of sequential active mean estimation. The key outcome is that uniform querying, constrained only by the label budget, is at least as effective as policies exploiting model uncertainty for sequential mean estimation with model predictions and limited label access. The results have substantive implications for the design of efficient hybrid inference protocols and clarify theoretical limits for sequential, model-assisted estimation tasks.