Online Conformal Prediction for Non-Exchangeable Panel Data

Abstract: Panel data, in which multiple units are repeatedly observed over time, arise throughout science and engineering. Quantifying predictive uncertainty in such settings is challenging because conformal prediction, while distribution-free and model-agnostic, classically relies on exchangeability assumptions that fail under temporal dependence and unit heterogeneity. We propose a simple online conformal framework for non-exchangeable panel data. The method exploits a key feature of online panel prediction: when a forecast is required for one unit, contemporaneous outcomes from related units may already be observed and can serve as a calibration panel. At each round, prediction sets are formed using currently observed calibration units together with two adaptive quantities: history-based similarity weights that emphasize calibration units resembling the target, and an adaptive miscoverage level that is updated whenever target feedback is revealed. This two-state design yields a stepwise coverage bound and a long-run coverage guarantee. Empirically, across synthetic and real panel data sets, the method improves coverage on the worst-covered target units through adaptive interval-width allocation rather than uniform inflation. The two states are complementary: similarity weights protect coverage when target feedback is sparse, while the adaptive level further improves coverage as feedback accumulates.

• The paper presents the W-TQA algorithm, extending conformal prediction to non-exchangeable panel data with adaptive miscoverage adjustment.
• It employs cross-sectional similarity weights and an adaptive temporal update to achieve robust coverage even under distribution shifts and sparse feedback.
• Empirical evaluations demonstrate superior tail coverage and efficient, adaptive interval allocation across financial, retail, and energy datasets.

Online Conformal Prediction for Non-Exchangeable Panel Data: An Expert Analysis

Problem Setting and Motivation

The paper addresses online predictive inference with panel data characterized by non-exchangeable structure: units (such as stocks, products, or sensors) are heterogeneous, and each unit is repeatedly observed over time, creating both temporal dependencies and cross-sectional variation. The standard conformal prediction (CP) framework offers distribution-free marginal coverage but relies on an exchangeability assumption, which is violated in these realistic panel settings due to unit heterogeneity, cross-sectional dependence, and time-series nonstationarity.

Practical applications such as asynchronous financial prediction, networked sensor forecasting, and retail demand modeling motivate the need for reliable uncertainty quantification where feedback may be intermittent and the behavior of the target unit may systematically differ from the calibration units.

Methodological Framework

The proposed Weighted Temporal Quantile Adjustment (W-TQA) algorithm extends conformal prediction to the online, non-exchangeable panel scenario. The key methodological innovation lies in the dual online state design:

• Cross-sectional Similarity Weights: At each time $t$, prediction is for a target unit whose contemporaneous outcome is unobserved, but outcomes from $N$ calibration units are available. The algorithm computes running feature means for each unit and applies a Gaussian kernel to compute weights reflecting similarity between each calibration unit and the target. These weights localize calibration, emphasizing peers historically similar to the target and downweighting disparate units.
• Adaptive Nominal Miscoverage Level: The algorithm maintains an adaptive (online) nominal miscoverage level ($\alpha_t$), updated via a gradient step only when target feedback is revealed, correcting persistent bias due to distribution shift or nonstationarity in the target.

Prediction sets at each step are constructed via split conformal calibration using the weighted calibration pool and the current adaptive miscoverage level.

The method is streaming-memory and computationally efficient: only running means, the adaptive level, and the weighting vector are tracked—no score archives or large historical buffers are required.

Theoretical Guarantees

Stepwise Conditional Coverage

The authors derive a stepwise (current-round) coverage result extending the weighted conformal prediction bounds of [7] to the present setting. Specifically:

$$P(Y_{N+1, t} \notin C_t(X_{N+1, t}) | \mathcal{F}_{t-1}) \leq \alpha_t + \Delta_t$$

where the excess term $\Delta_t$ is a weighted sum (by similarity weights) of total variation distances between the conditional law for the target and each calibration unit at time $t$. Theoretically, if the weight mass is concentrated on calibration units with distribution close to the target (in latent profile space), the coverage excess can be tightly controlled.

Under a latent-profile assumption—which is justified in typical factor models for financial panels—the authors show that this design yields a soft neighborhood calibration scheme akin to a "localized" oracle, with estimation error decaying in the online limit.

Long-Run Marginal Coverage

The second result concerns long-run average coverage over the sequence of deployment rounds. The adaptive update for $\alpha_t$ ensures that, on rounds where target feedback is revealed, the time-averaged miscoverage converges to the nominal target under mild regularity and step-size conditions, as in [18]. Under the missing-completely-at-random (MCAR) assumption, this guarantee can be extended to all deployment rounds.

These bounds clarify the division of labor in W-TQA: spatial weighting mitigates coverage loss from exchangeability failure in the cross-section, while the adaptive temporal update removes systematic bias accumulated due to target-side drift or covariate/outcome shift.

Empirical Evaluation

The authors conduct extensive experiments across synthetic and real-world panel data, including high-frequency financial returns, item-level retail demand, and residential electricity load. Evaluation metrics center on average and tail coverage (i.e., coverage on the worst-off 10% of test units) and interval adaptation (Width CoV).

Key findings:

• Strong Tail Coverage: W-TQA achieves the highest tail coverage in all experimental settings, with statistically significant gains over baselines, particularly pronounced under severe cross-sectional heterogeneity and when feedback is sparse or outcomes are highly non-uniform.
• Efficient Interval Allocation: The method achieves high coverage without resorting to uniform interval inflation; instead, interval width is adaptively reallocated across unit and time axes, as evidenced by larger Width CoV than uniform baselines, indicating nuanced output variability.
• Robustness: Parameter sweeps (kernel bandwidth, step size) show that W-TQA performance is not highly sensitive to precise hyperparameter choices, with gains over ablated variants preserved across the evaluated grid.

The two-state design exhibits complementary robustness: when target feedback is frequent, temporal updating dominates; when feedback is rare, cross-sectional weights alone provide strong fallback coverage.

Implications and Future Directions

The W-TQA method delineates a path for predictive inference in realistic panel data environments where distributional assumptions required by classical conformal methods fail. In practical terms, this allows reliable uncertainty quantification in mission-critical applications (finance, energy, transportation) where panel units are disparate and outcome feedback is not always immediate or guaranteed.

On the theoretical side, the paper combines advances on weighted, non-exchangeable conformal calibration with online adaptation to temporal drift, establishing oracle-style bounds under latent feature regularity. The adaptivity along both peer and longitudinal axes implies that the method is robust to future extensions in which more complex nonstationarity or informative label-revealing mechanisms (beyond MCAR) are present.

Natural extensions include:

• Incorporating richer profile summaries (covariance structure, score histories) for more expressive peer similarity, at the cost of additional online estimation burden.
• Selection-adjusted calibration for non-MCAR or outcome-dependent label reveal processes, building on the explicit bias decomposition outlined in their supplementary analysis.
• Online update of the underlying point predictor, particularly when drift in relationships between features and outcomes is severe.

Conclusion

The W-TQA framework operationalizes online conformal prediction for non-exchangeable, temporally dependent panel data. By integrating localized cross-sectional weighting with adaptive temporal calibration, it provides formal short- and long-term coverage guarantees. Empirical results demonstrate superior tail coverage and adaptive interval allocation compared to leading baselines, all while operating efficiently in an online streaming regime. The approach can be expected to inform further research at the intersection of distribution-free predictive inference, online learning, and high-dimensional panel modeling, and is immediately applicable in domains where exchangeability is an unrealistic modeling assumption.