Distribution-free changepoint localization after sequential change detection

Abstract: This paper introduces a distribution-free framework for constructing post-detection confidence sets for changepoints after stopping a sequential change detection procedure. It is well known that conformal test martingales can be used to sequentially detect changes in distribution, but by themselves provide no inference for the time at which a proclaimed change occurred. Past work on post-detection inference requires pre- and post-change classes of distributions to be known, but this paper accomplishes localization of the changepoint without any distributional assumptions. We establish finite-sample coverage guarantees (conditional on correct detection). We provide non-asymptotic bounds on the conditional expected size of the confidence sets. Under suitable asymptotic regimes, we proved that the conditional expected size of the confidence set remains uniformly bounded. and demonstrate strong empirical performance on simulated and real data. To the best of our knowledge, this is the first general distribution-free framework for sequential changepoint localization with a valid post-detection coverage guarantee.

• The paper introduces the first finite-sample, distribution-free method for constructing post-detection confidence intervals for changepoints.
• It leverages conformal martingales and p-values derived from user-specified score statistics to yield lower, upper, and two-sided confidence sets.
• Empirical evaluations on Gaussian, MNIST, and wine data demonstrate that the method achieves sharp, controlled localization even under challenging signal regimes.

Distribution-Free Changepoint Localization After Sequential Change Detection

Problem Formulation and Motivation

This paper addresses inferential questions in the context of sequential changepoint detection, considering a data stream $X_1, X_2, \dots$ where at some unknown $T$, the distribution shifts from $F_0$ to $F_1$—both otherwise unrestricted and possibly high-dimensional. While conformal test martingales enable nonparametric, distribution-free sequential change detection, they do not provide uncertainty quantification for the changepoint’s location. Historically, post-detection confidence sets for $T$ have assumed parametric models for $F_0$ and $F_1$ or limited the classes of distributions. The present work introduces the first framework for constructing finite-sample, distribution-free post-detection confidence intervals for changepoints, which is agnostic to both the data model and the detection algorithm.

This is critical in practical monitoring scenarios—such as quality assurance, industrial sensor streams, network security, or large-scale data monitoring—where model assumptions are often implausible. The framework ensures that, following a sequential detection alarm, one can specify with controlled (conditional or unconditional) coverage a valid confidence set for when the change occurred, regardless of the statistical characteristics of the signal distributions.

Distribution-Free Inferential Framework

The methodology leverages conformal inference, specifically conformal test martingales, to facilitate both lower and upper post-detection confidence sets, requiring only exchangeability within pre- and post-change segments. The construction does not assume independence between segments nor knowledge of $F_0$, $F_1$, or their parametric forms.

For each candidate changepoint $t \leq \tau$ (where $T$0 is the sequential detection stopping time), the null hypothesis $T$1 is tested against alternatives $T$2 ("lower" set) and $T$3 ("upper" set):

• Lower confidence set: For each $T$4, a conformal martingale based on p-values derived from a user-specified score statistic $T$5 is run forward from $T$6. The test aggregates evidence against $T$7, retaining $T$8 in the set if this evidence remains below a calibrated threshold.
• Upper confidence set: A similar process is run backward, using a potentially different score $T$9.
• Two-sided interval: The intersection of upper and lower sets provides a valid, doubly-sided $F_0$0 confidence set for $F_0$1.

Calibration utilizes Monte Carlo simulations (using any distribution due to the detection procedure's distribution-freeness) to estimate $F_0$2, ensuring adaptivity and coverage control. Notably, this "wrapper" approach is compatible with any underlying sequential detection scheme.

Theoretical Guarantees

Coverage: All confidence sets, under minimal exchangeability assumptions, are shown to achieve conditional coverage at the specified level (e.g., $F_0$3). For detectors controlling the probability of false alarm, the coverage is unconditional.

Set size analysis: Rigorous nonasymptotic and asymptotic analysis quantifies the expected size of the constructed confidence sets conditional on correct detection. With suitable signal-to-noise (score discriminability) and detection delay regime, the expected confidence set width is shown to remain $F_0$4 (i.e., uniformly bounded) as $F_0$5 or decays as $F_0$6. Notably, even without distributional assumptions, the localization error does not increase with sample size in regimes typical of practical changepoint detection.

Score function selection: While validity is guaranteed for any score, set sharpness improves dramatically if the score approximates the likelihood ratio or a monotone surrogate (in the sense of Neyman-Pearson optimality); in practice, data-driven or pretrained scores can be incorporated.

Empirical Evaluation

The framework is thoroughly evaluated on simulated and real-world changepoint scenarios:

• Gaussian mean change: Demonstrates that two-sided confidence sets localize the changepoint tightly around the true $F_0$7 with high probability, even as $F_0$8 increases.

Figure 1: (Left) Raw data stream with detected changepoint and corresponding two-sided confidence sets; (Right) Averaged test statistics exhibit a clear drop at the changepoint, with the threshold marked for easy interpretation.

• MNIST digit shift: On image streams transitioning between digits, the two-sided confidence sets remain narrow at the true transition time, even with class mixing.

Figure 2: (Left) Pure digit change stream yields exceptionally sharp confidence sets; (Right) Mixed regime remains precise despite partial class overlap.

• Wine quality contamination: Applying the framework in a production monitoring scenario, the lower confidence set accurately bounds the contamination onset, enabling safe operational decisions.

  Figure 3: Lower test martingale statistic on wine data, with lower confidence set accurately covering the true changepoint.

Across all settings, the empirical results match the sharpness predicted by theory—confidence intervals are short, even under challenging signal regimes, and maintain the prescribed coverage.

Implications and Future Directions

Practically, this construction equips practitioners with a rigorous uncertainty quantification tool for changepoint localization in complex, nonparametric data streams, circumventing the need for restrictive assumptions about the data-generating mechanisms or change structure. Theoretically, it elucidates the power of conformal methods in sequential inference and demonstrates the separability of the detection and localization tasks in a distribution-free context.

The framework’s generality—requiring only black-box detector access and minimal statistical assumptions—suggests broad applicability to streaming, monitoring, and surveillance problems in high-dimensional or structured domains. Extensions to multi-stream scenarios, adaptive scoring via representation learning, or higher-order changepoint structures represent promising avenues for future research.

Conclusion

This work establishes the first general, distribution-free, finite-sample valid method for constructing confidence sets for changepoint localization following sequential change detection (2606.01256). By leveraging conformal test martingales and careful calibration, the methodology achieves guaranteed coverage and sharp, practical localization for arbitrary data—laying a new foundation for uncertainty quantification in modern changepoint analysis.