Inductive Conformal Anomaly Detection (ICAD)

• ICAD is a split-conformal method that converts non-conformity scores into calibrated p-values, providing rigorous false-alarm guarantees under exchangeability.
• It separates the model training from the calibration process, allowing existing anomaly detectors to be enhanced without altering their core objectives.
• Empirical results demonstrate that ICAD improves detection power across univariate/multivariate time series, OOD, and functional data, balancing calibration precision with computational efficiency.

Inductive Conformal Anomaly Detection (ICAD) is a split-conformal procedure that converts anomaly scores into calibrated $p$-values by comparing a test example’s non-conformity score against scores computed on a held-out calibration set. In the literature summarized here, ICAD appears as a model-free or detector-agnostic layer for anomaly detection in univariate and multivariate time series, functional data, out-of-distribution detection, and new-physics searches; under exchangeability, it provides finite-sample control of the marginal false-alarm or Type I error rate without requiring a parametric model for the inlier distribution (Burnaev et al., 2016, Hennhöfer et al., 2024).

1. Core split-conformal construction

The canonical ICAD setup begins with a reference sample of inliers, split once into a proper training set and a calibration set. A base anomaly detector or non-conformity measure is fit or defined on the proper training set, producing a score function in which larger values indicate greater anomalousness. Calibration scores are then computed on the held-out calibration set, and a new test point is assigned a conformal $p$-value by ranking its score against those calibration scores (Hennhöfer et al., 2024).

A representative split-conformal formula is

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$

with $\alpha_i=\hat s(X_i)$ on the calibration set and $\alpha(x)=\hat s(x)$ for the test point. Closely related formulations also appear in the time-series and functional-data literature, including

$$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$

and

$$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$

as well as a smoothed version with random tie-breaking for functional data (Burnaev et al., 2016, Adams et al., 1 Apr 2025).

The decision rule is likewise standard: fix a significance level $\epsilon$ or $\alpha$, and flag the test point as anomalous when the conformal $p$-value is at most that threshold. In this formulation, the $p$0-value is not a heuristic score; it is a calibrated rank statistic with a direct false-alarm interpretation under the null assumptions.

This construction separates modeling from calibration. The proper training split is used to define the score, while the calibration split is used only to turn that score into a $p$1-value. A plausible implication is that ICAD can be attached to many existing anomaly detectors without changing their internal training objectives, provided they output a scalar score.

2. Validity guarantees and their assumptions

The central guarantee of ICAD is marginal validity under exchangeability. In split-conformal form, the literature states that for any $p$2,

$p$3

and therefore, at nominal significance $p$4,

$p$5

Equivalent statements appear across domains: for time-series ICAD, $p$6; for OOD detection, $p$7; and for functional data, $p$8 (Hennhöfer et al., 2024, Burnaev et al., 2016, Adams et al., 1 Apr 2025).

These guarantees are finite-sample and distribution-free in the specific sense used by the cited works: no parametric assumption is made on the distribution of the anomaly score, and validity is obtained from exchangeability or IID sampling. Several papers emphasize that the key assumption is merely exchangeability of training, calibration, and test examples under the null, with continuity assumptions added in some formulations to avoid ties (Hennhöfer et al., 13 May 2026, Kaur et al., 2022).

The same rank argument also underlies more specialized guarantees. In Mondrian conformalization, calibration is performed within a discrete bin $p$9, yielding

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$0

under exchangeability within that bin. Under covariate shift, a weighted conformal $$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$1-value of the form

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$2

recovers the same bound when the likelihood ratio $$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$3 is known or learned (Araz et al., 11 Jun 2026).

The literature also distinguishes ICAD from full or transductive conformal anomaly detection. Full conformal offers exact small-sample validity by recomputing scores in a leave-one-out style, but ICAD trades that quadratic or repeated-fit cost for a fixed proper-training split and a fixed calibration split. This suggests that ICAD is typically preferred when online or large-scale deployment makes repeated retraining impractical.

3. Non-conformity measures

ICAD does not prescribe a single anomaly score. Instead, it uses a non-conformity measure defined relative to the proper training set, and the reviewed literature develops domain-specific choices for that measure.

In time-series anomaly detection, the most explicit early constructions are distance- and density-based. Burnaev and Ishimtsev describe two non-conformity measures on SSA “Caterpillar” embeddings of a univariate series: a $$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$4-nearest-neighbor score

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$5

and a Local Outlier Factor score

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$6

The same time-delay-embedding perspective appears in the conformal $$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$7-NN streaming detector, where

$$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$8

is the average $$\hat p(x)\;=\;\frac{1 \;+\;\bigl|\{\,X_i\in\mathcal D_{\rm cal}: \alpha_i \ge \alpha(x)\}\bigr|}{|\,\mathcal D_{\rm cal}\!| \;+\;1}\,,$$9-nearest-neighbor distance in a sliding proper-training window (Burnaev et al., 2016, Ishimtsev et al., 2017).

In OOD detection for deep models, the non-conformity measure is adapted to learned invariances or robustness properties. iDECODe defines a base score from equivariance error,

$\alpha_i=\hat s(X_i)$0

aggregated over multiple sampled transforms. “Polysemantic Dropout” instead uses layer-wise dropout tolerance: if $\alpha_i=\hat s(X_i)$1 is the fraction of neurons that can be dropped before the response flips, then

$\alpha_i=\hat s(X_i)$2

Layer-wise $\alpha_i=\hat s(X_i)$3-values are then merged by valid merging functions $\alpha_i=\hat s(X_i)$4 to retain the ICAD false-alarm guarantee (Kaur et al., 2022, Gupta et al., 4 Sep 2025).

For functional data, Adams et al. construct non-conformity from elastic distances to the Karcher mean of the training set. With amplitude distance $\alpha_i=\hat s(X_i)$5 and phase distance $\alpha_i=\hat s(X_i)$6, the score for a calibration or test curve is an equally weighted, scaled average of its amplitude and phase deviations from the training mean. This explicitly targets both magnitude outliers and shape outliers, including phase variation captured through alignment warps (Adams et al., 1 Apr 2025).

In multivariate time-series forecasting, CoCAI uses split conformal prediction to calibrate forecast intervals, then defines anomaly scoring from normalized distance-to-interval trajectories, B-spline coefficients, and a copula-based Mahalanobis index. The anomaly score is $\alpha_i=\hat s(X_i)$7, where $\alpha_i=\hat s(X_i)$8 is the tail probability of the fitted copula law. This is cast explicitly as an ICAD-style workflow in which conformal calibration and downstream anomaly scoring are coupled but distinct (Pearson et al., 23 Jul 2025).

4. Streaming calibration and time-series formulations

A major line of ICAD research concerns data streams and non-stationary time series. In the conformal $\alpha_i=\hat s(X_i)$9-NN detector for univariate streams, ICAD is implemented with two fixed-size structures: a proper training set $\alpha(x)=\hat s(x)$0 of size $\alpha(x)=\hat s(x)$1 and a calibration queue $\alpha(x)=\hat s(x)$2 containing the $\alpha(x)=\hat s(x)$3 most recent non-conformity scores. At each time step, the method computes $\alpha(x)=\hat s(x)$4, forms

$\alpha(x)=\hat s(x)$5

updates the calibration queue, and slides the training window forward (Ishimtsev et al., 2017).

This streaming form is designed to adapt to non-stationarity. The cited practical guidance is explicit. The embedding dimension $\alpha(x)=\hat s(x)$6 should capture seasonality or autocorrelation; the calibration window size $\alpha(x)=\hat s(x)$7 trades adaptivity against statistical power; and the training window length $\alpha(x)=\hat s(x)$8 controls how many points the $\alpha(x)=\hat s(x)$9-NN reference uses. The paper also notes a simple post-processing “throttling” rule, such as holding the output at $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$0 for the next $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$1 points when $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$2, to reduce spurious false alarms (Ishimtsev et al., 2017).

The earlier time-series ICAD formulation based on SSA embedding is similar in spirit but organized as a proper-training matrix $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$3, a calibration matrix $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$4, and a test vector $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$5 built from the next $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$6 observations. The underlying feature extractor is the $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$7 Hankel matrix, whose columns act as the embedded objects on which $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$8-NN or LOF is applied. The detector then thresholds the conformal $$p(z)=\frac{1+|\{i\in\{1\ldots C\}:\alpha_i\ge \alpha_{\text{test}}\}|}{1+C}$$9-value $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$0 (Burnaev et al., 2016).

These formulations show that ICAD can be used in strictly online settings as well as in sliding-window batch settings. A plausible implication is that the conformal layer provides a common interface between classical nearest-neighbor scoring and streaming false-alarm control, even when the underlying series is drifting.

5. Variants beyond basic split ICAD

Several works position basic ICAD as the computationally cheapest member of a larger family. The 2024 study on leave-one-out-, bootstrap-, and cross-conformal anomaly detectors explicitly contrasts full-conformal, split-conformal, cross-conformal, jackknife, and jackknife-after-bootstrap methods. In that comparison, split-conformal ICAD requires one model fit on $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$1 points, uses $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$2 calibration points, and is described as having lower statistical efficiency than the data-reusing alternatives, while cross-conformal and jackknife sit between split and full conformal in the efficiency-versus-cost trade-off (Hennhöfer et al., 2024).

The same extension pattern appears in the nonconform framework. There, ICAD is the foundational split-calibration strategy, exposed alongside CrossValidation, JackknifeBootstrap, probabilistic approximation, calibration-conditional estimation, and weighted conformal anomaly detection. The package also supports Benjamini–Hochberg for batch false discovery rate control after $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$3-value computation, and presents the conformal layer as a meta-estimator around scikit-learn, pyod, or custom anomaly detectors (Hennhöfer et al., 13 May 2026).

Domain-specific extensions add further structure. Weighted and Mondrian conformalization address sideband-to-signal-region exchangeability failures in resonant searches, while a Gross–Vitells step turns valid local conformal $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$4-values into a look-elsewhere-aware global significance. In that setting, the local count statistic

$$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$5

is scanned across windows, and a Gross–Vitells bound is used to control the maximum over the scan (Araz et al., 11 Jun 2026).

These developments do not replace ICAD so much as generalize its calibration logic. This suggests that “ICAD” often functions as the baseline split-conformal layer from which more data-efficient, conditional, or shift-aware procedures are constructed.

6. Empirical behavior across application domains

Empirical studies consistently evaluate ICAD by separating statistical validity from detection power. In univariate time-series benchmarking on the Numenta Anomaly Benchmark and Yahoo! S5, the 2016 KNN-ICAD method reports a NAB Standard score of $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$6, behind Numenta HTM at $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$7 and ahead of Twitter ADVec at $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$8, with LOF-ICAD at $$p_{n+1}=\frac{1+\bigl|\{i\in\mathcal I^{cal}:s_i\ge s_{n+1}\}\bigr|}{n_2+1}\,,$$9, LoOP at $\epsilon$0, LOF at $\epsilon$1, and Null at $\epsilon$2. Under both “Low-FP” and “Low-FN” profiles, KNN-ICAD remains the runner-up to Numenta HTM and significantly outperforms the non-conformal LOF/LoOP baselines (Burnaev et al., 2016).

The 2017 streaming conformal $\epsilon$3-NN detector reports two parameterizations, $\epsilon$4 and $\epsilon$5. On Yahoo! S5 under the “Standard” NAB profile these achieve $\epsilon$6 and $\epsilon$7, respectively; on the Numenta corpus, conformal $\epsilon$8-NN$\epsilon$9 achieves $\alpha$0 versus the Numenta reference of $\alpha$1 (“Numenta™”) or $\alpha$2 (“Numenta” open-source), placing it among the top 3 entrants in the 2016 competition (Ishimtsev et al., 2017).

Outside classical time-series data, ICAD-based OOD detectors also show strong reported performance. iDECODe improves AUROC by $\alpha$3–$\alpha$4 points on CIFAR-10 natural-image OOD sets when moving from a base non-conformity score to aggregation over multiple transforms, improves mean AUROC from $\alpha$5 to $\alpha$6 on audio OOD, and pushes AUROC to approximately $\alpha$7–$\alpha$8 on DeepFool and CW adversarial examples. “Polysemantic Dropout” reports AUROC gains of $\alpha$9 to $p$0 over baselines on medical-specialized LLMs, with the merged-p-value method consistently highest in the reported table (Kaur et al., 2022, Gupta et al., 4 Sep 2025).

In low-data anomaly detection benchmarks, resampling-based conformal variants are reported to improve power relative to split-conformal ICAD while preserving Type I control. Using Isolation Forest on ten ADBench datasets, split-conformal power is described as often low, whereas CV, CV$p$1, Jackknife, and Jackknife$p$2 substantially increase power, especially on smaller datasets (Hennhöfer et al., 2024).

Scientific-search applications provide a different empirical perspective: not only power, but also calibration under dataset shift. In the LHC Olympics experiments, unweighted sideband calibration yields anti-conservative background $p$3-values with $p$4, whereas the label-free weighted correction restores $p$5. In a blind 5-window scan over $p$6 TeV, the weighted-conformal procedure with toy-null standardisation reports local $p$7 and discrete 5-window global $p$8, while asymptotic and unweighted procedures produce fake excesses (Araz et al., 11 Jun 2026).

7. Limitations, trade-offs, and recurring misconceptions

The most persistent misconception is that ICAD eliminates modeling assumptions altogether. The cited papers make a narrower claim: validity is distribution-free with respect to the score distribution, but it still depends on exchangeability, or on modified assumptions such as exchangeability within a Mondrian bin or weighted exchangeability under covariate shift. When those assumptions fail, raw conformal $p$9-values can be anti-conservative, as the new-physics study demonstrates explicitly (Araz et al., 11 Jun 2026).

A second trade-off is between calibration size and power. Small calibration sets produce coarse $p$00-value resolution and wider PAC-style slack in streaming settings; large calibration sets provide more reliable $p$01-values but may slow adaptation or reduce the amount of data available for fitting the base detector. This trade-off is stated directly for streaming $p$02-NN, where $p$03 balances adaptivity against statistical power, and for general split-conformal calibration, where smaller $p$04 yields a coarser $p$05-value grid and reduced power (Ishimtsev et al., 2017, Hennhöfer et al., 13 May 2026).

A third issue is computational. Split ICAD is cheap because it needs only one fit, but more data-efficient methods require many fits. For functional data, elastic-distance computation is $p$06 per test curve; for streaming $p$07-NN, each new non-conformity score costs $p$08 or $p$09 depending on the nearest-neighbor data structure; for wide scans in resonant searches, weighting and Gross–Vitells correction add toy-based calibration overhead (Adams et al., 1 Apr 2025, Ishimtsev et al., 2017, Araz et al., 11 Jun 2026).

Finally, ICAD’s practical role is often calibrative rather than generative. It does not, by itself, determine the best anomaly score; rather, it provides a statistically interpretable layer over scores derived from $p$10-NN, LOF, equivariance error, dropout tolerance, elastic distances, or copula-based tail models. This suggests that progress in ICAD research has two largely separable fronts: designing stronger non-conformity measures and extending the conformal calibration step to handle low-data regimes, structured dependence, and distribution shift.