---
title: Higher Criticism Statistic
url: https://www.emergentmind.com/topics/higher-criticism-statistic
type: topic
---

# Higher Criticism Statistic

The higher criticism (HC) statistic is a central tool for large-scale detection of rare and weak signals, particularly within high-dimensional multiple-testing, change-point detection, and signal recovery problems. Introduced by Donoho and Jin, HC quantifies the aggregate excess of small $p$-values relative to the null expectation, enabling detection of alternatives where only a small, unknown fraction of features or streams are affected. The statistic has a rich mathematical structure, precise asymptotic theory, and connections to optimal detection boundaries in sparse regimes.

## 1. Definition and Formulation

Let $p_1,\dots, p_n$ be one-sided $p$-values, sorted so that $p_{(1)}\leq p_{(2)}\leq \cdots \leq p_{(n)}$. The canonical higher criticism statistic is
\[
\mathrm{HC}_n = \max_{1 \leq i \leq n} \frac{\frac{i}{n} - p_{(i)}}{\sqrt{p_{(i)} (1-p_{(i)}) / n}}
\]
In large-scale testing applications, it is customary to restrict $i$ to $1 \leq i \leq \alpha_0 n$ for some fixed $\alpha_0<1$; this restricts attention to the smallest $p$-values, which are most informative under sparse alternatives.

HC can also be written functionally in terms of the empirical distribution function $\widehat{F}(x) = n^{-1} \sum_{j=1}^n \mathbf{1}\{p_j \leq x\}$:
\[
\widehat{HC}(x) = \frac{\left|\widehat{F}(x) - x\right|}{\sqrt{\widehat{F}(x)(1-\widehat{F}(x))/n}}
\]
and $\mathrm{HC}_n^* = \max_{1 \leq i \leq n} \widehat{HC}(p_{(i)})$.

In the context of sequential or multi-stream problems, the statistic is applied at each time $t$ to the set of per-stream $p$-values, leading to a sequence $\{\mathrm{HC}_t^\star\}$.

## 2. Operational Principle and Detection Boundary

HC is designed to optimally detect the presence of a sparse mixture, where an unknown, vanishing fraction of the population exhibits a weak deviation. Under the rare/weak normal means model—$X_j \sim (1-\epsilon) N(0,1) + \epsilon N(\mu,1)$ with $\epsilon = n^{-\beta}, \mu = \sqrt{2r\log n}$—there is a sharp "detection boundary" in the $(\beta, r)$ parameter space:
\[
\rho^*(\beta) =
\begin{cases}
  \beta - \tfrac{1}{2}, & \tfrac{1}{2} < \beta < \tfrac{3}{4} \\
  (1 - \sqrt{1-\beta})^2, & \tfrac{3}{4} \leq \beta < 1
\end{cases}
\]
For $r > \rho^*(\beta)$, HC is fully powered—i.e., asymptotic type I plus type II error tends to 0. Below this curve, no test, including HC, is powerful.

In heteroscedastic or multistream settings, the boundary generalizes. Let $p = N^{-\beta}$ be the affected stream fraction, $\mu = \sqrt{2r\log N}$, and for post-change variance $\sigma^2$:
\[
\rho^*(\beta, \sigma) =
\begin{cases}
  (2-\sigma^2)(\beta-\tfrac{1}{2}), & \tfrac{1}{2}<\beta<1-\tfrac{\sigma^2}{4},\ 0<\sigma^2<2 \\
  [1-\sigma\sqrt{1-\beta}]^2, & 1-\tfrac{\sigma^2}{4}\leq\beta<1,\ 0<\sigma^2<2 \\
  0, & \tfrac{1}{2}<\beta<1-\tfrac{1}{\sigma^2},\ \sigma^2\geq 2 \\
  [1-\sigma\sqrt{1-\beta}]^2, & 1-\tfrac{1}{\sigma^2}\leq\beta<1,\ \sigma^2\geq 2
\end{cases}
\]
This boundary governs the minimum detection delay in multi-stream fastest change-point detection [2409.15597].

## 3. Calculation of Stream-wise $p$-values and Multi-stream Aggregation

In multi-stream change-point detection, consider observations $X_{n,t}$, $n=1,\dots, N$, $t \in \mathbb{N}$, where under the null $X_{n,t} \sim N(0,1)$, and under the alternative a sparse unknown subset of streams undergoes a post-$\tau$ shift to $N(\mu,\sigma^2)$. The per-stream $p$-values depend on the underlying detection statistic:

- **CUSUM / Likelihood-Ratio (LR) Statistic ($\mu$ known, $\sigma=1$):**
  \[
  Y_t^{LR} = \max_{0 \leq k < t} [(S_t - S_k) - \tfrac{\mu}{2}(t-k)]\mu, \qquad S_t = \sum_{s=1}^t X_s
  \]
  The $p$-value is $\pi_t^{LR}(x) = \mathbb{P}(Y_t^{LR} \geq x | H_0)$.

- **Generalized Likelihood-Ratio (GLR) Statistic ($\mu$ unknown, $\sigma=1$):**
  \[
  Y_t^{GLR} = \max_{t-w < k < t} \frac{|S_t - S_k|}{\sqrt{t-k}}
  \]
  The $p$-value is $\pi_t^{GLR}(x) = \mathbb{P}(Y_t^{GLR} \geq x | H_0)$.

For each time $t$, the set $\{\pi_{n,t}\}_{n=1}^N$ is constructed, ordered, and HC is applied to aggregate evidence across all streams.

## 4. Stopping Rule, False Alarm Control, and Detection Delay

The global detection procedure is
\[
T = \inf\{ t \geq 1 : \mathrm{HC}_t^\star > b_t \}
\]
where $\mathrm{HC}_t^\star$ is the HC statistic at time $t$ over streams $n = 1,\dots, N$ and $b_t$ is a threshold to guarantee a desired false-alarm rate (often taken constant).

**Threshold Calibration:**
- Under the null, one chooses $b_t = b(N)$ so that $\sup_t \mathbb{P}(\mathrm{HC}_t^\star > b | H_0) \rightarrow 0$ as $N \rightarrow \infty$.
- This can be achieved via Monte Carlo or from the large-sample null theory of HC, which gives asymptotic Gumbel-type distributions.

**Detection Delay:**
- When a change occurs at unknown time $\tau$, with $p = N^{-\beta}$ affected streams and mean shift $\mu = \sqrt{2r\log N}$, the delay converges in distribution:
\[
T-\tau \stackrel{d}{\rightarrow} \left\lceil \frac{\rho^*(\beta, \sigma)}{r} \right\rceil
\]
- Under $H_0$, no alarm occurs with probability tending to 1.

**Key Theorem ([2409.15597], Gong–Kipnis–Xie):**
- There exists $b(N)$ such that (i) $\mathbb{P}(T<\infty|H_0)\to 0$, and (ii) $\mathbb{P}(T-\tau = \lceil \rho^*(\beta, \sigma)/r \rceil | H_1) \to 1$.
- Uniformly over $\tau$, the worst-case expected detection delay satisfies
\[
\sup_\tau \mathbb{E}[(T-\tau) | T \geq \tau] = \lceil \rho^*(\beta, \sigma)/r \rceil + o(1)
\]

## 5. Proof Techniques and Moderate Deviations Analysis

The proof combines:

- Uniformity under $H_0$: the per-stream $p$-values are i.i.d. Uniform(0,1), so the maximal HC is bounded by a threshold with high probability.
- Under $H_1$: the $N^{1-\beta}$ affected streams yield $p$-values exhibiting moderate-deviation (or log-$\chi^2$) behavior:
  \[
  -2\log \pi_{n,t} \overset{d}{=} (\sigma Z + \mu\sqrt{t - \tau + 1})^2 (1 + o_p(1))
  \]
  where $Z$ is standard normal. This drives a localized excess of small $p$-values detectable by HC.
- Classical HC power analysis (Donoho–Jin framework) demonstrates that the detection occurs as soon as $r(t-\tau+1) > \rho^*(\beta, \sigma)$, pinning down the minimal delay.

This approach generalizes to the heteroscedastic case ($\sigma \neq 1$), accommodating unknown post-change variances.

## 6. Implementation, Calibration, and Tuning Considerations

**Algorithmic Steps:**
1. For each time $t$, and each stream $n$, compute a change-point detection statistic ($Y_{n,t}^{LR}$ or $Y_{n,t}^{GLR}$).
2. Calculate per-stream $p$-values using the exact null distribution.
3. Collect and sort these $p$-values; compute $\mathrm{HC}_t^\star$ using a rank cutoff (e.g., top $\alpha_0 N$ smallest $p$-values).
4. Signal a detected change if $\mathrm{HC}_t^\star > b_t$.

**Threshold $b_t$ determination:**
- Can be set empirically via Monte Carlo under the null model, or via asymptotic approximations: for large $N$, $\mathrm{HC}_N$ is approximately Gumbel, scaling as $O(\sqrt{2\log\log N})$.
- Limiting null distributions may be slow to set in finite $N$; empirical calibration is often preferred for stringent control.

**Practical recommendations:**
- For large $N$, restrict maximization to $i \leq \alpha_0 N$ (e.g., $\alpha_0 = 0.2$ or $0.5$) to avoid instability from extreme-order $p$-values.
- Under strong dependence or heteroscedastic variance, ensure uniformity of null $p$-values holds.
- Computational cost is $O(N\log N)$ per time step (due to sorting and scan).

## 7. Significance, Limitations, and Comparison to Information-Theoretic Bounds

**Significance:**
- HC attains the optimal (information-theoretic) detection delay for sparse change-point detection, without requiring knowledge of which streams are affected or the precise value of $\mu$.
- The approach extends to general settings (unknown variance, weak or moderate signals, heteroscedasticity) as long as streamwise $p$-values are exactly or approximately uniform under the null.

**Limitations:**
- When the fraction of affected streams $p$ is not sparse (i.e., $\beta$ near 0), HC is suboptimal compared to bulk averaging procedures.
- Under heavy-tailed or serially dependent data, uniformity of $p$-values may fail, requiring cautious model checking or adaptation.
- In the very low-count regime, phase transitions in detectability become non-Gaussian, and HC may require thresholding or cell selection for optimality (see [2311.03763]).

**Comparison:**
- In the special case $\sigma=1$, the HC-based procedure matches the delay lower bound derived in prior work (Chan, 2017), achieving minimax optimality among sequential detectors.
- The derived phase diagram in $(\beta, r)$ precisely coincides with the Donoho–Jin boundary, generalizing the result from mean-shift detection in high-dimensional mean testing to quickest change detection in multi-stream scenarios.

**Summary Table: Detection Delay and Boundary**

| Model Parameterization            | Detection Delay                    | Detection Boundary $\rho^*$                   |
|-----------------------------------|------------------------------------|-----------------------------------------------|
| $p=N^{-\beta}$, $\mu=\sqrt{2r\log N}$, $\sigma^2=1$ | $T-\tau \sim (\rho^*(\beta,1)/r)$  | $\rho^*(\beta,1) = [\beta-\frac{1}{2}]\ (\frac{1}{2}<\beta<\frac{3}{4})$<br>$~=(1-\sqrt{1-\beta})^2\ (\frac{3}{4} \leq \beta<1)$                 |
| $p=N^{-\beta}$, $\mu=\sqrt{2r\log N}$, $\sigma^2\neq1$ | $T-\tau \sim (\rho^*(\beta,\sigma)/r)$  | See definition in Section 2                   |

HC for multi-stream change-point detection achieves the theoretical detection delay lower bound under general settings, provided careful calibration and accurate $p$-value computation are maintained. This framework is robust, adaptive, and achieves rate-optimal performance without requiring explicit signal localization.

Source: https://www.emergentmind.com/topics/higher-criticism-statistic