Higher Criticism Statistic

• Higher Criticism Statistic is a method that aggregates and evaluates small p-values to detect sparse, weak signals in high-dimensional data.
• It leverages both empirical distributions and asymptotic theory to establish optimal detection boundaries in multiple-testing frameworks.
• Practical applications include multi-stream change-point detection with careful threshold calibration to minimize detection delays.

The higher criticism (HC) statistic is a central tool for large-scale detection of rare and weak signals, particularly within high-dimensional multiple-testing, change-point detection, and signal recovery problems. Introduced by Donoho and Jin, HC quantifies the aggregate excess of small $p$-values relative to the null expectation, enabling detection of alternatives where only a small, unknown fraction of features or streams are affected. The statistic has a rich mathematical structure, precise asymptotic theory, and connections to optimal detection boundaries in sparse regimes.

1. Definition and Formulation

Let $p_1,\dots, p_n$ be one-sided $p$-values, sorted so that $p_{(1)}\leq p_{(2)}\leq \cdots \leq p_{(n)}$. The canonical higher criticism statistic is

$$\mathrm{HC}_n = \max_{1 \leq i \leq n} \frac{\frac{i}{n} - p_{(i)}}{\sqrt{p_{(i)} (1-p_{(i)}) / n}}$$

In large-scale testing applications, it is customary to restrict $i$ to $1 \leq i \leq \alpha_0 n$ for some fixed $\alpha_0<1$; this restricts attention to the smallest $p$-values, which are most informative under sparse alternatives.

HC can also be written functionally in terms of the empirical distribution function $$\widehat{F}(x) = n^{-1} \sum_{j=1}^n \mathbf{1}\{p_j \leq x\}$$: $$\widehat{HC}(x) = \frac{\left|\widehat{F}(x) - x\right|}{\sqrt{\widehat{F}(x)(1-\widehat{F}(x))/n}}$$ and $$\mathrm{HC}_n^* = \max_{1 \leq i \leq n} \widehat{HC}(p_{(i)})$$.

In the context of sequential or multi-stream problems, the statistic is applied at each time $t$ to the set of per-stream $p$-values, leading to a sequence $\{\mathrm{HC}_t^\star\}$.

2. Operational Principle and Detection Boundary

HC is designed to optimally detect the presence of a sparse mixture, where an unknown, vanishing fraction of the population exhibits a weak deviation. Under the rare/weak normal means model—$X_j \sim (1-\epsilon) N(0,1) + \epsilon N(\mu,1)$ with $\epsilon = n^{-\beta}, \mu = \sqrt{2r\log n}$—there is a sharp "detection boundary" in the $(\beta, r)$ parameter space: $$\rho^*(\beta) = \begin{cases} \beta - \tfrac{1}{2}, & \tfrac{1}{2} < \beta < \tfrac{3}{4} \ (1 - \sqrt{1-\beta})^2, & \tfrac{3}{4} \leq \beta < 1 \end{cases}$$ For $r > \rho^*(\beta)$, HC is fully powered—i.e., asymptotic type I plus type II error tends to 0. Below this curve, no test, including HC, is powerful.

In heteroscedastic or multistream settings, the boundary generalizes. Let $p = N^{-\beta}$ be the affected stream fraction, $\mu = \sqrt{2r\log N}$, and for post-change variance $\sigma^2$: $$\rho^*(\beta, \sigma) = \begin{cases} (2-\sigma^2)(\beta-\tfrac{1}{2}), & \tfrac{1}{2}<\beta<1-\tfrac{\sigma^2}{4},\ 0<\sigma^2<2 \ [1-\sigma\sqrt{1-\beta}]^2, & 1-\tfrac{\sigma^2}{4}\leq\beta<1,\ 0<\sigma^2<2 \ 0, & \tfrac{1}{2}<\beta<1-\tfrac{1}{\sigma^2},\ \sigma^2\geq 2 \ [1-\sigma\sqrt{1-\beta}]^2, & 1-\tfrac{1}{\sigma^2}\leq\beta<1,\ \sigma^2\geq 2 \end{cases}$$ This boundary governs the minimum detection delay in multi-stream fastest change-point detection (Gong et al., 2024).

3. Calculation of Stream-wise $p$-values and Multi-stream Aggregation

In multi-stream change-point detection, consider observations $X_{n,t}$, $n=1,\dots, N$, $t \in \mathbb{N}$, where under the null $X_{n,t} \sim N(0,1)$, and under the alternative a sparse unknown subset of streams undergoes a post-$\tau$ shift to $N(\mu,\sigma^2)$. The per-stream $p$-values depend on the underlying detection statistic:

• CUSUM / Likelihood-Ratio (LR) Statistic ($\mu$ known, $\sigma=1$):

$$Y_t^{LR} = \max_{0 \leq k < t} [(S_t - S_k) - \tfrac{\mu}{2}(t-k)]\mu, \qquad S_t = \sum_{s=1}^t X_s$$

The $p$-value is $\pi_t^{LR}(x) = \mathbb{P}(Y_t^{LR} \geq x | H_0)$.

• Generalized Likelihood-Ratio (GLR) Statistic ($\mu$ unknown, $\sigma=1$):

$$Y_t^{GLR} = \max_{t-w < k < t} \frac{|S_t - S_k|}{\sqrt{t-k}}$$

The $p$-value is $$\pi_t^{GLR}(x) = \mathbb{P}(Y_t^{GLR} \geq x | H_0)$$.

For each time $t$, the set $\{\pi_{n,t}\}_{n=1}^N$ is constructed, ordered, and HC is applied to aggregate evidence across all streams.

4. Stopping Rule, False Alarm Control, and Detection Delay

The global detection procedure is

$T = \inf\{ t \geq 1 : \mathrm{HC}_t^\star > b_t \}$

where $\mathrm{HC}_t^\star$ is the HC statistic at time $t$ over streams $n = 1,\dots, N$ and $b_t$ is a threshold to guarantee a desired false-alarm rate (often taken constant).

Threshold Calibration:

• Under the null, one chooses $b_t = b(N)$ so that $$\sup_t \mathbb{P}(\mathrm{HC}_t^\star > b | H_0) \rightarrow 0$$ as $N \rightarrow \infty$.
• This can be achieved via Monte Carlo or from the large-sample null theory of HC, which gives asymptotic Gumbel-type distributions.

Detection Delay:

• When a change occurs at unknown time $\tau$, with $p = N^{-\beta}$ affected streams and mean shift $\mu = \sqrt{2r\log N}$, the delay converges in distribution: $$T-\tau \stackrel{d}{\rightarrow} \left\lceil \frac{\rho^*(\beta, \sigma)}{r} \right\rceil$$
• Under $H_0$, no alarm occurs with probability tending to 1.

Key Theorem ((Gong et al., 2024), Gong–Kipnis–Xie):

• There exists $b(N)$ such that (i) $\mathbb{P}(T<\infty|H_0)\to 0$, and (ii) $$\mathbb{P}(T-\tau = \lceil \rho^*(\beta, \sigma)/r \rceil | H_1) \to 1$$.
• Uniformly over $\tau$, the worst-case expected detection delay satisfies

$$\sup_\tau \mathbb{E}[(T-\tau) | T \geq \tau] = \lceil \rho^*(\beta, \sigma)/r \rceil + o(1)$$

5. Proof Techniques and Moderate Deviations Analysis

The proof combines:

• Uniformity under $H_0$: the per-stream $p$-values are i.i.d. Uniform(0,1), so the maximal HC is bounded by a threshold with high probability.
• Under $H_1$: the $N^{1-\beta}$ affected streams yield $p$-values exhibiting moderate-deviation (or log-$\chi^2$) behavior:

$$-2\log \pi_{n,t} \overset{d}{=} (\sigma Z + \mu\sqrt{t - \tau + 1})^2 (1 + o_p(1))$$

where $Z$ is standard normal. This drives a localized excess of small $p$-values detectable by HC.

• Classical HC power analysis (Donoho–Jin framework) demonstrates that the detection occurs as soon as $r(t-\tau+1) > \rho^*(\beta, \sigma)$, pinning down the minimal delay.

This approach generalizes to the heteroscedastic case ($\sigma \neq 1$), accommodating unknown post-change variances.

6. Implementation, Calibration, and Tuning Considerations

Algorithmic Steps:

1. For each time $t$, and each stream $n$, compute a change-point detection statistic ($Y_{n,t}^{LR}$ or $Y_{n,t}^{GLR}$).
2. Calculate per-stream $p$-values using the exact null distribution.
3. Collect and sort these $p$-values; compute $\mathrm{HC}_t^\star$ using a rank cutoff (e.g., top $\alpha_0 N$ smallest $p$-values).
4. Signal a detected change if $\mathrm{HC}_t^\star > b_t$.

Threshold $b_t$ determination:

• Can be set empirically via Monte Carlo under the null model, or via asymptotic approximations: for large $N$, $\mathrm{HC}_N$ is approximately Gumbel, scaling as $O(\sqrt{2\log\log N})$.
• Limiting null distributions may be slow to set in finite $N$; empirical calibration is often preferred for stringent control.

Practical recommendations:

• For large $N$, restrict maximization to $i \leq \alpha_0 N$ (e.g., $\alpha_0 = 0.2$ or $0.5$) to avoid instability from extreme-order $p$-values.
• Under strong dependence or heteroscedastic variance, ensure uniformity of null $p$-values holds.
• Computational cost is $O(N\log N)$ per time step (due to sorting and scan).

7. Significance, Limitations, and Comparison to Information-Theoretic Bounds

Significance:

• HC attains the optimal (information-theoretic) detection delay for sparse change-point detection, without requiring knowledge of which streams are affected or the precise value of $\mu$.
• The approach extends to general settings (unknown variance, weak or moderate signals, heteroscedasticity) as long as streamwise $p$-values are exactly or approximately uniform under the null.

Limitations:

• When the fraction of affected streams $p$ is not sparse (i.e., $\beta$ near 0), HC is suboptimal compared to bulk averaging procedures.
• Under heavy-tailed or serially dependent data, uniformity of $p$-values may fail, requiring cautious model checking or adaptation.
• In the very low-count regime, phase transitions in detectability become non-Gaussian, and HC may require thresholding or cell selection for optimality (see (Chan, 2023)).

Comparison:

• In the special case $\sigma=1$, the HC-based procedure matches the delay lower bound derived in prior work (Chan, 2017), achieving minimax optimality among sequential detectors.
• The derived phase diagram in $(\beta, r)$ precisely coincides with the Donoho–Jin boundary, generalizing the result from mean-shift detection in high-dimensional mean testing to quickest change detection in multi-stream scenarios.

Summary Table: Detection Delay and Boundary

Model Parameterization	Detection Delay	Detection Boundary $\rho^*$
$p=N^{-\beta}$, $\mu=\sqrt{2r\log N}$, $\sigma^2=1$	$T-\tau \sim (\rho^*(\beta,1)/r)$	$$\rho^*(\beta,1) = [\beta-\frac{1}{2}]\ (\frac{1}{2}<\beta<\frac{3}{4})$$<br>$~=(1-\sqrt{1-\beta})^2\ (\frac{3}{4} \leq \beta<1)$
$p=N^{-\beta}$, $\mu=\sqrt{2r\log N}$, $\sigma^2\neq1$	$T-\tau \sim (\rho^*(\beta,\sigma)/r)$	See definition in Section 2

HC for multi-stream change-point detection achieves the theoretical detection delay lower bound under general settings, provided careful calibration and accurate $p$-value computation are maintained. This framework is robust, adaptive, and achieves rate-optimal performance without requiring explicit signal localization.