---
title: Signature-Based Test Statistics
url: https://www.emergentmind.com/topics/signature-based-test-statistics
type: topic
---

# Signature-Based Test Statistics

Signature-based test statistics form a class of statistical methods designed to detect structured deviations from a null hypothesis by transforming and summarizing raw data using specific signatures—mathematical constructions that compactly encode key features or ordered structure in the data. These statistics offer distinctive sensitivity to non-global departures (such as local anomalies or multimodality) and are increasingly utilized in clustering, anomaly detection, and hypothesis testing. Two prominent approaches include the Signature Test (Sigtest) for unimodality and the weighted-runs statistic for local deviation detection [1401.1895][1005.3233].

## 1. Mathematical Formulation and Problem Setting

Signature-based tests operate by mapping a collection of observed scalar samples, often denoted as $y = [y_1,\dots,y_N]^T \in \mathbb{R}^N$, to a signature sequence designed to expose structure hidden in the original data. The typical hypothesis testing scenario involves a composite hypothesis, such as:

\[
H_0: \text{The distribution of } y \text{ is unimodal} \quad \text{vs.} \quad H_1: \text{not unimodal}
\]

or, in the context of evaluating local deviations,

\[
H_0: X_i \sim \mathcal{N}(\mu_i, \sigma_i^2) \ \forall i,\quad H_1: \text{Existence of local structure or clusters of anomalous deviations}
\]

Signature transformation techniques often exploit sorting, cumulative means, or runs of consecutive exceedances to reduce noise and concentrate statistical evidence relating to $H_1$. The transformation is governed by explicit forms such as order statistics, empirical quantiles, or cumulative averages.

## 2. Signature Test (Sigtest) for Unimodality

Sigtest specifically targets unimodality by transforming the data via absolute value sorting:

\[
z = [z_1, \dots, z_N]^T = \text{sort}\left(|y|\right)
\]

Two signatures are considered:

- **Identity signature:** $g_1(z_n) = z_n$
- **Cumulative-mean signature:** $g_2(z_n) = \frac{1}{n}\sum_{j=1}^n z_j$

These signatures are chosen because, for data drawn from a unimodal distribution, their sequence exhibits sharply reduced variance. The method exploits pointwise probabilistic envelopes:

\[
U(n) = E[g_i(w_n)] + \gamma \sqrt{\operatorname{Var}(g_i(w_n))},\qquad L(n) = E[g_i(w_n)] - \gamma \sqrt{\operatorname{Var}(g_i(w_n))}
\]

where $w_n$ are the ideal sorted absolute values under $H_0$, and $\gamma$ calibrates the width of the confidence band. The following decision rule is applied:

\[
c_n = 
\begin{cases}
0 & \text{if } L(n) \leq g_i(z_n) \leq U(n) \\
1 & \text{otherwise}
\end{cases},\qquad
C = \frac{1}{N} \sum_{n=1}^N c_n
\]

The null hypothesis $H_0$ is rejected (evidence for multi-modality) if $C > T$ for a chosen threshold $T$ [1401.1895].

## 3. Weighted-Runs Test Statistic for Local Deviation Detection

Motivated by the insensitivity of classical global statistics (such as $\chi^2$) to clustering of deviations, the weighted-runs statistic tests for local, consecutive excesses:

- An observation $X_i > \mu_i$ is a "success", otherwise a "failure".
- The sequence is partitioned into alternating maximal runs of successes/failures.
- For each success run $A_j$, compute the run-weight:

\[
w(A_j) = \sum_{X_i \in A_j} \frac{(X_i - \mu_i)^2}{\sigma_i^2}
\]

- The test statistic is

\[
T = \max_{1 \leq j \leq M} w(A_j)
\]

where $M$ is the number of success runs. $T$ measures the size of the most extreme local deviation in the data, distinguishing it from the classical aggregate $\chi^2$ which is order-invariant [1005.3233].

## 4. Sampling Distributions, P-value Computation, and Calibration

Both approaches provide exact or approximate null distributions for hypothesis testing:

- **Sigtest:** Computes expected value and variance of the signature sequence under any unimodal distribution using classical order statistics theory. Confidence bands are calibrated deterministically via $\gamma$.
- **Weighted-runs:** The null distribution is derived via combinatorial summation over all integer partitions of the number of successes, accounting for all possible run-length profiles. Explicit formulas relate the cumulative probability:

\[
P(T < T_{\text{obs}}\,|\,N) = \sum_{r=1}^N \sum_{M=1}^{\min(r,N-r+1)} \sum_{\pi \vdash r, |\pi|=M} \frac{W(\pi)}{2^N-1} \prod_{\ell=1}^r [F_{\chi^2_\ell}(T_{\text{obs}})]^{n_\ell}
\]

where $n_\ell$ is the number of runs of length $\ell$ in the partition $\pi$ and $F_{\chi^2_\ell}$ is the chi-squared CDF. For $N \lesssim 80$, exact $p$-values are tractable; for larger $N$, Monte Carlo methods or empirical pre-tabulation are used [1005.3233].

## 5. Computational Complexity and Empirical Performance

- **Sigtest:** Sorting dominates with $O(N\log N)$ complexity. All signature and decision operations are $O(N)$. This is markedly faster than Hartigan’s dip ($O(N^2)$), KS, or AD tests, especially under calibration via resampling [1401.1895].
- **Weighted-runs:** Enumeration of all integer partitions grows nearly exponentially, $\nu(N) = p(N+1)-1$ (where $p(n)$ is the partition function). For small to moderate $N$, this is feasible; for large $N$, the Monte Carlo approach offers $O(KN)$ complexity for $K$ simulations [1005.3233].

Empirical results demonstrate the higher sensitivity of signature-based statistics to local and multimodal features. For instance, in the detection of weakly separated normal mixtures, Sigtest's detection rate dominates that of AD, KS, and dip tests, with sub-millisecond average computation times. Weighted-runs achieves up to 30–40 percentage points higher power than global $\chi^2$ in detecting localized anomalies.

| Test       | 2σ   | 2.25σ | 2.5σ | 2.8σ | 3σ   | Time (s)    |
|------------|------|-------|------|------|------|-------------|
| Sigtest₂   | 56%  | 93%   | 99%  | 100% | 100% | 0.2×10⁻⁴    |
| Sigtest₁   | 69%  | 97%   |100%  | 100% | 100% | 0.2×10⁻⁴    |
| AD         | 29%  | 76%   | 97%  | 100% | 100% | 3.96×10⁻⁴   |
| KS         | 10%  | 37%   | 74%  | 95%  | 100% | 30×10⁻⁴     |
| Dip        | 3%   | 8%    | 21%  | 82%  | 94%  | 2197×10⁻⁴   |

## 6. Applications in Clustering and Anomaly Detection

Signature-based tests are integrated within top-down hierarchical clustering workflows as splitting criteria. Projected data from a candidate cluster undergoes a signature transformation and is tested for unimodality. If Sigtest or a weighted-runs statistic rejects the unimodal or i.i.d. null, the cluster is split (commonly via $k$-means). Empirical studies show that using Sigtest in place of classical methods yields improved clustering quality as measured by Variation of Information and Adjusted Rand Index. For example, on the Optical Digits UCI set, G-means$^+$ attains $VI=1.14\pm0.10$, $ARI=0.66\pm0.04$, compared to $VI=1.31\pm0.08$, $ARI=0.57\pm0.03$ for standard G-means [1401.1895].

## 7. Theoretical Properties and Implementation Considerations

Theoretical justification derives from the variance-reducing effect of the signature mapping under the null hypothesis, which tightens confidence bands and magnifies sensitivity to departures (secondary modes, localized excesses). Practically, implementation requires careful numerical treatment (e.g., logarithmic evaluation of incomplete gamma functions for weighted-runs), and efficient algorithms for partition enumeration or simulation for larger data sizes. Both methods are robust to composite nulls if null simulations use fitted parameterizations, and can be implemented in a few dozens of lines in high-level scientific computing environments [1005.3233][1401.1895]. 

A plausible implication is that signature-based statistics, given their computational efficiency and statistical power for structured alternatives, will continue to play a critical role in large-scale, high-dimensional inference. They are particularly well-suited as plug-in modules in clustering, signal segmentation, and local anomaly detection.

Source: https://www.emergentmind.com/topics/signature-based-test-statistics