Unified Asymptotic Efficiency Theory

• Unified Asymptotic Efficiency Theory defines a robust framework that quantifies performance limits for p-mean tests by relating sample size, detection power, and alternative structures in high-dimensional settings.
• It leverages advanced probabilistic methods, including the Berry–Esseen bound and Schur-convexity, to establish explicit scaling conditions and phase transitions in test efficiency.
• The theory provides actionable insights for selecting optimal test statistics by demonstrating how efficiency varies with the sparsity or density of the alternative mean vector.

Unified Asymptotic Efficiency Theory encompasses a rigorous and unified characterization of the performance limits for statistical tests of multivariate mean vectors based on p-mean functionals in high-dimensional spaces. This theory captures, with mathematical precision, how the detection power and required sample size of different testing procedures relate in the high-dimensional asymptotic regime, and reveals a sharp phase transition in their relative efficiency that depends intricately on the structure (“direction”) of the alternative mean vector. The framework leverages advances in classical probability theory, such as the Berry–Esseen bound, and modern results on the Schur-convexity properties of Gaussian measures, to establish a complete solution to the efficiency comparison problem for p-mean-based tests in this setting.

1. Asymptotic Efficiency for p-mean Tests

Let $X_1, \ldots, X_n$ be i.i.d. $\mathbb{R}^d$-valued observations from a normal distribution with mean $\theta$ and known identity covariance, and denote $S_n = n^{-1} \sum_{i=1}^n X_i$. For each $p \in \mathbb{R}$ (with definitions extended by continuity for $p=0,\pm\infty$), the $p$-mean statistic is

$$x_p = \left( d^{-1} \sum_{j=1}^d |x_j|^p \right)^{1/p},$$

and the associated test for $H_0: \theta = 0$ against $H_1: \theta = \theta_1$ rejects for large values of $x_p(S_n)$.

Asymptotic relative efficiency (ARE), denoted $\mathrm{ARE}_{p,2,u}$, quantifies the limit (as $d\to\infty$) of $n_2 / n_p$ for which the $p$-mean and $2$-mean (likelihood ratio test, LRT) have prescribed asymptotic levels $\alpha$ (type I) and $\beta$ (power) for a fixed “direction” $u$ (the unit vector of the scaled mean shift). The $p$-mean test is asymptotically efficient if, for suitable $n$ and $\theta_1$, the following scaling holds: $$\sum_{j=1}^d f_p\left( \sqrt{n} \theta_{1,j} \right) \sim K_p \Delta_p(d),$$ where $f_p(\cdot), K_p, \Delta_p(d)$ are explicit quantities depending on $p$ (see paper for precise definitions). These “just-sufficient” scaling conditions provide necessary and sufficient requirements for prescribed asymptotic power.

Moreover, asymptotic relative efficiency admits the explicit oracle form: $$\mathrm{ARE}_{p,2,u} = \lim_{d\to\infty} \|s_2\|^2 / \|s_p\|^2,$$ where $s_2, s_p$ are the minimal shifts needed for the 2-mean and $p$-mean tests, respectively, to reach the prescribed power against alternatives in direction $u$.

2. Comparison of p-mean Tests with LRT (p=2)

For $p=2$, the test equals the standard LRT; $x_2$ boils down to the Euclidean (ℓ2) norm. For $p \ne 2$, new behaviors emerge:

• For $p>2$, tests become highly sensitive to “unequalized” alternatives—i.e., alternatives where only a few coordinates of $\theta_1$ are large. The ARE exhibits a phase transition: for sparse alternatives (few large components), $\mathrm{ARE}_{p,2,u}$ can be arbitrarily large (even infinite), indicating that the $p$-mean test vastly outperforms the LRT. For “equalized” alternatives (many components share the signal), ARE is $O(1)$.
• Explicitly, for $p=3$, $\mathrm{ARE}_{3,2}$ ranges from about 0.96 (completely equalized) up to infinity (one dominant coordinate).

The efficiency phase transition is characterized, in the high-dimensional regime, by a threshold on the $p$-mean $u_p$ of the direction vector: $$\mathrm{ARE}_{p,2,u} = \begin{cases} a_p, & \text{if}\ u_p \ll d^{(p-2)/(4p)}, \ \infty, & \text{if}\ u_p \gg d^{(p-2)/(4p)}. \end{cases}$$ This demonstrates that no $p$-mean test is uniformly superior to the LRT; choice of $p$ must be matched to the anticipated alternative structure.

3. Full Characterization of (n, $\theta_1$) Pairs for Prescribed Power

A complete solution is given for which sequences $(n, \theta_1)$ (or their higher-dimensional analogues) ensure that the $p$-mean test achieves size $\alpha$ and power $\beta$ as $d \to \infty$: $$\sum_{j=1}^d f_p\left( \sqrt{n} \theta_{1,j} \right) \sim K_p \Delta_p(d).$$ This characterization is valid uniformly over a wide range of alternatives—including those where the signal is arbitrarily sparse or dense. Notably, the precise forms of $f_p, K_p, \Delta_p$ differ by $p$, demarcating regimes such as $p<0$, $0$p>2$. Thus, the theory yields a unified, dimension-robust parametrization of the testing trade-off between sample size, effect configuration, and power.

4. Mathematical Methods—Berry–Esseen Bound, Infinitely Divisible Limits, Schur–Convexity

The paper’s proofs use several advanced probabilistic and convex analysis tools:

• Berry–Esseen Theorem: Controls the convergence rates for the normalized sum statistics to the limiting Gaussian (or non-Gaussian) laws, crucial for calibrating critical values and establishing limiting power.
• Convergence to Infinitely Divisible Laws: In regimes with heavy-tailed $p$, the limiting law of the $p$-mean statistic may be non-Gaussian; convergence is analyzed using the Lévy–Khintchine representation of infinitely divisible distributions.
• Schur-convexity/concavity: Recent results are leveraged to establish that the ARE function is Schur-convex (for $p>2$) or Schur-concave ($p<2$) in the vector $(u_1^2,\ldots,u_d^2)$, encoding that efficiency increases as alternatives become more sparse for $p>2$, and the reverse for $p<2$. This majorization ordering yields precise phase transition boundaries for efficiency.

The proofs are organized hierarchically: foundational lemmas (Berry–Esseen, Schur properties) underpin core propositions, which are then assembled into the global theorems describing ARE and the scaling of alternatives.

5. Implications for Unified Theory and Applications

The developed unified theory of asymptotic efficiency:

• Reveals that $p$-mean tests for $p>2$ deliver large efficiency gains against sparse alternatives, sometimes infinitely outpacing LRTs, while suffering only minor losses ($<5\%$) when alternatives are not sparse.
• Provides actionable prescriptions for test design: if the practitioner anticipates alternatives with only a few large mean components, $p>2$ should be chosen; else, the classical LRT (or even $p<2$) suffices.
• Suggests that the $p$-mean family of tests is robust in high-dimensions: never much worse than LRT and potentially orders of magnitude better, depending on the alternative.

The theory applies directly to signal detection, multiple testing, and sparse recovery settings where one must balance the power against a universe of possible alternative configurations. The explicit, unified formulae for required sample size, prescribed power, and ARE function enable practitioners to tune testing procedures to the anticipated alternative structure.

Future research could address adaptive methods for $p$ selection, extend to dependent or heavy-tailed scenarios, or seek minimax-optimal choices over classes of alternatives.

Summary Table: Key Ingredients and Results

Concept	Mathematical Formulation or Tool	Implementation Impact
p-mean statistic	$x_p = (d^{-1}\sum_j |x_j|^p)^{1/p}$	Index class of tests/power structures
Asymptotic Relative Efficiency (ARE)	$\mathrm{ARE}_{p,2,u}$ = $\lim (n_2/n_p)$ as $d\to\infty$	Directly compares test sample complexity
JUST-sufficient scaling condition	$$\sum_j f_p(\sqrt{n}\theta_{1,j}) \sim K_p\Delta_p(d)$$	Full characterization of (n, $\theta_1$)
Phase transition in ARE	Threshold: $u_p \asymp d^{(p-2)/(4p)}$	Guides selection of p-mean test
Schur–convexity/concavity	Efficiency increases with sparsity (p>2), decreases (p<2)	Non-uniform dominance in alternatives

Conclusion

Unified Asymptotic Efficiency Theory for $p$-mean tests in high dimensions establishes exact conditions under which alternative configurations and sample sizes yield prescribed power, derives sharp relative efficiency results that depend on the sparsity structure of alternatives, and provides both a comprehensive mathematical framework and actionable guidance for high-dimensional hypothesis testing and related inference tasks (Pinelis, 2010).