Tetrachotomy of Optimal Rates

• Tetrachotomy of Optimal Rates is a rigorous classification that divides convergence rates into four distinct regimes based on structural properties like ill-posedness and combinatorial complexity.
• The framework applies across diverse domains—such as agnostic learning, ERM, NPIV regression, and Gaussian process estimation—by matching minimax lower bounds with algorithmic upper bounds.
• This taxonomy aids estimator design by linking practical criteria (e.g., operator ill-posedness, combinatorial dimensions, process memory) to achievable convergence rates.

The tetrachotomy of optimal rates refers to a rigorous fourfold classification emerging in diverse areas of statistical learning, estimation, and inference, where all admissible rates fall into precisely four regimes. Across fields such as agnostic and empirical risk minimization, nonparametric instrumental variables regression, phylogenetic inference, and Gaussian process parameter estimation, the rate-tetrachotomy delineates the boundary between learnability, attainable efficiency, and the underlying mechanism or complexity that dictates such rates. This concept organizes possibilities into an exhaustive partition based on operator ill-posedness, combinatorial dimension, process memory, or other structural features, providing sharp necessary and sufficient conditions for each regime.

1. General Form of Tetrachotomy in Optimal Rates

The tetrachotomy classification manifests where two orthogonal dichotomies—such as norm type and degree of ill-posedness, or combinatorial property and distributional regularity—cross, yielding a four-region grid of rate regimes. In each scenario, matching minimax lower and algorithmic upper bounds define each rate regime's sharpness, highlighting both achievable and unattainable regions. These classifications reflect core structure: for instance, whether a functional class is finite, possesses infinite complexity of a particular combinatorial type, or whether a stochastic process falls into one of several memory regimes.

Across applications, the four regimes identified universally include:

1. Exponential/Ultra-fast: Rates such as $e^{-n}$, characteristic of finite models or classes with bounded combinatorial dimension.
2. Polynomial/Intermediate: Rates such as $n^{-1}$, $n^{-a}$, or involving logarithmic factors, typically arising under moderate complexity or mild ill-posedness.
3. Logarithmic/refined polynomial: Rates such as $\log n / n$, $1/(\log n)^a$, or similar, appearing at critical transitions or under near-critical complexity/memory.
4. Arbitrarily Slow/Nonparametric Limit: Rates that can be made slower than any pre-specified vanishing envelope, dictated by extreme complexity or severe ill-posedness.

This fourfold structure provides a universal language to describe sharp statistical phase transitions, where changes in operator, combinatorial, or memory structure yield a sudden drop in attainable rate.

2. Tetrachotomy in Universal and Agnostic Learning

In the context of universal agnostic binary classification, the tetrachotomy is precisely characterized in "A Theory of Universal Agnostic Learning" (Hanneke et al., 28 Jan 2026). For any concept class $H \subset \{0,1\}^X$, the optimal universal convergence rate for minimax excess error in agnostic learning is one of exactly four:

Regime	Structural Condition	Optimal Rate
Exponential	$H$ finite	$e^{-n}$
Near-exponential	$H$ infinite, no infinite Littlestone tree	$e^{-o(n)}$
Super-root	Infinite Littlestone, no infinite VCL tree	$o(n^{-1/2})$
Arbitrarily slow	Infinite VCL tree	no fixed rate

Key combinatorial objects—Littlestone trees and VCL-trees—determine each regime. The Littlestone tree captures the sequential mistake bound (online learnability), while the VCL tree intertwines VC-shattering at each node, signaling intractable learning. Representative examples clarify the range:

• Finite $H$: Exponential.
• Countable thresholds: Near-exponential.
• Real-valued thresholds: Super-root, faster than $n^{-1/2}$ but not exponential.
• Infinite axis-aligned monotone classifiers: Arbitrarily slow, no agnostic rate.

This structure extends and refines the classical dichotomy—realizable-case trichotomy and agnostic-case dichotomy—by revealing and precisely characterizing the two intermediate behaviors unique to universal agnostic learning.

3. Tetrachotomy in Empirical Risk Minimization

In the realizable-case setting for empirical risk minimization (ERM), Hanneke and Xu (Hanneke et al., 2024) establish that all possible universal learning curves for worst-case error of ERM decay at exactly one of four rates, determined by the concept class's combinatorial structure:

Regime	Defining Property	Rate
Exponential	$|H|<\infty$ / finite eluder dimension	$e^{-n}$
Linear	Infinite eluder, finite star-eluder	$1/n$
Log-linear	Infinite star-eluder, finite VC-eluder ($\VC(H)<\infty$)	$\log n / n$
Arbitrarily slow	Infinite VC-eluder ($\VC(H)=\infty$)	o(1), arbitrarily slow

Combinatorial dimensions—eluder, star-eluder, VC-eluder—provide if-and-only-if criteria for each phase. The analysis also delivers asymptotically tight bounds in the polynomial cases. Each regime is witnessed by a corresponding combinatorial obstruction: infinite eluder precludes exponential, infinite star-eluder precludes linear, infinite VC-eluder precludes log-linear.

The tetrachotomy framework here not only exhausts all theoretical possibilities but guides the selection and interpretation of ERM performance for arbitrary concept classes, contrasting with the more limited classical PAC model.

4. Tetrachotomy in Nonparametric Instrumental Variables Regression

In nonparametric instrumental variables regression (NPIV), the convergence rate for estimation in either sup-norm or $L^2$ norm depends on the interplay between the norm type and the degree of ill-posedness of the operator, as established in "Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression" (Chen et al., 2015). The singular values of the operator $T$ satisfy either mild ($\mu_j \asymp j^{-\alpha}$) or severe ($\mu_j \asymp \exp(-j^\alpha)$) decay, defining the degree of ill-posedness.

Norm	Ill-posedness	Minimax Lower Bound	Series-2SLS Upper Bound	Rate Expression
Sup-norm	Mild	$c r_n$	$O_P(r_n)$	$(n/\ln n)^{-s/(2(s+\alpha)+d)}$
Sup-norm	Severe	$c q_n$	$O_P(q_n)$	$(\ln n)^{-s/\alpha}$
$L^2$-norm	Mild	$c n^{-s/(2(s+\alpha)+d)}$	$O_P(n^{-s/(2(s+\alpha)+d)})$	$n^{-s/(2(s+\alpha)+d)}$
$L^2$-norm	Severe	$c (\ln n)^{-s/\alpha}$	$O_P((\ln n)^{-s/\alpha})$	$(\ln n)^{-s/\alpha}$

Key features:

• In the severely ill-posed case, sup-norm and $L^2$-norm rates coincide exactly.
• In the mildly ill-posed case, the sup-norm rate is slower by only a logarithmic factor relative to $L^2$.
• The same series-2SLS (spline or wavelet) estimator achieves minimax-optimality in all four cases.
• This classification directs the practitioner to select the proper series dimension $J$ according to the identified ill-posedness, optimizing rate performance.

This tetrachotomy unifies inference for $h_0$, its derivatives, and nonlinear functionals, enabling uniform Gaussian approximations and valid uniform confidence bands even under strong endogeneity.

5. Tetrachotomy in Parameter Estimation for Gaussian Processes

A further realization appears in the theory of parameter estimation for stationary Gaussian processes, notably in the discrete-time drift estimation for fractional Ornstein–Uhlenbeck (fOU) models, as analyzed in "Optimal rates for parameter estimation of stationary Gaussian processes" (Es-Sebaiy et al., 2016). The Hurst parameter $H$ (memory exponent) induces four sharp regimes:

Regime	Hurst Exponent $H$	Normalization $r_n$	Limiting Law	Rate
Short Memory	$0 < H < 5/8$	$\sqrt{n}$	CLT, $N(0,V_{\text{short}})$	$1/\sqrt{n}$ (Berry–Esséen)
Intermediate	$5/8 < H < 3/4$	$\sqrt{n}$	CLT, $N(0,V_{\text{int}})$	$n^{(4H-3)/2}$ (slower)
Critical	$H = 3/4$	$\sqrt{n}/\log n$	CLT, $N(0,V_{\text{crit}})$	$1/\log n$
Long Memory	$3/4 < H < 1$	$n^{2-2H}$	Non-Gaussian (Rosenblatt distribution)	$n^{3-4H}$

Critical thresholds $H_1=5/8$ and $H_2=3/4$ separate these phases:

• For $H<5/8$, classical Berry–Esséen CLT rates are attainable.
• For $5/8
• At $H=3/4$, even slower ($\sqrt{n}/\log n$) convergence occurs due to log-divergence of variance and cumulant.
• For $H>3/4$, the limit law ceases to be Gaussian (Rosenblatt limit), and normalization shifts to $n^{2-2H}$.

This tetrachotomy is derived from precise Wiener chaos expansions, cumulant asymptotics, and sharp lower/upper bounds, yielding an exhaustive classification for drift estimation optimality in these models.

6. Tetrachotomy in Phylogenetic Reconstruction Optimal Rate

The tetrachotomy also arises in optimal substitution-rate selection when distinguishing near-polytomies in molecular phylogenies, as rigorously detailed in "The optimal rate for resolving a near-polytomy in a phylogeny" (Steel et al., 2016). Considering the optimal rate $\lambda^*$ that maximizes Kullback–Leibler or Euclidean separation between resolved and unresolved trees as the interior edge length $\epsilon \to 0$, there are exactly four regimes:

Model/Metric	Pendant Lengths	$\lambda^*(\epsilon\to 0)$ Limit
Two-state / KL	All equal	$0$
Two-state / KL	Unequal (strong imbalance)	$c>0$, $c$ solves $$\frac{d}{d\lambda}[\frac{\lambda^2}{e^{4\lambda L_1}-1}]=0$$
Infinite-state / KL	All equal	$1/(4L)$
Two-state / Euclid	All equal	$1/(4L)$

Interpretation aligns each case:

• Balanced, two-state: signal is preserved best by slowing substitutions, forcing optimal rate to zero.
• Unbalanced: discrimination becomes dominated by short-edge leaves, yielding nonzero optimal rate.
• Infinite alleles or Euclidean metric: avoids homoplasy, causing nonzero limiting optimal rate regardless of edge length.

Biologically, these regimes reflect the trade-off between accumulating substitutions on the critical edge and preserving variability elsewhere to maximize signal.

7. Theoretical and Practical Implications

The tetrachotomy paradigm offers a unified, exhaustive framework for optimal rate analysis across diverse statistical domains. It provides:

• Necessary and sufficient complexity or structural criteria for each regime.
• Sharp matching minimax lower and attainable upper bounds.
• Algorithmic guidance for estimator design (e.g., sieve dimension choices, learning rule selection).
• Insights into empirical behavior where "slow learning" or impossibility prevails due to intrinsic structural or distributional complexity.

This framework informs statistical methodology, identifies phase transitions in learnability and efficiency, and clarifies the universal limitations imposed by ill-posedness, combinatorial structure, or process memory. The emergence of intermediate regimes—such as log-linear or super-root—demonstrates the fine granularity required for modern statistical theory, extending classical dichotomies or trichotomies and offering a comprehensive taxonomy for understanding optimal rates of convergence.