Entropic Learnability Horizon (ELH)

• Entropic Learnability Horizon (ELH) defines an entropy-based threshold that demarcates the boundary between learnable and unlearnable regimes.
• ELH spans diverse frameworks—acquisition screening, metric entropy, thermodynamic transport, and recurrent architectures—each quantifying resource limits differently.
• Empirical results across imaging, massive MIMO, and deep networks show that optimizing entropy-related parameters can extend effective learnability.

Searching arXiv for the cited ELH-related papers to ground the synthesis. arXiv Search Tool Result: query="Entropic Learnability Horizon"

• Found: "Informational Frustration in Neural Manifolds: Shannon Bottlenecks and the Limits of Learnability" (P. et al., 29 Jun 2026)
• Found: "A Thermodynamic Theory of Learning I: Irreversible Ensemble Transport and Epistemic Costs" (Okanohara, 24 Jan 2026)
• Found: "A Thermodynamic Theory of Learning Part II: Critical Period Closure and Continual Learning Failure" (Okanohara, 8 Feb 2026)
• Found: "Phase-space entropy at acquisition reflects downstream learnability" (Wang et al., 22 Dec 2025)
• Found: "Learnability Window in Gated Recurrent Neural Networks" (Livi, 5 Dec 2025)
• Found: "The Asymptotic Cost of Complexity" (Cripps, 2024)
• Found: "An Entropy-Based Model for Hierarchical Learning" (Asadi, 2022)
• Found: "Theoretical bound of the efficiency of learning" (Su et al., 2022)
• Found: "Bounds and guarantees for learning and entanglement" (Peters, 2024) Entropic Learnability Horizon (ELH) is a family of entropy-based threshold concepts used to demarcate the boundary between learnable and unlearnable regimes. In the cited literature, the term is attached to several formally distinct objects: a threshold on acquisition-induced phase-space entropy change, a metric-entropy sample-complexity law, a Wasserstein reachable set under entropy-production and time budgets, a largest statistically certifiable hierarchy depth, a temporal lag window in gated recurrence, and an inequality balancing data, weight, and boundary entropies. The common motif is that learnability is not treated as a purely algorithmic property, but as a constrained regime determined by entropy, complexity growth, transport cost, or information concentration (Wang et al., 22 Dec 2025, Cripps, 2024, Okanohara, 24 Jan 2026, P. et al., 29 Jun 2026).

1. Terminological scope and formal variants

Across the cited works, ELH does not denote a single canonical scalar. Instead, it denotes a boundary object whose exact form depends on the state space, the entropy notion, and the operational question: acquisition screening before training, minimax estimation under growing hypothesis complexity, finite-time transport in parameter-distribution space, hierarchical scale selection, temporal dependency recovery, or decision-boundary feasibility.

Framework	Entropic object	Horizon form
Acquisition auditing	$|ΔS_{\mathcal B}|$	threshold $\tau_{\mathrm{ELH}}$
Minimax nonparametrics	$H_K(\varepsilon)=\log N_K(\varepsilon)$	$$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$
Irreversible ensemble transport	entropy production, $W_2$ geometry	$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$
Hierarchical learning	multiscale prefix entropies	largest scale $K^*$ with guaranteed risk
Gated RNNs	effective-rate envelope under $\alpha$-stable noise	$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$
Neural manifold bottlenecks	$H_S(D)$, $\tau_{\mathrm{ELH}}$0, $\tau_{\mathrm{ELH}}$1	$\tau_{\mathrm{ELH}}$2

In some cases the term is explicit, and in others it is defined within an existing framework rather than introduced by the paper itself. The acquisition paper states that it does not use the term ELH explicitly, then defines ELH as a threshold on $\tau_{\mathrm{ELH}}$3 within its phase-space entropy formalism; the first thermodynamic paper similarly derives the Epistemic Speed Limit and then defines ELH as the reachable set implied by that bound (Wang et al., 22 Dec 2025, Okanohara, 24 Jan 2026).

A common misconception is to treat ELH as a universal invariant. In the cited literature, the horizon may mean a performance threshold, a sample-complexity law, a geometric reachability radius, a task-compatibility limit, or a boundary-complexity inequality. The unifying content is therefore structural rather than notational: each ELH specifies where an entropy-controlled resource balance ceases to support the target learning behavior.

2. Acquisition-space ELH

One major formulation places the horizon at acquisition rather than training. In this framework, an instrument-resolved phase-space density is built by smoothing the Wigner distribution with a nonnegative kernel $\tau_{\mathrm{ELH}}$4,

$\tau_{\mathrm{ELH}}$5

then band-normalizing over a Nyquist band $\tau_{\mathrm{ELH}}$6,

$\tau_{\mathrm{ELH}}$7

and defining local and global band entropy,

$\tau_{\mathrm{ELH}}$8

For an acquisition operator $\tau_{\mathrm{ELH}}$9, the acquisition-induced entropy change is

$H_K(\varepsilon)=\log N_K(\varepsilon)$0

Small $H_K(\varepsilon)=\log N_K(\varepsilon)$1 indicates preservation of joint space-frequency structure at instrument scale, whereas large $H_K(\varepsilon)=\log N_K(\varepsilon)$2 indicates mixing or removal of that structure (Wang et al., 22 Dec 2025).

Within this framework, ELH is defined as the smallest threshold on $H_K(\varepsilon)=\log N_K(\varepsilon)$3 beyond which downstream performance falls below a desired tolerance or where the empirical performance curve develops a knee:

$H_K(\varepsilon)=\log N_K(\varepsilon)$4

or, in knee-based form,

$H_K(\varepsilon)=\log N_K(\varepsilon)$5

The paper does not provide a closed-form $H_K(\varepsilon)=\log N_K(\varepsilon)$6, but it reports a monotone relationship across modalities: performance degrades as $H_K(\varepsilon)=\log N_K(\varepsilon)$7 grows. This makes ELH an acquisition-side regime boundary rather than a model-side capacity measure (Wang et al., 22 Dec 2025).

The theoretical mechanism is modality-dependent but phase-space coherent. For periodic spatial subsampling, band-normalized spectra become convex mixtures of shifted copies of the original spectrum, yielding $H_K(\varepsilon)=\log N_K(\varepsilon)$8 and therefore $H_K(\varepsilon)=\log N_K(\varepsilon)$9 except in degenerate cases. Random masking breaks coherence and gives $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$0 and $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$1 under mild lower bounds on local band energy. For MRI, where frequency coefficients are removed rather than folded, accessible $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$2 becomes more concentrated, so $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$3 and $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$4 becomes the modality-agnostic difficulty measure (Wang et al., 22 Dec 2025).

The empirical program spans masked image classification, accelerated MRI, and massive MIMO. In masked mini-ImageNet classification, random masks outperform periodic masks at all budgets: $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$5 vs $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$6 for $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$7, $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$8 vs $$T_{\mathrm{ELH}(\varepsilon)} \asymp H_K(\varepsilon)/\varepsilon^2$$9 for $W_2$0, and $W_2$1 vs $W_2$2 for $W_2$3. In accelerated MRI, Poisson variable-density masks yield the best PSNR/SSIM and the smallest $W_2$4, and minimizing $W_2$5 over $W_2$6 selects variable-density masks whose downstream PSNR/SSIM match designs optimized by k-space $W_2$7 or zero-filled PSNR. In massive MIMO, NMSE increases monotonically with $W_2$8, and periodic deactivation produces larger positive $W_2$9 and worse NMSE than random deactivation, including in over-the-air measurements (Wang et al., 22 Dec 2025).

3. Metric-entropy and minimax ELH

A second formulation treats ELH as the sample-complexity boundary induced by metric entropy growth of the hypothesis space. In this setting, the unknown environment is indexed by $$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$0, observations are i.i.d. samples from $$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$1, loss is squared Hellinger distance, and learnability is quantified by minimax risk

$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$2

The central complexity quantity is the covering entropy

$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$3

where $$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$4 is the $$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$5-covering number under the KL-induced semi-metric $$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$6 (Cripps, 2024).

The upper and lower Yang–Barron-type bounds exhibit the governing trade-off:

$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$7

and

$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$8

ELH is then formalized as the balance point between accuracy cost and complexity cost,

$$W_2(q_0,q_{\mathcal T}) \le \sqrt{\Sigma_{\max}\mathcal T}$$9

or, equivalently, by the implicit accuracy law

$K^*$0

This is a direct minimax feasibility threshold: above the horizon the target resolution is achievable up to constants; below it, the lower bound forbids that resolution (Cripps, 2024).

When metric entropy obeys a power law, $K^*$1, the horizon becomes

$K^*$2

with risk scaling

$K^*$3

For Hölder-smooth density classes with smoothness index $K^*$4, the entropy law $K^*$5 yields

$K^*$6

Rougher classes therefore have longer horizons because their metric entropy grows faster at fine resolution (Cripps, 2024).

The same paper instantiates this horizon in two economic learning models. In the manager–worker allocation model, the payoff gap decays at the same metric-entropy-controlled rate as the minimax risk. In the exploration model, the horizon is longer because maximizing over a small set requires sup-norm accuracy rather than Hellinger accuracy. A plausible implication is that ELH here functions as a resolution-dependent complexity barrier: the learner must pay not merely for estimation noise, but for the exponential growth in distinguishable alternatives at the target scale.

4. Thermodynamic and transport ELH

A third family of formulations models learning as finite-time irreversible transport of distributions over parameter space. Let $K^*$7, let the learning state at time $K^*$8 be a distribution $K^*$9 with density $\alpha$0, and let the ensemble evolution satisfy a continuity equation

$\alpha$1

With epistemic free energy

$\alpha$2

and entropy production

$\alpha$3

the Epistemic Speed Limit gives

$\alpha$4

ELH is then the feasible set of terminal ensembles reachable within entropy-production budget $\alpha$5 and time budget $\alpha$6,

$\alpha$7

Equivalently, for transport radius $\alpha$8, the minimal irreversible cost is

$\alpha$9

The horizon is thus a Wasserstein ball whose radius expands with available time and entropy-production budget (Okanohara, 24 Jan 2026).

In Gaussian ensembles this becomes fully explicit. For $$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$0 and $$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$1,

$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$2

hence

$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$3

Mean shift and covariance contraction then have direct cost-speed trade-offs, and isotropic contraction scales linearly with ambient dimension $$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$4 (Okanohara, 24 Jan 2026).

The continual-learning extension replaces endpoint reachability by residual reconfiguration capacity. Learning phases compose as transport maps, with Jacobians obeying

$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$5

and rank submultiplicativity implies contraction of dynamically accessible directions. The remaining adaptable capacity within the task-preserving manifold of task $$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$6 is captured by the compatible effective rank

$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$7

while the new task $$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$8 imposes curvature demand through the stable rank

$$\mathcal H_N=\sup\{\ell:f(\ell)\ge\varepsilon_{\mathrm{th}(\ell)}\}$$9

The capacity-threshold theorem states: if $H_S(D)$0, then no learning trajectory that remains within the task-preserving manifold of task $H_S(D)$1 can accommodate task $H_S(D)$2; any sufficient adaptation necessarily incurs forgetting. ELH is therefore the earliest time or phase at which $H_S(D)$3 falls below the future task requirement,

$H_S(D)$4

or

$H_S(D)$5

This makes critical period closure a trajectory-level compatibility threshold rather than a statement about the nonexistence of multitask solutions (Okanohara, 8 Feb 2026).

5. Architectural horizons in hierarchical and recurrent models

In hierarchical learning, ELH is tied to multiscale decomposition of both the input domain and the target function. The domain is partitioned into scale bands $H_S(D)$6, the target is expressed through a ladder decomposition

$H_S(D)$7

and the learner is built level-by-level via

$H_S(D)$8

Training is regularized by a telescoping mixture of prefix entropies,

$H_S(D)$9

or conditionally $\tau_{\mathrm{ELH}}$00, and the Gibbs factorization over levels minimizes a smoothed loss minus this multiscale entropy regularizer. The resulting chained-risk bound leads to a population-risk control term

$\tau_{\mathrm{ELH}}$01

together with

$\tau_{\mathrm{ELH}}$02

ELH is defined as the largest scale $\tau_{\mathrm{ELH}}$03 such that

$\tau_{\mathrm{ELH}}$04

while for any $\tau_{\mathrm{ELH}}$05 the guarantee fails. The corresponding sample requirement to learn up to level $\tau_{\mathrm{ELH}}$06 is

$\tau_{\mathrm{ELH}}$07

Because readout depth depends on input scale, this horizon is also computational: if training stops at stage $\tau_{\mathrm{ELH}}$08, the model remains valid for $\tau_{\mathrm{ELH}}$09 (Asadi, 2022).

In gated recurrent neural networks, the horizon is temporal and statistical rather than spatial. The learnability window is defined by the largest lag for which gradient information remains statistically recoverable:

$\tau_{\mathrm{ELH}}$10

or equivalently

$\tau_{\mathrm{ELH}}$11

Here $\tau_{\mathrm{ELH}}$12 is the effective learning-rate envelope induced by gate-controlled Jacobian products, and under symmetric $\tau_{\mathrm{ELH}}$13-stable gradient noise the per-lag sample size satisfies

$\tau_{\mathrm{ELH}}$14

so that

$\tau_{\mathrm{ELH}}$15

The paper interprets ELH as the information-theoretic counterpart of this learnability window:

$\tau_{\mathrm{ELH}}$16

with $\tau_{\mathrm{ELH}}$17 under the same detectability criterion. Closed-form scaling follows from the decay of $\tau_{\mathrm{ELH}}$18: logarithmic in $\tau_{\mathrm{ELH}}$19 under exponential decay, algebraic under polynomial decay, and stretched-exponential under logarithmic decay of the envelope. The central claim is that stability of Jacobian products is not sufficient; statistical recoverability is governed by effective learning rates and heavy-tail concentration (Livi, 5 Dec 2025).

6. Shannon–topological bottlenecks and neural-manifold ELH

A more speculative but explicitly formalized construction defines ELH for deep networks through three entropies: the differential Shannon entropy of the data manifold,

$\tau_{\mathrm{ELH}}$20

the von Neumann entropy of the steady-state weight distribution,

$\tau_{\mathrm{ELH}}$21

and the topological entropy of the target decision boundary,

$\tau_{\mathrm{ELH}}$22

The learnability horizon is

$\tau_{\mathrm{ELH}}$23

with the Horizon Principle

$\tau_{\mathrm{ELH}}$24

The paper’s Shannon–Topological Bottleneck Theorem states

$\tau_{\mathrm{ELH}}$25

If the boundary complexity exceeds this entropic budget, the system enters Informational Frustration (P. et al., 29 Jun 2026).

The frustrated regime is characterized by the entropic gap

$\tau_{\mathrm{ELH}}$26

with phase change at $\tau_{\mathrm{ELH}}$27 and frustration when $\tau_{\mathrm{ELH}}$28. The paper associates this phase with glassy memorization, low weight entropy, sharp minima, and poor generalization. “Grokking” is reinterpreted as an Entropic Release in which $\tau_{\mathrm{ELH}}$29 surges enough to cross the horizon, after which the network reorganizes toward a lower-complexity boundary (P. et al., 29 Jun 2026).

This framework also proposes Entropic Gradient Descent (EGD),

$\tau_{\mathrm{ELH}}$30

with covariance-tracking updates using

$\tau_{\mathrm{ELH}}$31

The formalism is ambitious in scope and the paper emphasizes practical estimation issues: $\tau_{\mathrm{ELH}}$32 is nonstandard and difficult to estimate, while $\tau_{\mathrm{ELH}}$33 is operationalized through a Gaussian covariance approximation that may fail in strongly non-Gaussian regimes (P. et al., 29 Jun 2026).

7. Related bounds, interpretation, and open issues

Two additional lines of work place ELH-like limits directly in information-theoretic and thermodynamic terms. In stochastic thermodynamic learning, the mutual-information learning rate of subsystem $\tau_{\mathrm{ELH}}$34 is bounded by environment entropy flow and a kinetic coefficient:

$\tau_{\mathrm{ELH}}$35

This yields an instantaneous ELH on learnability per unit time and a cumulative horizon

$\tau_{\mathrm{ELH}}$36

The framework is developed for bipartite Markovian systems such as double quantum dots and biochemical cell networks, where $\tau_{\mathrm{ELH}}$37 is computable from transition rates and stationary probabilities (Su et al., 2022).

In classical and quantum information theory, ELH appears as a pair of conditional-entropy thresholds after discretization. For classical learning at resolution $\tau_{\mathrm{ELH}}$38, guaranteed success is obtained if

$\tau_{\mathrm{ELH}}$39

while guaranteed failure follows if

$\tau_{\mathrm{ELH}}$40

The quantum extension replaces classical success probability by singlet fraction or entanglement fraction, with sufficiency controlled by conditional entropy and necessity by a Quantum Fano-type bound (Peters, 2024).

Taken together, these formulations suggest that ELH is best understood as a general name for entropy-governed regime boundaries rather than as a settled invariant. The entropy being measured may be phase-space Shannon entropy, metric entropy, conditional entropy, thermodynamic entropy production, or von Neumann entropy; the horizon variable may be accuracy, sample size, Wasserstein displacement, task sequence depth, temporal lag, or decision-boundary complexity. A plausible implication is that the term is most coherent when read relationally: each ELH identifies a point where an entropy budget ceases to support a target form of learning.

Open issues are correspondingly framework-specific. Acquisition ELH depends on a reference $\tau_{\mathrm{ELH}}$41, choice of instrument scale, and an empirically fitted $\tau_{\mathrm{ELH}}$42 rather than a universal analytic map (Wang et al., 22 Dec 2025). Metric-entropy ELH carries loose constants and relies on global covering or packing bounds (Cripps, 2024). Transport ELH has a clean endpoint law but Part II does not provide an explicit quantitative map from entropy production to compatible effective-rank loss (Okanohara, 8 Feb 2026). Neural-manifold ELH depends on difficult estimators for $\tau_{\mathrm{ELH}}$43 and on Gaussian approximations for $\tau_{\mathrm{ELH}}$44 (P. et al., 29 Jun 2026). These limitations do not erase the common theme: across the literature, ELH names the point at which entropy, in one formal sense or another, becomes the bottleneck to further learnability.