Practical Learnability

• Practical learnability is a framework defining when learning is feasible by considering finite data, computational constraints, and operational criteria.
• It shifts emphasis from theoretical asymptotic guarantees to instance-level diagnostics, curriculum design, and finite-sample behavior in real-world settings.
• The concept integrates thresholds, phase transitions, and specialized regimes such as privacy and defendability to yield actionable, performance-based insights.

Practical learnability denotes a family of operational notions of when learning is possible, useful, or diagnostically predictable under finite data, finite compute, fixed data sources, architecture constraints, or evolving training states, rather than only under worst-case asymptotic criteria for an abstract hypothesis class. In contemporary work, the term spans finite-instance solvability in applications, distribution-dependent learning curves, sample- and question-level usefulness of training examples, phase transitions induced by objective hyperparameters, and constrained variants such as private learnability or runtime defendability (Sapir, 2018, Bousquet et al., 2020, Wang et al., 2015, Foster et al., 17 Feb 2025).

1. Conceptual scope and major formulations

A central distinction in the literature is between classical learnability and operational learnability. In agnostic PAC learning, a class $H$ is learnable if a learner returns $h\in H$ such that

$L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$

with probability at least $1-\delta$, once the sample size exceeds $m_H(\epsilon,\delta)$. By contrast, “Applied learning” asks for a solution to one fixed finite-data instance $(\epsilon,S)_D$: given a particular sample $S_D$, find $h$ with $L_D(h)\le \epsilon$ (Sapir, 2018).

Formulation	Object of learnability	Operational criterion
Agnostic PAC	Class $H$ over every distribution $h\in H$0	Excess risk $h\in H$1 with confidence $h\in H$2 after $h\in H$3 samples (Sapir, 2018)
Applied learning	One fixed instance $h\in H$4	Absolute error $h\in H$5 on a concrete task instance (Sapir, 2018)
Universal learning	One fixed realizable distribution $h\in H$6	$h\in H$7 for some $h\in H$8 depending on $h\in H$9 (Bousquet et al., 2020)
Sample-wise / frontier notions	Individual samples or questions	High temporal correct-label confidence dynamics or intermediate success probability $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$0 (Lee et al., 2019, Foster et al., 17 Feb 2025)

This plurality is not merely terminological. One line of work proves that there is no universal decision procedure that takes an arbitrary computable hypothesis class and always determines whether it is PAC learnable, uniformly online learnable, universally online learnable, or finitely teachable. Learnability assessment is therefore not fully automatable in general, even before computational efficiency is considered (Caro, 2021). A plausible implication is that practical learnability is inherently family-specific: it is usually analyzed through structural assumptions, restricted model classes, or task-specific diagnostics rather than by a single universal criterion.

2. Finite-data and distribution-dependent perspectives

In the finite-data literature, the key claim is that practical learning is not well modeled by asymptotic access to arbitrarily many i.i.d. samples. Applied learning is explicitly instance-specific, and PAC theory helps only under strong conditions: a known finite-VC hypothesis class $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$1, sample size larger than both uniform-convergence and PAC thresholds, and an empirical-risk regime that is decisively small or decisively large. In that regime, if $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$2, then $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$3 is a solution with probability at least $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$4; if $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$5, then there is no solution within $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$6 (Sapir, 2018). The same paper argues that the sample sizes implied by PAC/VC theory can be in the hundreds of thousands or millions even for modest VC dimensions, which it presents as a mismatch with many applied datasets.

A different finite-source formalization appears in universal learning. Here the question is not whether one bound works uniformly for all distributions, but whether a single learner succeeds for every fixed realizable distribution $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$7, with distribution-dependent constants. Formally, a class is learnable at rate $L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$8 if

$L_D(h)\le \min_{h'\in H}L_D(h')+\epsilon$9

for every realizable $1-\delta$0, for some $1-\delta$1 depending on $1-\delta$2. The theory yields a trichotomy: every concept class has exactly one optimal universal rate—$1-\delta$3, $1-\delta$4, or arbitrarily slow—characterized by the absence or presence of infinite Littlestone and VCL trees (Bousquet et al., 2020). This relocates practical learnability from worst-case envelopes to the learning curve of a fixed source.

A more explicitly finite-sample formulation appears in the “Practical learning paradigm,” which treats learning as selecting a hypothesis that minimizes total inconsistency

$1-\delta$5

over the combined structure $1-\delta$6, under an implicit smoothness assumption that similar inputs tend to have similar feedback. In that formulation, local smoothing, $1-\delta$7-NN, decision trees, Naive Bayes, SVM classification, and SVR are all described as minimizing different inconsistency functionals on small samples rather than relying on asymptotic guarantees (Sapir, 3 Jan 2025). This suggests a finite-data synthesis in which practical learnability is less about eventual convergence than about whether a model class and an inconsistency criterion encode the right local regularity assumptions.

3. Instance-level and curriculum-based operationalizations

A prominent operational turn in recent work is to define learnability at the level of individual examples. In deep classification, sample-wise learnability is defined for a sample $1-\delta$8 by

$1-\delta$9

the average probability assigned to the correct label across training. The associated rank

$m_H(\epsilon,\delta)$0

orders samples from easiest to hardest. On CIFAR-10, these scores are strongly correlated across ResNet-20, VGG-16, and MobileNet, with reported pairwise raw-score correlations $m_H(\epsilon,\delta)$1, $m_H(\epsilon,\delta)$2, and $m_H(\epsilon,\delta)$3, and even stronger rank correlations $m_H(\epsilon,\delta)$4, $m_H(\epsilon,\delta)$5, and $m_H(\epsilon,\delta)$6 (Lee et al., 2019). The practical implication is that “difficulty” can be treated as a data property, not only a model property, and can support curriculum design or data auditing.

In reinforcement learning for LLM reasoning, practical learnability is made dynamic and policy-dependent. A question $m_H(\epsilon,\delta)$7 is practically learnable at the current training point when repeated rollouts yield mixed success and failure within the same step. If the rollout success rate is $m_H(\epsilon,\delta)$8, learnability is defined as

$m_H(\epsilon,\delta)$9

which is maximal at $(\epsilon,S)_D$0 and zero at $(\epsilon,S)_D$1 or $(\epsilon,S)_D$2. Questions always solved or always failed yield vanishing true policy-gradient signal in the binary terminal-reward setting; questions on the “frontier of learnability” do not (Foster et al., 17 Feb 2025). The resulting Sampling for Learnability (SFL) curriculum estimates $(\epsilon,S)_D$3 from repeated rollouts, scores questions by $(\epsilon,S)_D$4, and preferentially trains on the highest-variance subset. On MATH and GSM8K, this shifts training away from degenerate easy or hopeless questions and accelerates train and test accuracy under both PPO and VinePPO (Foster et al., 17 Feb 2025). In this sense, practical learnability is not a static property of an example but a moving boundary between already learned, currently learnable, and currently unlearnable instances.

4. Thresholds, phase transitions, and representation-dependent learnability

Another recurring pattern is that practical learnability appears as a threshold phenomenon. In the Information Bottleneck, a dataset $(\epsilon,S)_D$5 is $(\epsilon,S)_D$6-learnable if there exists a representation $(\epsilon,S)_D$7 such that

$(\epsilon,S)_D$8

If $(\epsilon,S)_D$9 is too small, the trivial encoder $S_D$0 is globally optimal, so learning is impossible in practice even though optimization may converge. The paper proves a universal necessary condition $S_D$1, defines a learnability threshold $S_D$2, and derives sufficient conditions based on the “largest confident, typical, and imbalanced subset” of examples, called the conspicuous subset (Wu et al., 2019). Empirically, noisy MNIST exhibits a sharp onset near $S_D$3 at $S_D$4 label noise, while a CIFAR10 experiment with $S_D$5 label noise reported $S_D$6 from the subset estimator and an empirical threshold $S_D$7 (Wu et al., 2019).

Prompt-based adaptation yields another representation-dependent notion. In a PAC-style theory of in-context learning, a pretrained model $S_D$8 remains frozen, while adaptation occurs through a prompt

$S_D$9

The downstream error is measured after conditioning on $h$0, and efficient in-context learnability is proved under a latent-task mixture model of pretraining, together with approximate independence across delimited examples, lower bounds on task priors and token probabilities, and KL separation among latent tasks (Wies et al., 2023). The theory’s central interpretation is that in-context learning is “more about identifying the task than about learning it”: the prompt reweights posterior mass over pretrained latent tasks, rather than creating a new predictor by parameter update (Wies et al., 2023).

In variational quantum learning, practical learnability is predicted without training by the relative fluctuation

$h$1

where $h$2 is the effective dimension from the rank of the quantum Fisher information matrix. The reference fluctuation is $h$3, derived from a standard learnable decoupled sine landscape (Zhang et al., 2024). Reported experiments indicate that $h$4 acts as an empirical phase boundary separating learnable from unlearnable regimes, while unifying insufficient expressibility, barren plateaus, bad local minima, and overparameterization (Zhang et al., 2024). This is a further instance of practical learnability as a thresholded, target-dependent property of a training landscape rather than a purely combinatorial property of a hypothesis class.

5. Constrained and specialized regimes

Under differential privacy, practical learnability is radically reshaped by stability constraints. In Vapnik’s general learning setting $h$5, a problem is privately learnable if and only if there exists a differentially private asymptotic empirical risk minimizer: private learnability is equivalent to the existence of a private universally AERM algorithm, and also equivalent to the existence of a private always-AERM algorithm (Wang et al., 2015). The key mechanism is that $h$6-DP implies $h$7-stability, so privacy supplies the generalization/stability side of the learning problem. This creates a distinctly practical design rule: in the private regime, one can search for private ERM-like procedures rather than treating stability and empirical risk minimization as separate obstacles.

For hypothesis classes of random or statistical objects, learnability depends strongly on whether one works in agnostic or realizable regimes. If a base class $h$8 has finite fat-shattering dimension, then its randomization, distribution, and dual distribution classes remain agnostically PAC learnable; for concept classes the sample complexity becomes

$h$9

and analogous preservation holds for agnostic online learning via sequential fat-shattering dimension (Anderson et al., 1 Apr 2025). But realizable PAC and realizable online learnability need not be preserved under these randomization operations (Anderson et al., 1 Apr 2025). A plausible implication is that practical learnability of statistical summaries is better captured by robust agnostic formulations than by exact realizability.

In reliable communication over unknown discrete memoryless channels, practical learnability is defined operationally: from i.i.d. channel samples, choose a decoding metric $L_D(h)\le \epsilon$0 and a code rate $L_D(h)\le \epsilon$1 such that

$L_D(h)\le \epsilon$2

with probability at least $L_D(h)\le \epsilon$3. A naive plug-in decoder can fail catastrophically because if $L_D(h)\le \epsilon$4 on a truly possible pair, then $L_D(h)\le \epsilon$5. The proposed virtual sample algorithm instead sets

$L_D(h)\le \epsilon$6

and yields a non-asymptotic guarantee that $L_D(h)\le \epsilon$7 is within $L_D(h)\le \epsilon$8 of $L_D(h)\le \epsilon$9 given sufficiently many samples (Liu et al., 2024). Here practical learnability is neither estimation of the channel law nor abstract PAC classification; it is learning enough from finite data to communicate reliably at near-mutual-information rate.

The relation between learning and defense against runtime backdoors provides another constrained notion. In the computationally unbounded setting, defendability against random-trigger backdoors is essentially determined by VC dimension, with optimal confidence

$H$0

and efficient PAC learnability implies efficient defendability in the bounded setting (Christiano et al., 2024). But the converse fails, and under $H$1 and $H$2, polynomial-size Boolean circuits are not efficiently defendable (Christiano et al., 2024). This positions defendability as an intermediate notion: weaker than efficient learning, stronger than what obfuscation permits.

6. Limits, evaluation pitfalls, and synthesis

A persistent lesson across domains is that representability does not guarantee practical learnability. In student–teacher experiments on deep random networks, the target function is exactly representable by a network of the same architecture, yet learnability drops exponentially with depth in the sign-activation theory and sharply with depth in practice even with state-of-the-art training methods (Das et al., 2019). The mechanism is depth-induced correlation decay: deep random compositions become increasingly pseudorandom-looking to feasible learners. This is a direct refutation of any identification of practical learnability with mere membership of the target in the hypothesis class.

Even the empirical measurement of learnability can be misleading if causal structure is ignored. In formal-language experiments induced by probabilistic finite automata, observed frequency-performance curves for subtasks are confounded by the data-generating automaton: conditioning on datasets with a given task frequency changes the posterior over automata. To address this, the paper formulates the evaluation pipeline as a causal graphical model, introduces the binning semiring to intervene on corpus-level property counts, and measures localized learnability using decomposed KL divergences at the state, transition, and symbol level (Snæbjarnarson et al., 8 Jun 2026). A striking result is that correlational and causal learnability curves can differ not only quantitatively but qualitatively, including trend inversion at low occurrence counts (Snæbjarnarson et al., 8 Jun 2026). This suggests that practical learnability is not just about whether a task is learned, but about whether the evaluation protocol correctly isolates the amount of task-specific evidence responsible for learning.

Taken together, these lines of work support a broad but technically coherent interpretation. Practical learnability is the study of when learning is feasible, diagnostically visible, or worth additional optimization budget in concrete regimes: fixed datasets, fixed distributions, prompt-conditioned inference, privacy constraints, sparse-reward RL, optimizer-sensitive loss landscapes, or domain-specific operational objectives. What unifies these regimes is not a single theorem but a common shift in emphasis—from asymptotic existence to finite-resource usefulness, from class-level possibility to instance- and task-level signal, and from abstract consistency to the structure of actual training dynamics and deployment constraints.