Iterated Learning Paradigms

• Iterated learning paradigms are iterative teacher-student frameworks that model cultural transmission with strict data bottlenecks, amplifying inductive biases to yield emergent structure.
• They employ Bayesian and neural mechanisms, balancing expressivity and compressibility to produce systematic, low-complexity mappings in complex AI systems.
• Applications span human language evolution, vision-language integration, and reinforcement learning, achieving measurable improvements in interpretability and generalization.

Iterated learning paradigms formalize and exploit the principle of cultural transmission—successive knowledge transfer with systematic information bottlenecks—to analyze and induce emergent structure in agents and artificial learners. Originally motivated by the study of human language evolution, iterated learning frameworks now pervade LLMs, vision-language architectures, reinforcement learning, and interactive agent protocols. A distinguishing feature is the repeated alternation of generations of "teacher" and "student" agents, typically under strict data constraints, inducing biases toward learnable, systematic, and generalizable structure.

1. Theoretical Foundations of Iterated Learning

Iterated learning frameworks model a potentially infinite chain of learner–teacher pairs in which each learner receives a restricted sample from the previous generation and generalizes to produce data for the next. In the original Bayesian iterated learning model (ILM), each agent has a hypothesis space $\mathcal H$ (e.g., grammars, mappings $M \rightarrow S$), a prior $p_0(h)$, and learns from a training set $D_t$ composed of $b$ samples (the bottleneck) generated by the previous agent using their own hypothesis $h_{t-1}$. The update follows

$$p(h | D_t) \propto p_0(h) \prod_{(x, y) \in D_t} p(y | h, x).$$

The process amplifies inductive biases: if the bottleneck is sufficiently tight, the stationary distribution concentrates on the most "learnable" or "prior-preferred" hypotheses, even if the process is initialized differently (Bullock et al., 2024, Bunyan et al., 2024, Ren et al., 2024). Theories of compressibility–expressivity trade-offs (e.g., Kolmogorov complexity, Information Bottleneck) show that iterated learning pressures can select for structure with low description length and high functional capacity (Ren et al., 2023, Carlsson et al., 2023).

2. Algorithmic Realizations and Formal Structures

Modern instantiations of iterated learning implement the transmission bottleneck and student/teacher alternation in varied algorithmic forms:

• Neural supervision/autoencoding loops: Agents are parameterized as encoder/decoder networks, with next-generation "pupils" trained on the bottlenecked dataset produced by the current "tutor." Semi-supervised variants (mixing supervised and autoencoder losses) replace combinatorially expensive inference procedures (such as obversion) and enable scaling to meaning-signal spaces of size $2^{20}$ and beyond (Bunyan et al., 2024, Bullock et al., 2024).
• Multi-label image classification (MILe): In multi-label iterated learning for vision, each teacher network performs brief updates on the true (single-label) supervision, then produces multi-label predictions serving as binary pseudo-labels for a student trained under a strict budget. Output is binarized (sigmoid activation, thresholding) and only systematic co-occurrences persist across generations, inducing emergent multi-label structure (Rajeswar et al., 2021).
• Contrastive vision-language alignment: Iterated learning in vision-LLMs operates by periodically respawning the language encoder, distilling knowledge from the fixed vision encoder through brief supervised and contrastive interaction phases. This cycle encourages representations that are smoother, more compositional, and easier for successive generations to learn, evaluated as lower Lipschitz constants and improved compositional retrieval (Zheng et al., 2024).
• In-context learning as iterative reasoning: Layer-recurrent architectures pass demonstration representations through multiple rounds of forward attention accumulation ("Deep-Thinking"), storing meta-gradients as internalized convention for inference on new queries—algorithmically mirroring iterated learning processes within transformer self-attention (Yang et al., 2023).

3. Bottlenecks, Information Constraints, and Emergent Structure

The defining mechanism in iterated learning is the generational transmission bottleneck—agents observe only a small, often random, subset of possible data from the teacher. This constraint serves several functions:

• Amplification of prior bias: When the number of examples $N$ is small, the influence of the learner's prior on the stationary distribution of hypotheses is exponentially amplified. In Bayesian ILM, if two hypotheses have prior ratio $r>1$, the posterior ratio after $t$ generations grows as $r^{t/N}$, causing even subtle biases to dominate (Ren et al., 2024).
• Compression/expressivity trade-off: Only patterns that are compactly described and consistently learnable survive repeated transmission; structure that is idiosyncratic to specific data instances is lost. Kolmogorov complexity bounds show that compositional, modular, or low-complexity mappings are preserved, while arbitrary mappings are not (Ren et al., 2023).
• Cultural "ratchet" and threshold phenomena: Rate-limited communication channels (explicitly constrained bits transmitted per generation) induce sharp, nonlinear transitions in the sustainability of knowledge. Crossing a critical channel rate threshold (measured in bits) can shift systems from zero long-term accumulation to robust ratcheting of structure—connecting cognitive models to rate–distortion theory (Prystawski et al., 22 Nov 2025).

4. Applications Across Domains

Iterated learning has been deployed in diverse AI and cognitive domains:

• Vision-language and compositionality: Iterated learning regularization in CLIP-style or neural module network architectures systematically increases compositional generalization beyond what is achievable by standard large-scale pretraining—demonstrating gains across visual reasoning and layout induction benchmarks (Zheng et al., 2024, Vani et al., 2021).
• Interactive language and language drift: Seeded iterated learning, and its supervised variant, constrain emergent partner protocols to remain interpretable and close to natural language, preventing drift into arbitrary codes typical of unregularized multi-agent training. Multitask supervision applied in the teacher phase further stabilizes language grounding (Lu et al., 2020).
• Semantic categorization (e.g., color naming): When paired with in-generation communication/coordination, iterated learning reproduces the efficient trade-off between simplicity and informativeness characteristic of natural semantic systems. IL alone tends to drive systems toward pathologically simple lexica; communication alone leads to complexes exceeding human complexity, but their combination converges to human-like efficiency and category geometry (Carlsson et al., 2023).
• Reinforcement learning representation transfer: "Iterated relearning" alternates RL policy/value function optimization with repeated distillation into freshly re-initialized networks, breaking the legacy "memory" effect of neural policies and improving generalization in challenging benchmark environments (Igl et al., 2020).

5. Extensions: Semi-Supervised, Contact, and Learning Dynamics

Semi-supervised ILM and contact simulations generalize the classical framework in several dimensions:

• Mixing of supervised and unsupervised training: Combining explicit example-pairing (supervised) with autoencoding over the raw meaning space (unsupervised) expands the learnable class of languages and accelerates emergence of expressivity, compositionality, and stability. This configuration aligns more closely with observed aspects of child language acquisition (Bunyan et al., 2024, Bullock et al., 2024).
• Modeling language contact and stability: Iterated learning simulations with admixture of multiple source languages and autoencoding uncover phase transitions in language retention: above a critical exposure threshold, dominant structure is preserved with high probability. Contact scenarios exhibit robust convergence to pre-existing grammars, provided only slight exemplar frequency advantages (Bullock et al., 2024).
• Self-sustaining iterated learning: Classic ILM collapses to the prior unless data sample lengths per generation are increased. Schedules where training set sizes grow (linearly or superlinearly) with time, or more generally, mixing ("hopped" models) over past teachers, ensure self-sustainability in both discrete and continuous hypothesis spaces (Chazelle et al., 2016).

6. Limitations, Generalizations, and Open Problems

Iterated learning is not universally powerful, and its properties depend on the specific data, bottleneck, and agent update constraints:

• Learning restrictions: The power of iterative learning algorithms is sensitive to data presentation (text vs. informant), monotonicity, conservativeness, and "non-U-shaped" behavior. Certain concept classes (e.g., half-spaces in rational space) are learnable only under specific iterative protocols (Khazraei et al., 2020).
• Bias amplification and run-away effects: Small prior biases, if left unchecked, are exponentially magnified. Without explicit selection or intervention, emergent systems can lock onto suboptimal or undesirable equilibria; thus, bottleneck size, prompt engineering, and interaction-phase filters are crucial in practical machine self-evolution (Ren et al., 2024).
• Communication constraints: The sustainability of cumulative structure through iterated learning can hinge on minuscule increases in communication bandwidth. Failure to cross ratchet thresholds condemns incipient structure to extinction, explaining the absence of cumulative culture in species with sub-threshold channel rates (Prystawski et al., 22 Nov 2025).

7. Empirical Outcomes and Benchmarks

Controlled experiments across vision, language, and reinforcement learning empirically validate the key predictions of iterated learning paradigms:

Domain	Key Metric Increased by IL	Typical Relative Gain
Visual Compositionality	SugarCrepe benchmark (CLIP)	+4–6 points (R@1) (Zheng et al., 2024)
VQA Systematicity	SHAPES-SyGeT OOD Accuracy	+10–30 pp (Vani et al., 2021)
Multi-label Classification	ReaL-F1/ImageNet Top-1	+1–5 points (Rajeswar et al., 2021)
Color Naming Efficiency	IB Proximity & gNID (NIL)	Human-like, not matched by IL/C alone (Carlsson et al., 2023)
RL Generalization	Zero-shot Return on ProcGen	+8–12% (Igl et al., 2020)

These paradigms consistently outperform single-generation, distillation-only, or continuous-optimization baselines, especially under limited supervision, noisy labeling, or distributional shift.

---

The iterated learning paradigm provides a general, domain-agnostic framework for constructing and analyzing learning systems in which cultural, architectural, or information-theoretic pressures act over time to amplify inductive biases, create systematic and compositional encodings, and constrain emergent communication. Its efficacy and limitations are tightly governed by the size and nature of the transmission bottleneck, the structure of agent hypotheses, and the fidelity of intergenerational communication, as established across cognitive and machine learning research (Bullock et al., 2024, Bunyan et al., 2024, Ren et al., 2023, Zheng et al., 2024, Rajeswar et al., 2021, Prystawski et al., 22 Nov 2025, Ren et al., 2024, Carlsson et al., 2023, Vani et al., 2021, Igl et al., 2020).