---
title: Emergent Analogical Reasoning
url: https://www.emergentmind.com/topics/emergent-analogical-reasoning
type: topic
---

# Emergent Analogical Reasoning

Emergent analogical reasoning refers to the spontaneous development of analogy-making capabilities in large neural models, particularly large language models (LLMs), without explicit supervision or rule-based programming for analogical inference. In these systems, analogical reasoning is defined operationally by the model's ability to discern, abstract, and apply relational structure across disparate domains, exhibiting patterns of performance and error qualitatively similar to humans, even though the models have not been directly endowed with such high-level reasoning mechanisms [2404.13070][2212.09196][2511.20344][2602.01992]. This phenomenon has become a central focus in computational cognitive science, theoretical machine learning, and neuro-symbolic AI, due to its implications for the emergence of abstraction and transfer in artificial intelligence.

## 1. Formal Definitions and Core Criteria

Analogical reasoning is classically formalized as finding a mapping such that the relation $R$ holding between two elements $a : b$ also holds between two others $c : d$, i.e.,

$$
R(a, b) \approx R(c, d)
$$

A system demonstrates emergent analogical reasoning if:

- It was not explicitly trained for analogy-solving,
- Solves analogy tasks in a zero-shot setting (no in-context demonstrations, no task-specific finetuning),
- Shows sharply superlinear scaling of analogy performance with model size,
- Exhibits behavioral signatures (e.g., distractor sensitivity, error patterns) resembling human analogical cognition.

For LLMs of parameter count $n$, analogy accuracy $A(M_n)$ is emergent if $dA/dn \gg 0$ and $A(M_n) \gg$ chance on unseen, held-out problem types [2404.13070].

Category theory formalizes analogical reasoning as the instantiation of a functor $F : \mathcal{C} \to \mathcal{D}$, mapping objects and morphisms (relations) between domain graphs such that relational structure is preserved (i.e., $F(r_2 \circ r_1) = F(r_2) \circ F(r_1)$) [2602.01992].

## 2. Benchmarks and Methodological Approaches

### Principal Task Domains

- **Text-based digit matrices**: Non-visual Raven-style tasks, with model performance exceeding human baselines ($\sim$65% for GPT-3 vs human $\sim$50%), far above random (11%) [2212.09196][2404.13070].
- **Letter-string analogies**: Transformation and generalization tasks on sequences, requiring abstraction of atomic and higher-order rewrite rules; observed model accuracy: GPT-3 near 60%, GPT-4 near 85%, approaching human $\sim$90% [2212.09196][2404.13070].
- **Verbal and story analogies**: Four-term analogies and narrative mappings, probing both first-order (surface) and higher-order (causal/systematic) relational abstraction [2310.00996][2511.20344][2212.09196].

### Counterfactual/Adversarial Variants

To distinguish genuine relational abstraction from data memorization, rigorous counterfactual benchmarks permute base symbol sets (e.g., random alphabet permutation) [2404.13070], alter transformation intervals [2308.16118][2411.14215], or introduce narrative analogies where surface and system mappings are orthogonal [2310.00996]. Such variants probe the system's ability to operate over entirely novel symbol systems, with no overlap in real-world text.

### Robustness and Negative Controls

Robustness is assessed by measuring drops in accuracy ($\Delta$accuracy) when presenting out-of-distribution analogies. Human performance remains invariant, but LLMs show pronounced brittleness—accuracy drops from $\sim$85% to $\sim$45% on permuted/symbol alphabets [2411.14215][2308.16118]. Synthetic and adversarial data construction (e.g., LABC methodology [1902.00120]) ensures that success is only possible via abstract relational mapping, not shortcut heuristics.

## 3. Mechanistic Insights: Inductive and Neural Foundations

### Feature Resemblance and Relational Alignment

Theoretical analysis reveals that transformers acquire analogical reasoning by embedding entities that share properties into geometrically aligned regions ("analogical manifolds") of representation space. Analogical inference then emerges as a property of linear alignment: representations of $a_1$ and $a_2$ (with shared property $B$) become collinear, enabling attribute transfer through a common linear decoding [2603.05143].

Quantitatively, feature similarity $\cos(\mathbf{V} a_i, \mathbf{V} a_i') \rightarrow 1$ after joint or appropriate sequential curriculum training, yielding perfect generalization on analogy tests [2603.05143].

### Emergence as Geometric/Structural Alignment

Mechanistic studies link analogical success to the transformation of entity embedding constellations via self-attention and residual connections. Category-theoretic analogies are realized through "functor" vectors: for entity $e_s$ and functor $f$, the output $h_{F(e_s)} \approx h_{e_s} + h_f$ as a linear operation in hidden space [2602.01992].

Empirical evidence from pretrained LLMs (e.g., Gemma2-2B, Gemma2-9B) validates that, during inference, geometric alignment and attention weighting between source and target tokens are predictive of analogical accuracy. In both synthetic and natural settings, Dirichlet energy over aligned embeddings drops sharply just as analogical competence emerges [2602.01992].

### Probing Hidden-Representation Dynamics

Layer-wise analysis demonstrates that relational information is encoded and propagated through mid-to-upper transformer layers. Relational signals peak in the middle layers, while attributive content declines in later stages [2511.20344]. Patching activations and swapping hidden representations across entities can, in some proportion of failures, recover analogical success, revealing instances where the correct relation $R$ is abstracted but fails to propagate through the generation path [2511.20344].

### Probabilistic and Hyperdimensional Models

Bayesian graph-matching models on semantic relation networks (PAM) [2103.16704] and neuro-symbolic approaches leveraging complex hyperdimensional geometry [2411.08684] offer alternative perspectives where analogical mapping emerges from the structure of dense distributed representations, with analogy reduced to compositional vector operations or probabilistic matching, sidestepping explicit symbolic predicate manipulation.

## 4. Cognitive Comparisons and Robustness Controversy

### Human–Model Comparisons

Human reasoners exhibit uniform robustness to symbol rearrangement and novel analogical domains; LLMs and neural networks trained without adversarial controls suffer pronounced degradation in zero-generalization or symbol-permuted settings [2308.16118][2411.14215]. While model error patterns in familiar settings mirror human relational complexity signatures, failures on synthetic and narrative analogies reveal fundamental gaps in robust abstraction.

### Memorization Critique

Critiques argue that strong model performance on standard alphabets or common analogy forms may reflect memorization rather than true analogical computation [2308.16118]. Counterexamples based on synthetic alphabets or minimal transformation variants cause model accuracy to collapse towards chance, while human accuracy remains unchanged, emphasizing the need for adversarial and synthetic tasks to validate emergent reasoning claims [2411.14215].

### Transfer, Generalization, and Limitations

LLMs show high performance on near analogies (surface and system mapping overlap) but underperform on far analogies (system mapping only, surface mismatch), with accuracy below random guessing in some partitions [2310.00996]. Chain-of-thought and few-shot prompting improve far analogy accuracy, yet do not close the gap to human performance, indicating limited spontaneous generalization.

## 5. Extensions: Large-Scale Knowledge Integration and Multimodal Reasoning

### Symbolic Knowledge Bases

Analogical reasoning is further enhanced by integrating large-scale analogy knowledge bases (e.g., AnalogyKB), which provide diverse, structured relations—both “same-relation” and “analogous-relation” pairs—derived from Wikidata and ConceptNet [2305.05994]. Pretraining or prompt-augmented LMs on such curated data yields significant gains in recognition and generation accuracy (>15 points on average), approaching human-level benchmarks.

### Multimodal Emergence

Recent work demonstrates emergent analogical reasoning in multimodal LLMs (MLLMs), where models perform analogies across image–text pairs. Exploiting unified prompt templates and augmenting models as “explainers” or directly fine-tuning as “predictors,” MLLMs achieve above-chance accuracy on multimodal analogy datasets—even without explicit analogical supervision—demonstrating that analogical abstraction extends beyond language-only settings [2411.01307].

## 6. Implications, Open Problems, and Future Directions

The study of emergent analogical reasoning reveals that large-scale neural models, through appropriately structured data and scaling, instantiate a form of relational abstraction reminiscent of human cognition [2212.09196][1902.00120]. However, current systems manifest fragility: failures trace to inadequate relational encoding, insufficient propagation, or misalignment between latent representations, especially in adversarial or out-of-distribution conditions [2411.14215][2511.20344].

Foundational future challenges include:

- Formalizing finer-grained “relation similarity” metrics and mechanistic probes (e.g., hidden-state geometry, attention subspace alignment) [2404.13070][2603.05143],
- Designing architectural or optimization objectives that encourage persistent relational continuity across compositional operations [2511.20344][2602.01992],
- Expanding evaluation protocols to require not only accuracy but also robustness to counterfactual and adversarial variants,
- Investigating the integration of explicit structure-mapping modules and hybrid neuro-symbolic representations for scalable, general analogical abstraction [2411.08684][2310.00996],
- Extending analyses to richer domains: long-form narrative, temporally and causally intricate analogies, and cross-modal settings,
- Connecting training-time emergence and in-context manifestation of analogical capabilities with detailed mechanistic interpretability in large pretrained models [2603.05143][2511.20344].

This trajectory seeks to advance both the theoretical understanding and practical realization of human-level analogical reasoning in artificial intelligence systems.

Source: https://www.emergentmind.com/topics/emergent-analogical-reasoning