---
title: Generative Recursive Reasoning Models (GRAM)
url: https://www.emergentmind.com/topics/generative-recursive-reasoning-models-gram-6cae3c73-2cbd-4861-9fad-c616d1abd29c
type: topic
---

# Generative Recursive Reasoning Models (GRAM)

Generative Recursive reAsoning Models (GRAM) are a family of probabilistic, multi-trajectory neural architectures for extended computation and structured generation. The GRAM framework introduces stochastic, recursive latent-state refinement as a generative process, supporting both conditional reasoning and unconditional data generation. Recent advances in this domain demonstrate that recursive generative reasoning models outperform deterministic and purely autoregressive baselines on complex tasks that require multiple, alternative solution trajectories, integrated uncertainty, and explicit epistemic rationales [2605.19376][2509.02492][2510.21861].

## 1. Formal Definition and Recursive Process

GRAM implements reasoning as a latent-variable generative process over a recursive trajectory
\[
\tau = (z_0 \to z_1 \to \ldots \to z_{T_{\text{total}}})
\]
with $T_{\text{total}}$ the total number of latent transition steps. The process is factorized as:

- **Initial prior**: $z_0 \sim p(z_0)$, typically a fixed learned constant or a sample from $N(0,I)$.
- **Recursive transitions**: For $t=1\ldots T_{\text{total}}$,
  \[
  z_t \sim p_\theta(z_t \mid z_{t-1}, e_x)
  \]
  where $e_x = f_\text{enc}(x; \theta)$ is a (potentially empty) encoding of input $x$.

A hierarchical structure is adopted, decomposing each $z_t = (h_t, l_t)$ into high-level ($h \in \mathbb{R}^D$) and low-level ($l \in \mathbb{R}^D$) states, updated via a sequence of deterministic low-level steps and a stochastic high-level transition:
- Low-level: $l_{t,k} = f_L(h_{t-1}, l_{t,k-1}, e_x; \theta)$ for $k = 1\ldots K$; set $l_t = l_{t,K}$
- High-level: $u_t = f_H(h_{t-1}, l_t; \theta)$, $\epsilon_t \sim p_\theta(\epsilon_t \mid u_t) = N(\mu_\theta(u_t), \sigma^2_\theta(u_t) I)$, $h_t = u_t + \epsilon_t$

For **conditional reasoning**, the model outputs $p_\theta(y \mid x)$ by marginalizing over $\tau$:
\[
p_\theta(y \mid x) = \int p_\theta(y \mid z_{T_{\text{total}}}, x) \cdot p_\theta(\tau \mid x)\, d\tau
\]
using a decoder $f_\text{dec}(h_{T_{\text{total}}}; \theta)$. For **unconditional generation**, $x$ is empty and $p_\theta(x)$ is decoded directly [2605.19376].

## 2. Inference and Training

Inference uses amortized variational techniques. The variational posterior,
\[
q_\phi(\tau \mid x,y) = p(z_0) \prod_{t=1}^{T_{\text{total}}} q_\phi(\epsilon_t \mid u_t, y)
\]
shares structure with the prior but conditions on the target $y$. The evidence lower bound (ELBO) optimized during training is:
\[
\log p_\theta(y \mid x) \geq \mathbb{E}_{q_\phi}\left[ \log p_\theta(y \mid z_{T_{\text{total}}}, x) \right] - \operatorname{KL}(q_\phi(\tau|x,y)\,\|\,p_\theta(\tau|x))
\]
with KL divergence decomposed across time. Deep supervision is applied every $T$ steps, and gradients are truncated across supervision boundaries for computational efficiency.

Additional architectural components include:
- **Adaptive computation time (ACT)**: a temporal-difference loss for learned halting.
- **Latent Process Reward Model (LPRM)**: a value head $v_\psi(z_t)$, predicting task-specific rewards via MSE.

Inference-time scaling is achieved by increasing recursion depth ($T_{\text{total}}$), using ACT halting, or sampling multiple stochastic trajectories in parallel, with outputs selected by majority vote or LPRM value [2605.19376].

## 3. Experimental Results and Empirical Performance

GRAM demonstrates state-of-the-art results on structured reasoning and constraint satisfaction tasks:
- **Sudoku-Extreme**: GRAM achieves 97.0% accuracy, vs. 87.4% for TRM and 55.0% for HRM.
- **ARC-AGI-1**: 52.0% (GRAM) vs. 44.6% (TRM).
- **N-Queens (8×8)**: GRAM 99.7% (single-sample), 90.3% coverage (20 samples), outperforming AR Transformer, MDLM, and TRM.
- **Graph Coloring**: GRAM resolves 8-node graphs with 2.7 conflicts, outperforming AR’s 19.0 conflicts; 51.3% coverage on 10-node graphs.

In unconditional generation:
- **Sudoku puzzle generation**: GRAM (10.9 M params, 16 steps) produces 99.05% valid, near-unique puzzles—significantly exceeding D3PM-large at 91.33%.
- **Binarized MNIST**: GRAM achieves an Inception Score of 1.99 (vs. VAE’s 1.70) and FID of 74.30.

Distinctive findings include monotonic improvement in reasoning/generation quality with increasing recursion depth and parallel sample count, and ablation showing that learned stochastic guidance is necessary (naive noise injection fails) [2605.19376].

## 4. Connections to Recursive Generative Reasoning and the Mirror Loop

Recursive generative reasoning defines a discrete dynamical system over state-sequence trajectories:
\[
S_{n+1} = f(S_n, E_n)
\]
with $E_n$ representing external evidence. The *ungrounded mirror loop* corresponds to $E_n = \varnothing$, where reflection occurs solely via internal reformulation. Experimental analysis reveals a rapid decline of mean informational change (normalized edit distance $\Delta I$) during recursive steps, converging to an “attractor of epistemic stasis” with $\lim_{n\to\infty}\Delta I_n = 0$. This regime is marked by:
- Decline in metrics such as $\Delta I$, embedding drift, n-gram novelty, and character-level entropy.
- Empirical saturation across varied LLMs (GPT-4o-mini: $-58.6\%$ reduction in $\Delta I$; Claude 3 Haiku: $-84.3\%$; Gemini 2.0 Flash: $-36.8\%$).
- Only minimal grounding (e.g., an external verification at step 3) produces a $+28\%$ rebound in $\Delta I$ and restores sustained variance [2510.21861].

*This suggests* that unguided recursion induces a regime where generative reasoning becomes performative rather than epistemically informative, with progress reliant on deliberate injection of external evidence.

## 5. Design Principles, Failure Modes, and Interventions

Design recommendations for effective GRAM systems, derived from empirical and theoretical analysis, include:
- **Loop Detection**: Monitor metrics such as embedding drift and n-gram novelty over a rolling window, flagging when both drop below calibrated thresholds ($\sim 0.05$).
- **Mandatory Grounding**: Incorporate external verification (retrieval, code execution, oracle check) at least every $n$ iterations ($n=3$ is effective empirically).
- **State Forking**: Upon loop detection, spawn multiple continuations using varied sampling or prompt augmentation, selecting the most divergent.
- **Meta-Loss Penalties**: Apply explicit penalties in training (or RLHF) to consecutive hidden states with high cosine similarity, discouraging fixed-point convergence.
- **Dissipative Inference Architectures**: Structure the reasoning pipeline so every reflective pass is coupled with external informational influx, ensuring ongoing epistemic acquisition rather than recursive closure.

These principles are essential to convert syntactic recursion into genuine epistemic reflection and prevent the collapse into the mirror loop state [2510.21861].

## 6. GRAM as Foundation and Reward Models

The GRAM paradigm extends to reward modeling, as illustrated by GRAM-R$^2$, a generative foundation reward model. GRAM-R$^2$ produces both preference labels and explicit, natural-language reward rationales. The training regime leverages a recursive self-training loop:
- A “preference-proving” module synthesizes rationales for labeled pairs.
- GRAM-R$^2$ is then iteratively self-trained on unlabeled data by generating pseudo-labels and rationales, filtering low-confidence or generic outputs using a Bayesian-justified scoring mechanism.

Empirical results on RM-Bench, JudgeBench, and coding/math response ranking show that GRAM-R$^2$ achieves 85.7% accuracy (LLaMA-3.1-8B backbone), outperforming discriminative (76.0%) and generative no-rationale (79.2%) baselines. Downstream, GRAM-R$^2$ accelerates RLHF adaptation and achieves higher win rates in human feedback–driven tuning [2509.02492].

A plausible implication is that by folding explicit reasoning and rationale generation directly into the reward modeling process, GRAMs can serve as generalist reward models requiring significantly less human-labeled preference data.

## 7. Limitations and Directions for Future Research

Current GRAM training requires deep supervision with truncated BPTT, which is less efficient (both compute and memory) than standard feedforward Transformer architectures. Scaling to very large backbones and to open-domain or multi-turn reasoning remains challenging, with open questions on variational posterior calibration and reward model selection. The recursive self-training procedure for reward models, while empirically effective, lacks formal convergence analysis.

Potential future avenues include:
- Efficient scalable approximations for training and inference.
- Theorizing the properties and convergence behavior of recursive self-training procedures.
- Integration of richer reasoning structures, e.g., multi-step RL objectives and contrastive filtering.
- Extending GRAMs to domains such as multi-turn dialogue, real-world planning, and unsupervised structured data generation [2605.19376][2509.02492][2510.21861].

Source: https://www.emergentmind.com/topics/generative-recursive-reasoning-models-gram-6cae3c73-2cbd-4861-9fad-c616d1abd29c