Recursive Inference Machines (RIMs)

• Recursive Inference Machines (RIMs) are learned iterative inference systems that refine estimates recursively using neural networks to solve inverse problems and enable neural reasoning.
• They leverage mechanisms like explicit likelihood gradients and dynamic reweighting to iteratively update estimates, outperforming traditional static optimization methods.
• Empirical studies demonstrate that RIMs improve reconstruction quality, transferability, and computational speed across tasks such as image restoration and quantitative MRI.

Recursive Inference Machines (RIMs) denote a class of learned iterative inference systems in which a neural architecture repeatedly refines internal states and candidate solutions rather than producing a one-shot prediction. In the literature, the term has two closely related usages. In inverse problems, the foundational formulation is Recurrent Inference Machines, introduced as a framework that learns the inference algorithm itself from data while retaining an explicit likelihood gradient from the forward model (Putzky et al., 2017). In neural reasoning, Recursive Inference Machines were later defined as a more explicit Solver–Generator–Reweighter formalism inspired by classical inference engines such as Sequential Monte Carlo (SMC) and Gibbs sampling, with Tiny Recursive Models (TRMs) identified as a special case (Komisarczyk et al., 5 Mar 2026). The shared idea is that inference is represented as a trainable recursive process over estimates, latent states, and correction mechanisms.

1. Terminology, lineage, and scope

The inverse-problem literature introduced Recurrent Inference Machines as an RNN-based learning framework for iterative reconstruction, while the neural-reasoning literature later introduced Recursive Inference Machines as a formal framework for recursive proposal-and-correction dynamics. The 2017 paper explicitly states that the formal term is Recurrent Inference Machines (RIMs) and notes that “recursive” is not the chosen name there, even though the method is recursive in the algorithmic sense because each estimate depends on the previous one (Putzky et al., 2017). By contrast, the 2026 paper formally uses Recursive Inference Machines (RIMs) for neural reasoning systems built around recursive inference mechanisms and reweighting (Komisarczyk et al., 5 Mar 2026).

Usage	Core setting	Characterization
Recurrent Inference Machines	Inverse problems	Learn an iterative inference procedure from data
Recurrent Inference Machines for MR relaxometry	Quantitative MRI	Extend the inverse-problem framework to nonlinear parameter estimation
Recursive Inference Machines	Neural reasoning	Solver–Generator–Reweighter framework with recursive reweighting

This lineage matters because the two usages are conceptually adjacent but not identical. The earlier framework is centered on likelihood-gradient-guided iterative reconstruction, whereas the later framework emphasizes recursive proposal generation together with explicit correction or reweighting. A plausible implication is that the 2026 terminology generalizes the older “learned inference program” viewpoint to reasoning tasks in which posterior approximation over reasoning states is foregrounded.

2. Inverse-problem formulation: learning the inference algorithm

In the original inverse-problem setting, the target is a linear inverse problem

$y = Ax + n,$

where the goal is to reconstruct $x$ from measurements $y$, given a known forward model $A$. Classical approaches often solve a MAP objective

$\max_x \log p(y \mid x) + \log p_\theta(x),$

with iterative gradient-ascent-style updates

$$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$

The RIM formulation replaces this hand-crafted update with a learned recurrent update

$x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$

where $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$ is the likelihood gradient and $g_\phi$ is a neural network that learns how to update the estimate (Putzky et al., 2017).

The central conceptual move is that the model does not learn a prior density explicitly; it learns the effect of a prior through updates. The 2017 paper positions this as a reversal of standard algorithm unrolling. Instead of choosing a prior, choosing an optimization method, unrolling that method, and learning only some parameters, the framework trains an RNN to discover the inference strategy itself. The paper argues that traditional learned iterative inference still requires substantial hand design because the network architecture is directly derived from a hand-chosen inference algorithm, effectively limiting its capabilities (Putzky et al., 2017).

To make the update recurrent in the RNN sense, the method augments the state with a hidden memory variable $s_t$: $x$0

$x$1

Here, $x$2 is the current reconstruction estimate, $x$3 is information from the forward model or likelihood, and $x$4 is the internal memory of the inference process. The paper notes that the hidden state can help the algorithm remember progression across iterations, curvature information, preconditioning-like behavior, and stopping criteria or confidence. This is one of the principal ways the framework goes beyond simple gradient-based learned methods.

The same paper also discusses a differentiable link function $x$5 to enforce constraints: $x$6 In the reported image experiments, $x$7 is chosen as the logistic sigmoid so that image pixel values remain in $x$8 (Putzky et al., 2017). This preserves a separation between unconstrained iterative dynamics and constrained output space.

3. Architectural instantiation, optimization, and empirical behavior in image restoration

The training framework unrolls a fixed number of inference steps and applies loss at every step. For $x$9 steps, the total loss is

$y$0

where $y$1 is the ground-truth signal, $y$2 is the estimate at step $y$3, and $y$4 are positive weights. The paper follows Andrychowicz et al. and sets $y$5, so every intermediate prediction is trained to improve, not just the final one (Putzky et al., 2017). This encourages the recurrent module to behave like an inference algorithm rather than a terminal decoder.

The default architecture reported in the paper consists of convolutional layers, a GRU recurrent core, transposed convolution, and 20 inference steps during training. A typical configuration is: first hidden layer with strided convolution, 64 features, tanh; second hidden layer with a GRU of 256 features; third hidden layer with transpose convolution, 64 features, tanh; and a final convolution output layer (Putzky et al., 2017). The architecture is generic in the sense that it is not tied to a particular optimizer.

Several image restoration experiments are used to probe the framework. For reconstruction from random projections, the paper compares RIM against a gradient-descent network (GDN), which does not use the current estimate $y$6 as input, and a feed-forward network (FFN) with the same inputs as RIM but without recurrent state. The reported findings are that RIM outperforms both GDN and FFN consistently, that FFN degrades more on difficult tasks, and that GDN performs worst and degrades notably beyond the number of steps seen during training (Putzky et al., 2017). These results support the role of both the current estimate and the hidden recurrent state.

A distinctive result is cross-task transfer to inpainting. The RIM is trained on random projection tasks and then applied to image inpainting by swapping only the likelihood gradient while keeping the learned inference parameters fixed. The paper reports that RIM transfers best, while GDN fails and FFN transfers only somewhat (Putzky et al., 2017). The same work also emphasizes generalization from $y$7 patches during training to $y$8 reconstructions at test time.

For denoising, the likelihood gradient is written as

$y$9

where $A$0 is a learned stabilizer to avoid divergence when $A$1 is very small; for denoising, $A$2 (Putzky et al., 2017). In grayscale denoising, the paper compares BM3D, LSSC, EPLL, opt-MRF, MLP, RTF-5, RIM-3task, and RIM-denoise, and reports that RIM-denoise achieves the best PSNR among compared methods in the table, while RIM-3task remains competitive despite never being trained directly on denoising. In super-resolution on BSD-300 patches, one RIM is trained for $A$3, $A$4, and $A$5 scales, and the reported table shows the best mean PSNR and SSIM across all three scales against bicubic interpolation, SRCNN, A+, and SelfExSR (Putzky et al., 2017). The paper also notes that this multi-scale model is relatively small, around 500k parameters.

These experiments are used to support a broader claim: by retaining the known likelihood gradient and learning the update rule, the model can exhibit prior-like behavior without an explicit prior density and can transfer across related inverse problems when the data-consistency term is changed.

4. Extension to nonlinear quantitative MRI and MR relaxometry

A subsequent line of work applies Recurrent Inference Machines to nonlinear quantitative MRI, specifically $A$6 and $A$7 mapping. In this setting, the paper proposes RIMs as inverse-problem solvers for MR relaxometry and explicitly contrasts them with model-based MLE and direct deep learning via ResNet (Sabidussi et al., 2021). The stated rationale is that the framework combines model-based consistency through the MR signal model and its likelihood gradient with learned priors or learned update rules via a recurrent CNN.

The relaxometry task is formulated as

$A$8

where $A$9 is the parameter vector at voxel $\max_x \log p(y \mid x) + \log p_\theta(x),$0, $\max_x \log p(y \mid x) + \log p_\theta(x),$1 are the weighted MR images, and $\max_x \log p(y \mid x) + \log p_\theta(x),$2 is the forward signal model for the $\max_x \log p(y \mid x) + \log p_\theta(x),$3-th contrast. For SNR $\max_x \log p(y \mid x) + \log p_\theta(x),$4, the noise is approximated as Gaussian (Sabidussi et al., 2021). The conventional estimator is written as

$\max_x \log p(y \mid x) + \log p_\theta(x),$5

with a spatial prior applied only to the inversion efficiency map $\max_x \log p(y \mid x) + \log p_\theta(x),$6, using a Laplacian-squared regularizer with empirically chosen weight $\max_x \log p(y \mid x) + \log p_\theta(x),$7 (Sabidussi et al., 2021).

The RIM update at iteration $\max_x \log p(y \mid x) + \log p_\theta(x),$8 is given by

$\max_x \log p(y \mid x) + \log p_\theta(x),$9

$$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$0

As in the 2017 inverse-problem formulation, the network receives the current estimate, the gradient of the negative log-likelihood with respect to parameters, and a memory state, while the prior is learned implicitly from training labels rather than specified explicitly (Sabidussi et al., 2021). The paper emphasizes that the likelihood term is used to compute the gradient input in the data domain, whereas the training loss is an $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$1 error in the parameter-map domain supervising all iterations: $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$2

For $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$3 mapping, the signal model is

$$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$4

with $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$5. For $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$6 mapping, the model is

$$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$7

with $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$8 (Sabidussi et al., 2021). The RIM architecture in this study uses an RNNCell composed of 4 convolutional layers, 2 GRUs, and zero padding to preserve image size; the number of steps is fixed at $$x_{t+1} = x_t + \gamma_t \nabla_x \Big( \log p(y \mid x) + \log p_\theta(x) \Big)\Big|_{x_t}.$$9. Because the first layer input size depends on $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$0, not $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$1, the paper states that the RIM can process sequences of arbitrary length $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$2, unlike the ResNet baseline (Sabidussi et al., 2021).

The reported empirical results are specific. For a $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$3 slice with $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$4 contrasts, runtime is 3.8 s for RIM, 27 s for ResNet, and 575 s for MLE, so the RIM is approximately 150× faster than MLE (Sabidussi et al., 2021). In simulated data, for SNR $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$5, all methods have similar median Relative Bias, but RIM consistently has the lowest CV across SNRs. In a phantom experiment for $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$6 mapping over the relevant range of phantom spheres, the paper reports Relative Bias values of 1.34% for RIM, 1.71% for MLE, and 31.06% for ResNet, together with CV values of 3.21%, 7.56%, and 7.5%, respectively (Sabidussi et al., 2021). In noisier in-vivo $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$7 acquisitions, the average CV is 6.4% for RIM, 17.1% for MLE, and 11.06% for ResNet. For in-vivo $x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$8 mapping, the average difference from MLE is −1.13 ms for RIM and +26.2 ms for ResNet (Sabidussi et al., 2021).

This extension is significant because it shows that the same learned-inference principle can be applied beyond linear inverse problems to nonlinear parameter estimation. The paper also reports robustness to slight variations of scanning parameters, though it notes that the best performance is generally obtained with a model trained on the matching acquisition scheme (Sabidussi et al., 2021).

5. Recursive Inference Machines for neural reasoning

In the neural-reasoning literature, Recursive Inference Machines are defined as a formal framework inspired by classical approximate inference algorithms such as SMC, Gibbs sampling, and the broader Inference Machine paradigm (Komisarczyk et al., 5 Mar 2026). The central thesis is that neural reasoners can be understood as recursive inference engines that repeatedly refine an internal latent state and an output solution through structured iterations.

The formal object is the tuple

$x_{t+1} = x_t + g_\phi(\nabla_{y|x}, x_t),$9

where $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$0 is the problem description, $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$1 the initial solution, $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$2 the initial latent state, $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$3 the Solver, $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$4 the Generator, and $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$5 the Reweighter. The framework uses an inner loop repeated $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$6 times to update the latent state $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$7, and an outer loop repeated $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$8 times to update the solution $\nabla_{y|x} = \nabla \log p(y \mid x)(x_t)$9 (Komisarczyk et al., 5 Mar 2026). Operationally, the Solver proposes state updates, the Reweighter refines them, the Generator proposes solution updates, and the Reweighter again blends candidate updates with history.

A major claim of the paper is that TRMs are a special case of RIMs. The special case is called SimRIM, with Solver $g_\phi$0, Generator $g_\phi$1, and identity Reweighter $g_\phi$2, written as

$g_\phi$3

with updates

$g_\phi$4

$g_\phi$5

The paper states that SimRIM is identical to HRM, and that if $g_\phi$6 and $g_\phi$7 share the same backbone $g_\phi$8, this reduces to TRM (Komisarczyk et al., 5 Mar 2026). Under this interpretation, TRM corresponds to recursive latent-state refinement and recursive solution refinement but lacks an explicit correction mechanism beyond its raw updates.

The principal extension is the Reweighter. The paper argues that TRM-like systems mainly perform proposal generation and do not explicitly down-weight bad updates or retain useful history, whereas such correction is crucial in classical inference. This motivates two nontrivial RIM variants. The first, RIMA, uses exponential-moving-average-style reweighting: $g_\phi$9 with gating coefficient

$s_t$0

The paper notes that $s_t$1 may be a fixed scalar, learnable scalar, vector, or input-dependent gate, and explicitly connects this mechanism to GRU, LRU, Mamba, and RWKV (Komisarczyk et al., 5 Mar 2026).

The second, RIMFormer, introduces a $s_t$2-lookback Reweighter: $s_t$3 implemented with a Transformer block

$s_t$4

where

$s_t$5

This allows explicit attention over the reasoning trajectory and is described as especially useful for tasks requiring backtracking (Komisarczyk et al., 5 Mar 2026).

The framework also includes TabRIM for tabular classification under observational noise. Here, observed noisy features are denoted $s_t$6, latent clean features $s_t$7, and the target label $s_t$8. The Generator produces the prediction

$s_t$9

with

$x$00

and evidence-consistency weight

$x$01

The paper states that TabRIM is equivalent to Gibbs sampling from a factorization approximated by

$x$02

with the goal of estimating

$x$03

Inference uses 10 samples, 5 burn-in steps, and final prediction via mean predictive probability (Komisarczyk et al., 5 Mar 2026).

6. Empirical findings, limitations, and recurring themes

The empirical results in the neural-reasoning paper are organized around four questions: whether reweighting is important, whether dynamic reweighting is better than static reweighting, whether larger lookback helps, and whether the framework generalizes to noisy tabular data (Komisarczyk et al., 5 Mar 2026). On ARC-AGI-1, the reported scores are: SimRIM (TRM) pass@1 40.5, pass@2 44.38; RIMFormer pass@1 43.25, pass@2 47.13; RIMA pass@1 42.5, pass@2 47.5. On ARC-AGI-2, the values are: TRM pass@1 4.6, pass@2 7.4; RIMFormer pass@1 5.83, pass@2 7.1; RIMA pass@1 9.9, pass@2 11.3. On Sudoku Extreme, the paper reports 87.16 for TRM, 87.84 for RIMA using $x$04 only, 87.57 for RIMA using $x$05 only, 89.34 for full RIMA, and 80.21 for RIMFormer. On Maze-Hard, the results are 85.3 for TRM, 87.7 for RIMFormer, and 87.0 for RIMA (Komisarczyk et al., 5 Mar 2026).

For tabular classification under heavy noise, where 25% of feature values are replaced by random values, the paper compares TabPFN with TabRIM on Cleveland Heart Disease and Ljubljana Breast Cancer. On Cleveland Heart Disease, TabPFN achieves AUC-ROC 0.85 and AUC-PR 0.83, while TabRIM achieves 0.87 and 0.84. On Ljubljana Breast Cancer, TabPFN reports AUC-ROC 0.63 and AUC-PR 0.42, while TabRIM reports 0.74 and 0.54 (Komisarczyk et al., 5 Mar 2026). The paper interprets these results as evidence that RIM-style Gibbs reasoning improves robustness to noisy tabular inputs.

Across the 2026 ablations, three explicit conclusions are reported: reweighting matters, dynamic reweighting is better than static reweighting, and longer lookback can help, particularly on backtracking-heavy tasks such as Maze-Hard (Komisarczyk et al., 5 Mar 2026). In the inverse-problem literature, the recurring empirical themes are different but structurally related: the current estimate $x$06 matters, hidden recurrent state $x$07 matters, and keeping the likelihood gradient explicit enables task transfer by likelihood swapping (Putzky et al., 2017). In MR relaxometry, the same general pattern appears as lower CV than MLE and ResNet, faster inference than MLE, and reasonable robustness to modest acquisition changes (Sabidussi et al., 2021).

Several limitations are also explicit. The original inverse-problem formulation assumes a known forward model and focuses experimentally on linear inverse problems, with a fixed iteration budget during training and the need for task-specific likelihood gradients when transferring to new tasks (Putzky et al., 2017). The relaxometry extension shows applicability to nonlinear problems but still notes some blur from spatial context and slight variation in performance across acquisition settings (Sabidussi et al., 2021). In the reasoning formulation, RIMFormer is mixed on ARC-AGI-2 and underperforms TRM on Sudoku Extreme in the reported setup, which the paper attributes to possible over-parameterization or benchmark-specific mismatch (Komisarczyk et al., 5 Mar 2026).

Taken together, these strands define RIMs as learned recursive inference programs that preserve an explicit interface to model structure—via likelihood gradients in inverse problems or reweighting and evidence consistency in reasoning—while delegating the update policy to trainable recurrence. This suggests that the unifying feature of the RIM family is not a single architecture, but the representation of inference itself as a learned, stateful, iterative computation.