LoopViT: Recurrent Vision Transformer

• The paper introduces LoopViT, a recurrent vision transformer that replaces static feed-forward layers with a weight-tied iterative core, achieving up to 65.8% accuracy on ARC tasks.
• It employs a Hybrid Block that fuses global self-attention with local convolutional processing to effectively induce rules and propagate spatial patterns.
• The model uses a Dynamic Exit mechanism based on predictive entropy and test-time training to adapt computation, balancing performance and efficiency across diverse ARC challenges.

Searching arXiv for LoopViT and closely related cited work to ground the article with fresh identifiers. {"query":"LoopViT Scaling Visual ARC with Looped Transformers arXiv (Shu et al., 2 Feb 2026)", "max_results": 5} LoopViT is a recurrent vision transformer for ARC-style visual reasoning that replaces the standard one-pass encoder with a weight-tied iterative core, thereby decoupling reasoning depth from model capacity. The model is introduced as a response to the claim that ARC-AGI is not well matched to the usual feed-forward ViT paradigm, because many ARC tasks require iterative rule induction and repeated local and global transformation before a consistent output emerges. Its defining components are a shared recurrent transformer trunk, a Hybrid Block that combines self-attention with convolutional local processing, and a parameter-free Dynamic Exit rule based on predictive entropy. On ARC-AGI-1, the reported headline result is that an 18M-parameter LoopViT reaches $65.8\%$ accuracy, exceeding both an 18M feed-forward VARC baseline at $54.5\%$ and a 73M-parameter VARC ensemble at $60.4\%$, which the paper presents as evidence that scaling “Time” is more effective than scaling “Space” for abstract visual reasoning (Shu et al., 2 Feb 2026).

1. Problem formulation and design premise

LoopViT is motivated by the observation that ARC tasks often require an iterative procedure: infer a rule from a few demonstrations, then mentally simulate a sequence of local and global transformations until a coherent output is produced (Shu et al., 2 Feb 2026). In a vanilla vision transformer, computational depth is fixed by the number of layers, so additional reasoning capacity is usually purchased by increasing depth or width, and therefore parameter count. The paper argues that this is a poor scaling strategy for algorithmic induction.

The proposed alternative is recurrence with tied weights. Instead of learning many distinct feed-forward layers, LoopViT learns a reusable transition operator that can be executed for multiple loop steps. This means that the amount of computation can grow with the number of iterations $T$, while the parameter count remains essentially fixed. The paper explicitly frames this as scaling “Time” rather than “Space.”

At the input level, LoopViT keeps the pure-vision formulation used by prior ARC systems such as VARC, but changes the computational graph from a static encoder to an iterative dynamical system. The model takes a visual grid $x$ and task-specific context $c$, embeds them into $M$ tokens of hidden dimension $d$, and preserves the 2D visual topology rather than serializing ARC into 1D text. The token set is heterogeneous: “task tokens” encode demonstration or context information, while “image tokens” represent the spatial visual input. Positional structure is handled with Rotary Positional Embeddings (RoPE), and the output head maps hidden states back to per-pixel categorical predictions over ARC grid colors.

A plausible implication is that LoopViT is designed for tasks that resemble visual algorithm execution more than conventional image classification: small-data settings with strong spatial structure, compositional transformations, and variable reasoning depth across instances.

2. Recurrent transformer dynamics

The paper contrasts a standard feed-forward transformer

$$\mathcal{F}(\cdot) := \text{head} \circ \mathcal{L}_{\theta_L} \circ \cdots \circ \mathcal{L}_{\theta_1} \circ \text{emb}(\cdot)$$

with LoopViT, which recursively applies a shared core $\mathcal{M}_\theta$:

$54.5\%$0

where $54.5\%$1 is a learned step-dependent embedding indicating progress through the loop. The prediction at step $54.5\%$2 is

$54.5\%$3

Recurrence is applied over the whole shared transformer trunk rather than over a single sublayer. The trunk itself may contain multiple hybrid encoder layers, and that stack is reused at every loop step. The paper also states that, for inference beyond the training loop budget, step embeddings are extrapolated by identity.

The authors interpret the hidden-state sequence $54.5\%$4 as a latent chain of thought: not a textual rationale, but a sequence of internal refinements in representation space. Early loop steps are described as exploring the problem and integrating support examples; later steps are described as sharpening and executing the inferred rule. The supporting evidence is qualitative rather than formal, but the reported prediction refinement, entropy decay, and changing attention patterns are presented as consistent with this interpretation (Shu et al., 2 Feb 2026).

3. Hybrid Block: global attention and local cellular priors

The main architectural novelty inside the recurrent core is the Hybrid Block, which is intended to address the coexistence of global rule induction and local pattern propagation in ARC (Shu et al., 2 Feb 2026). Many ARC transformations are described as cellular or neighborhood-based, such as extending a line, filling a region, or moving an object under gravity-like dynamics, while others require long-range consistency, symmetry, or object-level coordination. The Hybrid Block therefore combines self-attention with a convolutional gated feed-forward module.

The attention sublayer is standard multi-head self-attention with RoPE. For input $54.5\%$5, each head computes queries, keys, and values, applies RoPE to queries and keys, and forms the usual attention output. Recurrent stability is supported by pre-norm RMSNorm and residual connections:

$54.5\%$6

$54.5\%$7

Local processing enters through the ConvGLU sublayer, called a Heterogeneous ConvGLU because it treats task tokens and image tokens differently. A linear projection produces a gate branch and a value branch. Task-token gates bypass spatial convolution so that abstract task-level information is not corrupted, whereas image-token gates are reshaped to a 2D grid and passed through a $54.5\%$8 depthwise convolution. The resulting gate tensor modulates the value branch, and the final feed-forward output is produced by a second linear layer.

The stated intuition is that attention handles global information routing and rule induction, while depthwise convolution injects a local cellular-update prior for image tokens. The ablations reported in the paper show that replacing the hybrid core with a vanilla transformer core causes a consistent performance drop across core depths. The paper does not provide a fine-grained removal study in the extracted text, but it does report that the Hybrid-vs-Vanilla comparison persistently favors the hybrid design.

4. Training protocol, test-time adaptation, and Dynamic Exit

Training is intentionally simple. LoopViT is trained offline on ARC plus RE-ARC synthetic data, following the vision-based ARC setup (Shu et al., 2 Feb 2026). RE-ARC is used to augment the limited ARC training split with procedurally generated examples. During training, recurrence is unrolled for a fixed number of steps; an example value given is $54.5\%$9. Only the final prediction is supervised, using per-pixel cross-entropy:

$60.4\%$0

This choice is presented as a way to train the recurrent transition operator to converge to the correct answer within a fixed compute budget, rather than relying on adaptive halting during optimization.

Evaluation uses a two-stage procedure: offline pretraining followed by test-time training (TTT) on the support examples of each ARC task. During TTT, the shared weights $60.4\%$1 are fine-tuned on the few-shot demonstrations from the current task, using augmentations such as rotations, flips, and color permutations. The authors describe this as specializing the generic recurrent “thought step” into a task-specific algorithm for the current puzzle. The extracted text mentions the framework and the TTT idea, but does not report optimizer hyperparameters, exact schedules, or batch settings.

The Dynamic Exit mechanism is parameter-free and used only at inference time. Let

$60.4\%$2

denote the predicted pixelwise categorical distribution at step $60.4\%$3. The model computes the average pixelwise Shannon entropy

$60.4\%$4

where $60.4\%$5 is the number of pixels and $60.4\%$6 is the number of output color classes. Inference halts when

$60.4\%$7

with $60.4\%$8, or else at a hard maximum $60.4\%$9. Once the model halts at step $T$0, the state is frozen for all subsequent steps. The paper describes this as “crystallization”: over iterations, predictions become less volatile and less uncertain, and the hidden state appears to approach a stable low-entropy attractor.

5. Empirical results and scaling behavior

The paper reports three model sizes: Small with 3.8M parameters, Medium with 11.2M, and Large with 18M parameters (Shu et al., 2 Feb 2026). On ARC-AGI-1, LoopViT Large achieves $T$1 Pass@2 accuracy. LoopViT Medium reaches $T$2, and LoopViT Small reaches $T$3. The cited baselines are an 18M feed-forward VARC model at $T$4 and a 73M-parameter VARC ensemble at $T$5. The comparison is central to the paper’s thesis: the 11.2M model already exceeds the 73M ensemble, and the 3.8M model exceeds the 18M feed-forward baseline while using about one-fifth the parameters.

On ARC-AGI-2, the reported scores are 10.0 for Small, 11.5 for Medium, and 14.2 for Large, compared with 8.3 for VARC and 11.1 for the VARC ensemble. The gains therefore carry over somewhat out of distribution, although ARC-AGI-2 remains substantially harder.

The paper also reports comparisons to other recurrent token-based models and to LLMs. HRM (27M) is reported at 40.3 on ARC-AGI-1, and TRM (7M) at 44.6. Among LLMs, DeepSeek R1 is reported at 15.8, GPT-5 at 44.0, Grok-4-thinking at 66.7, and a bespoke Grok-4-based system at 79.6. The paper explicitly notes that its intended comparison is not primarily to frontier proprietary systems, but to visual ARC architectures that are efficient.

Ablation results are described as consistent with the claim that iterative depth matters more than raw parameter count. In joint scaling experiments, the recurrent core depth $T$6 and loop count $T$7 are varied together. For shallow cores such as $T$8, increasing $T$9 yields very large gains, suggesting that recurrence can emulate much of the expressive benefit of a deeper non-shared network. For larger cores such as $x$0, performance still improves as $x$1 increases up to the tested computational limit, peaking around $x$2 in that scaling plot. The paper’s interpretation is that computation over time remains useful even when the core is already expressive, rather than saturating immediately.

Dynamic Exit is also reported to improve the accuracy-compute tradeoff. In the efficiency plot, the entropy-based early-exit version achieves higher accuracy at lower average GFLOPs than fixed-step baselines. A difficulty analysis with a $x$3 model reports that samples exiting early at step 5 achieve 83.33% accuracy, while samples requiring all the way to step 8 achieve 45.80% accuracy. This suggests that required loop count correlates with instance difficulty, and that entropy is a useful proxy for whether the model is finished computing.

6. Diagnostics, interpretation, and limitations

The paper provides several mechanistic diagnostics of the recurrent process (Shu et al., 2 Feb 2026). One figure shows iterative prediction refinement, in which outputs become visibly closer to ground truth across recurrent steps. Consecutive-step pixelwise difference maps shrink over time, indicating lower volatility. Another figure tracks $x$4 over iterations and shows monotonic decay on average. The paper also refers to an $x$5-normalized difference metric $x$6 between successive predictions or states; although the exact formula is not given in the extracted text, the reported plot shows synchronized decay of $x$7 and entropy. Attention visualizations are described as dense and broad at early steps, corresponding to global scanning of demonstrations and query, and sparse and focused at later steps, corresponding to local execution of the inferred rule.

These analyses support, but do not prove, the latent-deliberation narrative. The paper itself is careful enough to make clear that “latent chain of thought” is an interpretation of hidden-state dynamics rather than an explicit symbolic reasoning trace. That distinction matters because the model does not expose a textual rationale, a discrete program, or a formal proof of iterative inference.

The strengths claimed for LoopViT are straightforward. It is highly parameter-efficient on ARC-AGI-1; it supports adaptive test-time computation without a learned halting module; its architectural biases are matched to ARC’s 2D spatial structure; and its recurrence offers a clean separation between model capacity and reasoning time. The limitations are equally important. The extracted text omits many implementation details, including precise embedding and decoding specifics, optimizer hyperparameters, exact TTT schedule, training compute budget, and some architecture hyperparameters. The method also depends on test-time training, which complicates comparison to systems that do not adapt weights during evaluation and may increase wall-clock latency. The Dynamic Exit threshold is fixed at $x$8, but threshold sensitivity is not deeply analyzed in the provided text. Finally, although ARC-AGI-2 results improve over visual baselines, they remain far from top proprietary systems and far from human ceiling.

Conceptually, LoopViT is positioned at the intersection of recurrent transformers with weight tying, adaptive computation, early exiting, and test-time compute scaling. The paper states that it is close in spirit to Universal Transformers and more recent looped LLMs, but transplanted into pure vision for ARC. Within that framing, its central claim is narrow and explicit: repeated application of a shared transition map is a better inductive bias for iterative visual reasoning than simply enlarging a static feed-forward transformer.