LoopFormer: Iterative Transformer Refinement

• LoopFormer is a family of transformer methods that reapply shared blocks iteratively to refine hidden representations, enabling adaptive computation and length generalization.
• The architecture leverages parameter tying and explicit compute control through techniques like inner-loop inference and elastic-depth scheduling to improve model performance.
• It supports varied applications including algorithmic tasks and multi-view 3D reconstruction, demonstrating measurable gains in accuracy and efficiency across benchmarks.

Searching arXiv for the cited LoopFormer papers and closely related looped-transformer work. arxiv_search(query="^^^^^^^^5LoopFormer OR \5^^^^^^^^"Inner Loop Inference for Pretrained Transformers\"^^^^^^^^5 OR \5^^^^^^^^"Looped Transformers for Length Generalization\"^^^^^^^^5 OR \5^^^^^^^^"Elastic-Depth Looped Transformers\"^^^^^^^^5 OR \5^^^^^^^^"Déjà View: Looping Transformers\"^^^^^^^^5 OR \5^^^^^^^^"Stabilizing Extrapolation in Looped Transformers\"", max_results=^^^^5 OR \5LoopFormer OR \5^^^^, sort_by="relevance") LoopFormer denotes a class of looped-transformer methods in which a Transformer block, a small tied block stack, or a selected range of pretrained blocks is reapplied to an evolving hidden state, so that depth is treated as explicit iteration and compute can be varied at inference time. In the cited literature, the label is attached to several related but non-identical constructions: inference-time inner looping in frozen pretrained LLMs, weight-tied decoder-only Transformers trained for length generalization, elastic-depth looped Transformers for budget-conditioned language modeling and reasoning, and looped multi-view reconstruction transformers with variable refinement steps (&&&^^^^5LoopFormer OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&).

^^^^5 OR \5^^^^. Terminological scope and development

The term appears in multiple arXiv works rather than as a single canonical architecture. "Looped Transformers for Length Generalization" (^^^^5 OR \5LoopFormer OR \5 OR \5 OR \5^^^^) studies a decoder-only model that reuses one block for an adaptive number of steps on algorithmic tasks (&&&^^^^5LoopFormer OR \5^^^^&&&). "LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation" (^^^^5 OR \5LoopFormer OR \5 OR \5^^^^6) introduces a budget-conditioned looped LLM with shortcut-consistency training (&&&^^^^5 OR \5^^^^&&&). "Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Training" (^^^^5 OR \5LoopFormer OR \5 OR \5^^^^6) presents inference-time inner looping over selected mid-depth layers in pretrained off-the-shelf LLMs, and its technical summary explicitly names the method "LoopFormer" (&&&^^^^5 OR \5^^^^&&&). "Déjà View: Looping Transformers for Multi-View ^^^^5 OR \5^^^^D Reconstruction" (^^^^5 OR \5LoopFormer OR \5 OR \5^^^^6) uses the same summary label for a recurrent reconstruction transformer with a single shared LoopBlock (&&&^^^^5 OR \5^^^^&&&). "Stabilizing Extrapolation in Looped Transformers via Learned Stochastic Stopping" (^^^^5 OR \5LoopFormer OR \5 OR \5^^^^6) treats loop count as a learnable stochastic variable and frames stopping as a training-time design choice (&&&^^^^5 OR \5^^^^&&&).

This distribution of usages suggests that "LoopFormer" functions less as a uniquely standardized model name than as a family label for Transformer systems that expose iterative hidden-state refinement directly. Across these papers, the common premise is that repeated application of the same or selected Transformer computations can be preferable to allocating all depth to distinct parameters.

^^^^5 OR \5^^^^. Core mathematical idea

A central formulation comes from the residual interpretation of Transformer depth. For a standard pre-norm block,

PRESERVED_PLACEHOLDER_^^^^5LoopFormer OR \5^^^^

so layer index can be read as a refinement step applied to a propagated latent state. On this view, inference-time inner looping prolongs refinement by repeatedly re-applying a selected block range; if layers PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^ are looped PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^ times, the total number of block applications becomes PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^, with execution controlled by a mapping PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^ from step index to original layer index (&&&^^^^5 OR \5^^^^&&&).

The trained looped-transformer formulation makes the recurrence explicit. In the length-generalization setting, the update is

$h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$

where the same decoder-only block $M_\theta$ is reused for $T$ loop steps and the original input embeddings are injected at every step to prevent information loss (&&&^^^^5LoopFormer OR \5^^^^&&&). In the elastic-depth language-modeling setting, a tied stack $\Phi_k$ is looped while conditioning each iteration on normalized time and step size; with $c_i=\phi(t_{i-1})+\phi(\Delta_i)$, the shared stack is modulated so that coarse schedules approximate fine schedules without collapse (&&&^^^^5 OR \5^^^^&&&). In multi-view reconstruction, a shared LoopBlock evolves the token state as

PRESERVED_PLACEHOLDER_^^^^5 OR \5LoopFormer OR \5^^^^

with conditioning on interval endpoints so that one set of weights covers any PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ in a pre-specified range (&&&^^^^5 OR \5^^^^&&&).

Two technical motifs recur across these formulations. The first is parameter tying: the same block or block stack is reused over steps. The second is explicit compute control: loop count, step schedule, or repeated block interval becomes a user-visible degree of freedom at inference time.

^^^^5 OR \5^^^^. Principal architectural variants

The literature covers several distinct LoopFormer-style constructions.

Variant	Reused unit	Distinctive mechanism
Inner loop inference	Selected pretrained layers PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^	Test-time repetition with interpolation regularization
Length-generalization looped Transformer	One decoder-only block PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^	Input injection and adaptive loop count
Elastic-depth LoopFormer	Tied PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^-layer stack PRESERVED_PLACEHOLDER_^^^^5 OR \55^^^^	Time- and step-size conditioning with shortcut-consistency training
DéjàView	One LoopBlock	Frame attention, global attention, and variable-PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^6 reconstruction

In inference-time inner looping, the model itself is not retrained. A baseline hidden state at loop entry is saved, the selected block interval is re-applied PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^7 times, and the looped states are regularized by interpolation. Three concrete interpolation schemes are studied: uniform averaging, a moving average PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^8, and auto-alignment based on softmax-normalized inner products with the baseline state (&&&^^^^5 OR \5^^^^&&&).

In the length-generalization architecture, the loop is trained end-to-end from scratch. The base module is a GPT-^^^^5 OR \5^^^^ style decoder-only block with hidden size PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^9, PRESERVED_PLACEHOLDER_^^^^5 OR \5LoopFormer OR \5^^^^ attention heads, feed-forward size PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^, and no positional embeddings; block depth PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ varies by task, with PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ for Parity, PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ for Copy, PRESERVED_PLACEHOLDER_^^^^5 OR \55^^^^ for Addition, Binary-Sum, and Unique, and PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^6 for Multiplication (&&&^^^^5LoopFormer OR \5^^^^&&&).

Elastic-depth LoopFormer introduces explicit schedule awareness. Scalars in PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^7 are embedded by sinusoidal Fourier features and a small two-layer MLP of width PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^8 with SiLU activations, producing conditioning vectors that modulate each tied layer through AdaLN-Zero-style gating and scaling. The modulator emits PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^9, and the residual branches are gated and scaled rather than simply reused unchanged (&&&^^^^5 OR \5^^^^&&&).

DéjàView adapts the same loop principle to multi-view ^^^^5 OR \5^^^^D reconstruction. Each image is encoded by a frozen DINOv^^^^5 OR \5^^^^ ViT-B patch encoder; per-view tokens include learnable register tokens and a camera token, use ^^^^5 OR \5^^^^D rotary positional embeddings, and are processed by a single LoopBlock composed of frame attention followed by global attention. The block is repeated for PRESERVED_PLACEHOLDER_^^^^5 OR \5LoopFormer OR \5^^^^ refinement steps, with gated residual scales conditioned on the interval endpoints PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ (&&&^^^^5 OR \5^^^^&&&).

^^^^5 OR \5^^^^. Training regimes, stopping rules, and compute allocation

The training procedure depends on whether looping is introduced at inference or learned during pretraining or task training. Inner Loop Inference is the simplest case: it requires no training and applies “zero-shot” to off-the-shelf pretrained checkpoints. The block range is selected by fixing PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ small, for example PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^, sweeping candidate intervals PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ on a held-out WinoGrande dev set in the 5-shot setting, measuring PRESERVED_PLACEHOLDER_^^^^5 OR \55^^^^ under uniform interpolation, and reusing the top-performing interval across downstream benchmarks; the most beneficial loops lie in the mid-depth region, typically PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^6, PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^7 (&&&^^^^5 OR \5^^^^&&&).

For length generalization, supervision is placed only on the final prediction after the required number of loop steps. Each training example includes PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^8, the loop is unrolled to PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^9, and cross-entropy is computed on the final decode. Training uses a curriculum that gradually increases maximum sequence length, AdamW with initial learning rate PRESERVED_PLACEHOLDER_^^^^5 OR \5LoopFormer OR \5^^^^, cosine decay to zero once the maximum training length is reached, batch size PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^, and up to PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^k gradient steps per task. If the true PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ is unknown at test time, the paper proposes a maximum-confidence heuristic that selects the step minimizing cross-entropy or maximizing confidence over a bounded search up to PRESERVED_PLACEHOLDER_^^^^5 OR \5 OR \5^^^^ (&&&^^^^5LoopFormer OR \5^^^^&&&).

Elastic-depth LoopFormer is trained on variable-length trajectories. Each batch samples a full trajectory of length PRESERVED_PLACEHOLDER_^^^^5 OR \55^^^^ with uniform steps and a shortcut trajectory of random length PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^6 with randomly sampled step sizes summing to PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^7. The objective combines full-trajectory LM loss, shortcut-trajectory LM loss, and a consistency loss with stop-gradient on the full trajectory,

PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^8

At inference, the user chooses a compute budget PRESERVED_PLACEHOLDER_^^^^5 OR \5^^^^9 and a step schedule $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$^^^^5LoopFormer OR \5^^^^ summing to $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$^^^^5 OR \5^^^^ (&&&^^^^5 OR \5^^^^&&&).

A separate line of work studies how stopping itself shapes the learned computation. Stochastic schedules sample $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$^^^^5 OR \5^^^^ from a local window, for example $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$^^^^5 OR \5^^^^ with $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$^^^^5 OR \5^^^^. RL-Halting adds a halting head $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$5 over depths $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$6 and trains it with REINFORCE using reward $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$7, a moving-average baseline, and entropy regularization. The stated point is that “when to stop” should be treated as a training-time design choice, not merely an inference-time computation-allocation rule (&&&^^^^5 OR \5^^^^&&&).

In multi-view reconstruction, variable-$h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$8 training samples $h^{(t+1)}=M_\theta(h^{(t)}+x), \qquad h^{(0)}=x,$9 scaled and rounded into $M_\theta$^^^^5LoopFormer OR \5^^^^. At test time, any $M_\theta$^^^^5 OR \5^^^^ in this range may be chosen, and explicit re-partitioning of the time interval is reported to outperform early stopping a ^^^^5 OR \5^^^^6-step rollout at the same compute (&&&^^^^5 OR \5^^^^&&&).

5. Quantitative performance across tasks and domains

For inference-time inner looping in pretrained LLMs, reported gains are small but consistent on several benchmarks. On Gemma-^^^^5 OR \5^^^^B, 5-shot WinoGrande improves from 68.75% to 69.5^^^^5 OR \5^^^^% under uniform looping, a gain of +^^^^5LoopFormer OR \5^^^^.78; moving-average and auto-alignment reach 69.^^^^5LoopFormer OR \5^^^^6% and 68.98%, respectively. On Gemma-9B, 5-shot WinoGrande moves from 77.^^^^5 OR \55^^^^% to 77.^^^^5 OR \5 OR \5^^^^% under uniform looping. Gemma-^^^^5 OR \5^^^^B on ARC-Challenge, ^^^^5 OR \55^^^^-shot, improves from 5^^^^5 OR \5^^^^.8^^^^5 OR \5^^^^% to 5^^^^5 OR \5^^^^.^^^^5 OR \5 OR \5^^^^% (+^^^^5LoopFormer OR \5^^^^.59); GSM8K, 5-shot, from ^^^^5 OR \55^^^^.^^^^5LoopFormer OR \5 OR \5^^^^% to ^^^^5 OR \5^^^^6.^^^^5LoopFormer OR \5LoopFormer OR \5^^^^% (+^^^^5LoopFormer OR \5^^^^.98); HellaSwag, ^^^^5 OR \5LoopFormer OR \5^^^^-shot, from 7^^^^5 OR \5^^^^.5^^^^5 OR \5^^^^% to 7^^^^5 OR \5^^^^.7^^^^5LoopFormer OR \5^^^^% (+^^^^5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^9); and MMLU, 5-shot, from 5^^^^5 OR \5^^^^.9^^^^5 OR \5^^^^% to 5^^^^5 OR \5^^^^.^^^^5 OR \5^^^^9% (+^^^^5LoopFormer OR \5^^^^.56). ARC-Easy, ^^^^5LoopFormer OR \5^^^^-shot, shows a small drop for Gemma-^^^^5 OR \5^^^^B, while Gemma-9B holds steady. Llama-^^^^5 OR \5^^^^-8B shows less consistent gains, with some tasks improving by approximately ^^^^5LoopFormer OR \5^^^^.^^^^5 OR \55^^^^–^^^^5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^% and others degrading (&&&^^^^5 OR \5^^^^&&&).

On algorithmic tasks, the trained looped-transformer variant is presented as a solution to length generalization. Training lengths are $M_\theta$^^^^5 OR \5^^^^ for Parity, Copy, and Addition, and $M_\theta$^^^^5 OR \5^^^^ for Multiply. Standard Transformers in the paper’s NTP and FAP families drop to near-zero accuracy just beyond the maximum training length, whereas the Looped Transformer with oracle stopping stays near ^^^^5 OR \5LoopFormer OR \5LoopFormer OR \5^^^^% accuracy for Parity up to $M_\theta$^^^^5 OR \5^^^^–MθM_\theta5 and nearly perfect for Copy and Add up to $M_\theta$6. With maximum-confidence stopping, the accuracy is described as virtually unchanged (&&&^^^^5LoopFormer OR \5^^^^&&&).

Elastic-depth LoopFormer targets budget-conditioned language modeling and reasoning. At ^^^^5 OR \5 OR \5^^^^× FLOPs, corresponding to approximately ^^^^5 OR \5 OR \5^^^^ Transformer layers, Base has The Pile perplexity 9.^^^^5 OR \5^^^^9 and zero-shot reasoning average ^^^^5 OR \55^^^^.^^^^5 OR \5^^^^7%; Base-Loop has ^^^^5 OR \5LoopFormer OR \5^^^^.9^^^^5 OR \5^^^^ and ^^^^5 OR \5 OR \5^^^^.88%; TMLT has ^^^^5 OR \5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^8 and ^^^^5 OR \5 OR \5^^^^.69%; and LoopFormer has ^^^^5 OR \5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^8 and ^^^^5 OR \5 OR \5^^^^.8^^^^5 OR \5^^^^%. At ^^^^5 OR \5 OR \5^^^^× FLOPs, Base has perplexity 9.98 and accuracy ^^^^5 OR \5 OR \5^^^^.9^^^^5 OR \5^^^^%, while LoopFormer at ^^^^5 OR \5^^^^×^^^^5 OR \5^^^^ has ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5 OR \5^^^^ and ^^^^5 OR \5 OR \5^^^^.7^^^^5 OR \5^^^^%. At 6× FLOPs, Base has ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5 OR \5^^^^ and ^^^^5 OR \5 OR \5^^^^.7^^^^5 OR \5^^^^%, while LoopFormer at ^^^^5 OR \5^^^^×^^^^5 OR \5^^^^ has ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5LoopFormer OR \5^^^^ and ^^^^5 OR \5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^6%. The reported qualitative trend is smooth scaling with budget and no sudden collapse at $M_\theta$7, in contrast with early-exit looped baselines (&&&^^^^5 OR \5^^^^&&&).

In multi-view ^^^^5 OR \5^^^^D reconstruction, DéjàView reports a markedly different operating regime. From the model-efficiency table, LoopFormer uses ^^^^5 OR \5 OR \5^^^^7 M parameters, 75.9 total FLOPs, ^^^^5 OR \5^^^^.^^^^5 OR \5^^^^ FLOPs per image, and ^^^^5 OR \5^^^^.9 GiB peak memory, with average Inlier Ratio 8^^^^5LoopFormer OR \5^^^^.^^^^5 OR \5^^^^ and average AUC@^^^^5 OR \5LoopFormer OR \5^^^^° 9^^^^5 OR \5^^^^.8. The same table lists larger feed-forward baselines such as Pi^^^^5 OR \5^^^^ at 959 M parameters and VGGT at ^^^^5 OR \5 OR \55^^^^7 M parameters. The paper states that LoopFormer achieves the highest average Inlier Ratio and AUC@^^^^5 OR \5LoopFormer OR \5^^^^° at just ^^^^5 OR \5 OR \5^^^^7 M parameters and comparable or lower compute and memory (&&&^^^^5 OR \5^^^^&&&).

For extrapolation stability, learned stochastic stopping changes both mean performance and variance. Across five runs, Addition under fixed $M_\theta$8 has OOD ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5^^^^, Front.@9^^^^5LoopFormer OR \5^^^^ ^^^^5 OR \55^^^^, and Std. 7.^^^^5 OR \5^^^^; $M_\theta$9 has ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5^^^^, ^^^^5 OR \5LoopFormer OR \5^^^^, and 9.9; $T$^^^^5LoopFormer OR \5^^^^ has ^^^^5 OR \5LoopFormer OR \5^^^^.9, ^^^^5 OR \55^^^^, and 5.9; and RL-Halt has ^^^^5 OR \55^^^^.^^^^5LoopFormer OR \5^^^^, ^^^^5 OR \5LoopFormer OR \5^^^^, and ^^^^5 OR \5^^^^.7. On Dyck-^^^^5 OR \5^^^^, RL-Halt reaches OOD 97.5, Front.@9^^^^5LoopFormer OR \5^^^^ 6^^^^5LoopFormer OR \5^^^^, and Std. ^^^^5 OR \5^^^^.6. On Unique Set, RL-Halt gives 8^^^^5 OR \5^^^^.8, ^^^^5 OR \5LoopFormer OR \5^^^^, and ^^^^5 OR \5^^^^.^^^^5 OR \5^^^^. On Copy, RL-Halt gives ^^^^5 OR \5 OR \5^^^^.^^^^5 OR \5^^^^, ^^^^5 OR \55^^^^, and ^^^^5 OR \5^^^^.7, lower mean performance than some alternatives but with markedly lower variance (&&&^^^^5 OR \5^^^^&&&).

6. Representation dynamics, misconceptions, and limitations

Several papers argue that looped Transformers implement iterative refinement in hidden space. In the inference-time looping work, PCA-based trajectory plots show that looped states slightly diverge from the baseline trajectory during the loop segment, re-converge after exiting the loop, but remain distinct enough to induce small logit differences that can flip the argmax. The reported qualitative effect is further “semantic sharpening,” including some self-correction of early mis-predictions. Regularized looping yields smooth and stable trajectories, whereas naïve looping can produce erratic collapse (&&&^^^^5 OR \5^^^^&&&).

The elastic-depth language-modeling work makes this claim more explicit through diagnostics. It measures anisotropy, curvature, prompt entropy, and CKA similarity over normalized loop depth. Early-exit baselines remain nearly flat, with CKA approximately ^^^^5 OR \5^^^^ even across distant loops, which the paper interprets as stagnation. By contrast, LoopFormer shows rising anisotropy and entropy through mid-depth, a mid-trajectory curvature peak, and progressively drifting CKA, which the paper interprets as ongoing refinement. Ablations further report that removing consistency loss causes collapse under truncation, timestep-only conditioning is insufficient under aggressive truncation, and gating the residual is necessary to stabilize coarse schedules (&&&^^^^5 OR \5^^^^&&&).

A common misconception is that looping merely injects perturbations or simply trades parameter count for repeated identical computation. The cited results argue against both simplifications. In the inference-time setting, a noise ablation using random perturbations of the same average magnitude never matches structured looping improvements, indicating that the effect is not explained by generic perturbation. In DéjàView, one shared block repeated ^^^^5 OR \5^^^^6 times outperforms a variant with ^^^^5 OR \5^^^^6 independent blocks under matched training data and compute, which is presented as evidence that explicit iteration is not merely a compute-efficient substitute for capacity but a stronger inductive bias for multi-view ^^^^5 OR \5^^^^D reconstruction (&&&^^^^5 OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&).

The limitations are equally consistent across papers. Inner looping adds $T$^^^^5 OR \5^^^^ extra block applications; with $T$^^^^5 OR \5^^^^ and loop length approximately $T$^^^^5 OR \5^^^^, the stated overhead is approximately ^^^^5 OR \5LoopFormer OR \5^^^^% inference compute for an approximately ^^^^5LoopFormer OR \5^^^^.5%–^^^^5 OR \5^^^^% accuracy gain. In the length-generalization setting, training cost grows linearly with maximum loop steps, oracle stopping requires ground-truth $T$^^^^5 OR \5^^^^, and very large $T$5 may be difficult to train without intermediate supervision; the no-positional-embedding design may also fail on tasks requiring absolute positions beyond RASP-L primitives. In multi-view reconstruction, performance improves monotonically up to $T$6 but plateaus and eventually degrades beyond that range, with unbounded drift in a few channels and metric collapse reported for extrapolation beyond the trained range. In stochastic stopping, RL-Halting can stabilize a suboptimal computation, as illustrated by the Copy task (&&&^^^^5LoopFormer OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&, &&&^^^^5 OR \5^^^^&&&).

Proposed extensions remain closely aligned with the core theme of controllable iterative compute. These include early-exit criteria based on $T$7 or logit-margin stability, adaptive selection of $T$8 per sample, selective deployment on hard or low-confidence inputs, integration with chain-of-thought prompting using looped hidden states as soft “reasoning traces,” and application to other modalities or encoder-decoder models (&&&^^^^5 OR \5^^^^&&&).