Spiralformer: Dual Transformer Architectures

• Spiralformer is a dual-use name for two Transformer models that employ a spiral computation schedule to optimize either low-latency speech recognition or recursive language modeling.
• In streaming ASR, Spiralformer reduces latency by cyclically skipping encoder layers and enabling early output, achieving significant reductions in system word delay with minimal accuracy loss.
• In language modeling, Spiralformer leverages multi-resolution recursion to efficiently capture hierarchical dependencies, thereby improving perplexity and scaling performance.

Searching arXiv for the specified Spiralformer papers to ground the article in the latest paper records. Searching arXiv for (Tsunoo et al., 1 Oct 2025) and (Yu et al., 12 Feb 2026). The term Spiralformer designates two distinct Transformer-derived architectures introduced in 2025–2026 under nearly identical names. In streaming automatic speech recognition, “Spiralformer: Low Latency Encoder for Streaming Speech Recognition with Circular Layer Skipping and Early Exiting” defines a block-processing encoder that reduces encoding latency through circular layer skipping and early exiting (Tsunoo et al., 1 Oct 2025). In large language modeling, “SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion” defines a recursive (looped) Transformer whose central mechanism is multi-resolution recursion, in which loop iterations operate at progressively finer sequence resolutions (Yu et al., 12 Feb 2026). The shared name reflects a recurring “spiral” computation pattern, but the two systems target different problem settings, use different computational primitives, and optimize different efficiency–quality trade-offs.

1. Nomenclature and scope

The speech-recognition Spiralformer is a low latency encoder for streaming speech recognition with circular layer skipping and early exiting (Tsunoo et al., 1 Oct 2025). Its design is explicitly motivated by blockwise processing in streaming ASR, where each block contains left context $N_l$, current chunk $N_c$, and right context $N_r$, and where reducing token emission delay typically requires reducing $N_c$, which in turn raises computational cost (Tsunoo et al., 1 Oct 2025).

The language-model SpiralFormer is a looped Transformer that executes recurrence under a multi-resolution recursion schedule (Yu et al., 12 Feb 2026). Its motivation is different: prior recursive Transformers repeatedly apply shared layers at full sequence resolution, whereas SpiralFormer varies the resolution across loop iterations so that early iterations operate on compressed representations and later iterations refine at token level (Yu et al., 12 Feb 2026).

A common misconception is that these papers describe the same architecture adapted across modalities. The available descriptions do not support that interpretation. The ASR model combines layer dropping, cyclic layer selection, cached intermediate outputs, and early exiting for blockwise speech encoding (Tsunoo et al., 1 Oct 2025). The language-model architecture instead combines downscaling, a shared Transformer core, upscaling, causal right-shift, and a topology operator such as Anchor or MeSH within a recursive computation loop (Yu et al., 12 Feb 2026). This suggests that “Spiralformer” is presently best understood as a shared naming motif rather than a single unified model family.

2. Spiralformer in streaming speech recognition

In the ASR setting, streaming encoders based on Transformers or Conformers commonly use blockwise processing. The central latency quantity described in the paper is that system-wide latency = $N_c + N_r$, so smaller $N_c$ enables more frequent output and lower latency, but also increases compute because blocks are emitted more often (Tsunoo et al., 1 Oct 2025).

Spiralformer addresses this latency–compute trade-off by combining two mechanisms. The first is Circular Layer Skipping: instead of computing all $I$ encoder layers for every block, only a subset is computed per block in a cyclic fashion, with a fixed pitch $p$, and the identity of the computed layers shifts from block to block (Tsunoo et al., 1 Oct 2025). The paper defines the shift index for block $b$ as

$s = (b-1) \bmod p$

and the computed layer set as

$N_c$0

For each block, only

$N_c$1

layers are computed (Tsunoo et al., 1 Oct 2025). Over multiple blocks, all layers are periodically computed for each input frame, completing the “spiral.”

The second mechanism is Early Exiting. For each block, output is taken from the last layer that was actually computed, rather than always from the final encoder layer (Tsunoo et al., 1 Oct 2025). This allows token emission without waiting for the full stack. Intermediate outputs are supervised during training through additional loss terms, summarized as

$N_c$2

The encoder recurrence for ordinary block processing is written as

$N_c$3

whereas Spiralformer computes, for block $N_c$4 and layer $N_c$5 in the selected set,

$N_c$6

With cached computation, the formulation becomes

$N_c$7

where $N_c$8 is the cached output for the previous block’s lower layer (Tsunoo et al., 1 Oct 2025). The output for a block is then taken from the last computed layer in the set, denoted

$N_c$9

Within the paper’s framing, the essential significance of Spiralformer is that frequent small-shift block emissions can be made computationally manageable by computing only part of the encoder at each block, while cached context preserves dependency structure across blocks (Tsunoo et al., 1 Oct 2025). A plausible implication is that the design moves part of the latency optimization problem from transducer emission policy into encoder scheduling itself.

3. Quantitative behavior in speech recognition

The paper reports results on LibriSpeech and CSJ (Tsunoo et al., 1 Oct 2025). On LibriSpeech, the comparison highlighted in the summary is between a baseline Conformer configuration and Spiralformer variants. The Standard Baseline (B2), with 12 layers and $N_r$0, has Max theoretical latency: 640 ms, RTF: 0.15, WER (test-clean/other): 3.4% / 8.6%, and SWD (System Word emission Delay), P50: 589 ms (Tsunoo et al., 1 Oct 2025). The Spiralformer S3 configuration with $N_r$1 has Max theoretical latency: 400 ms, RTF: 0.20, WER: 3.6% / 9.1%, and SWD(P50): 462 ms (Tsunoo et al., 1 Oct 2025). The paper describes this as a 37.5% reduction in maximum theoretical latency and a 21.6% reduction in SWD(P50) relative to the baseline, while maintaining similar computational cost and word error rates (Tsunoo et al., 1 Oct 2025).

The same summary reports that S2 ($N_r$2) achieves SWD(P50): 389 ms, with the qualification that it comes with higher RTF/less WER benefit (Tsunoo et al., 1 Oct 2025). Table 2 is described as reporting median and 90th-percentile delays, including FWD(P50) 420 ms for Spiralformer S2 versus 990 ms for B2, and SWD(P50) 462 ms for Spiralformer S3 versus 589 ms for B2 (Tsunoo et al., 1 Oct 2025).

On CSJ, the reported baseline B3 ($N_r$3) has SWD(P50): 385 ms, whereas Spiralformer S3 has SWD(P50): 358 ms, a 7% reduction, while maintaining similar Character Error Rates (Tsunoo et al., 1 Oct 2025).

The summary also states that RTF for Spiralformer with aggressive skipping is lower or similar to baseline, even with much smaller chunk sizes ($N_r$4), and that First Word Delay and SWD are consistently reduced across Spiralformer models versus comparable compute or accuracy baselines (Tsunoo et al., 1 Oct 2025). An ablation finding is that training from scratch can increase emission delays, and that the best emission timings arise from fine-tuning from a pretrained blockwise encoder (Tsunoo et al., 1 Oct 2025).

Configuration	Key values	Source
Baseline B2	640 ms max latency; RTF 0.15; WER 3.4% / 8.6%; SWD(P50) 589 ms	(Tsunoo et al., 1 Oct 2025)
Spiralformer S3	400 ms max latency; RTF 0.20; WER 3.6% / 9.1%; SWD(P50) 462 ms	(Tsunoo et al., 1 Oct 2025)
CSJ comparison	B3 SWD(P50) 385 ms; S3 SWD(P50) 358 ms	(Tsunoo et al., 1 Oct 2025)

These figures support the paper’s specific claim that encoding latency in blockwise streaming ASR can be reduced without a large accuracy penalty by restructuring how encoder depth is distributed across blocks rather than evaluating the full depth at every step (Tsunoo et al., 1 Oct 2025).

4. SpiralFormer as a recursive LLM

The 2026 SpiralFormer is introduced as a recursive (or looped) Transformer architecture that uses multi-resolution recursion to improve parameter and compute efficiency and to enable learning of hierarchical dependencies (Yu et al., 12 Feb 2026). The motivating contrast is with standard recursive Transformers, which repeatedly apply a shared Transformer block at full sequence resolution during every recurrence (Yu et al., 12 Feb 2026).

The architecture defines, for loop iteration $N_r$5, a resolution $N_r$6 and effective sequence length

$N_r$7

where $N_r$8 is the token sequence length (Yu et al., 12 Feb 2026). The schedule is typically coarse-to-fine, for example

$N_r$9

so early loops operate at $N_c$0 or $N_c$1 resolution and the final loop reaches full token resolution (Yu et al., 12 Feb 2026).

One loop iteration comprises five operations. First, Downscale compresses token-level hidden states $N_c$2 into chunk-level latents $N_c$3: $N_c$4 This is typically implemented as self-aggregation, a learnable weighted mean within each chunk: $N_c$5

Second, the model applies the shared loop core to the chunk-level sequence: $N_c$6

Third, Upscale maps chunk-level updates back to token positions: $N_c$7 where $N_c$8 is predicted by a lightweight router $N_c$9, and $N_c + N_r$0 (Yu et al., 12 Feb 2026).

Fourth, a causal right-shift enforces strict autoregressive causality: $N_c + N_r$1 By default, $N_c + N_r$2, producing a single-token overlap between generating and receiving chunk (Yu et al., 12 Feb 2026).

Fifth, the running state is updated using a topology operator $N_c + N_r$3: $N_c + N_r$4 The paper identifies two topology choices: Anchor, which is stateless and adds to a fixed anchor state, and MeSH, a learned memory buffer that manages state across iterations (Yu et al., 12 Feb 2026).

The overall computation is summarized as Pre-block → $N_c + N_r$5 multi-resolution loop steps → post-block (Yu et al., 12 Feb 2026). The intended effect is that early iterations perform low-cost global interaction on compressed representations, while later iterations perform fine-grained local refinement on token-level states.

5. Hierarchical specialization and empirical scaling in language modeling

A central claim of SpiralFormer is that multi-resolution recursion enables the model to learn hierarchical dependencies by inducing iteration-wise functional specialization across different scales (Yu et al., 12 Feb 2026). The paper reports probing evidence for this claim. Specifically, key-marginal entropy of attention heads decreases with higher resolution, meaning attention becomes more selective at finer scales, and Local Attention Mass (LAM) increases at higher resolution, indicating that later iterations focus more on local neighborhoods (Yu et al., 12 Feb 2026). The paper contrasts this with full-resolution looping, which does not exhibit the same systematic cross-loop specialization.

The empirical evaluation is conducted on the Pythia suite (160M, 410M, 1B, 1.4B parameter scales) pretrained on the Pile (250B tokens, one epoch) (Yu et al., 12 Feb 2026). The comparison includes a Baseline non-recursive Transformer, LoopedFormer with full-resolution recursion, SpiralFormer-B, and SpiralFormer-L (Yu et al., 12 Feb 2026).

The principal quantitative findings are stated at a high level. SpiralFormer-B, with the same parameter allocation as the full-resolution recursive baseline, consistently improves validation perplexity and few-shot task accuracy while reducing FLOPs by 7–11% versus LoopedFormer (Yu et al., 12 Feb 2026). SpiralFormer-L, with the same parameter count as the non-looped baseline, reduces prefill FLOPs by 3–10% and improves or matches perplexity and downstream accuracy (Yu et al., 12 Feb 2026). At the 1.4B scale, the paper reports a reduction in FLOPs from 14.08T → 13.13T and an increase in average 5-shot accuracy from 51.93 to 54.37 (Yu et al., 12 Feb 2026).

The paper further states that both SpiralFormer variants define a superior loss–compute and accuracy–parameter frontier relative to both baselines, and that the efficiency and performance gap widens at larger scale (Yu et al., 12 Feb 2026). This suggests that the architecture’s main contribution is not merely a constant-factor efficiency gain but a different scaling behavior in recursive Transformer design.

6. Ablations, system implications, and comparative interpretation

The language-model paper identifies several architectural choices as important. Multiresolution scheduling (coarse-to-fine) is essential; reversing the schedule (fine to coarse) degrades performance (Yu et al., 12 Feb 2026). MeSH topology outperforms Anchor, indicating the importance of a learned memory across loops (Yu et al., 12 Feb 2026). Learnable self-aggregation/upscaling is better than mean pooling/uniform upscaling, and the causal overlap regime is best in quality, whereas the parallel regime (no chunk overlap) enables system-level speedups (Yu et al., 12 Feb 2026).

It also states that right-shift scheduling is provably necessary for causality with chunked latent aggregation, and that chunk offset $N_c + N_r$6 modulates the periodic pattern and helps ensure uniform per-token computation (Yu et al., 12 Feb 2026). In the no-overlap regime $N_c + N_r$7, low-resolution chunked computations for earlier tokens can be overlapped with high-resolution computations for the current token, enabling inference-time pipelining (Yu et al., 12 Feb 2026).

By contrast, the ASR Spiralformer’s deployment implications are centered on emission timing rather than autoregressive decoding pipelines. Its key finding is that frequent block emissions with small chunk shifts can be reconciled with manageable compute by skipping layer computation in a cyclic manner and relying on cached intermediate outputs plus early exiting (Tsunoo et al., 1 Oct 2025). Its ablation result that fine-tuning from a pretrained blockwise encoder yields the best emission timings further distinguishes it from the language-model SpiralFormer, whose focus is recursive scaling behavior rather than transfer from a blockwise pretrained encoder (Tsunoo et al., 1 Oct 2025).

Taken together, the two Spiralformer architectures illustrate two distinct uses of a spiral computation schedule. In the ASR system, the spiral is a cyclic traversal of encoder depth across successive blocks (Tsunoo et al., 1 Oct 2025). In the LLM, the spiral is a coarse-to-fine traversal of sequence resolution across recursive iterations (Yu et al., 12 Feb 2026). The shared conceptual theme is staged computation under tight efficiency constraints, but the underlying state evolution, training supervision, and deployment objectives are not the same.

7. Position within Transformer research

The 2025 ASR Spiralformer is situated within work on streaming speech recognition, Transformer-based encoders, Conformers, and block processing, with a specific emphasis on encoding latency of the block processing, a dimension the abstract describes as having received relatively little attention compared with emission latency in transducers (Tsunoo et al., 1 Oct 2025). Its contribution is therefore best interpreted as an encoder-scheduling method for real-time ASR.

The 2026 SpiralFormer is situated within recursive/looped Transformer research and is described as building on architectures such as the Universal Transformer, while also connecting to work on latent reasoning, memory-augmented models, and efficient LLM scaling (Yu et al., 12 Feb 2026). Its stated novelty is to make multi-resolution recursion an explicit architectural primitive (Yu et al., 12 Feb 2026).

Because the two papers use near-identical names, bibliographic ambiguity is likely. For precision, the ASR model is most accurately identified by the subtitle “Low Latency Encoder for Streaming Speech Recognition with Circular Layer Skipping and Early Exiting” (Tsunoo et al., 1 Oct 2025), whereas the recursive LLM is most accurately identified by “Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion” (Yu et al., 12 Feb 2026). A plausible implication is that future usage of the term “Spiralformer” will require modality-specific disambiguation unless one of the two lineages becomes dominant.

At present, the name therefore refers not to a single canonical Transformer variant but to two technically separate proposals: one for low-latency blockwise speech encoding and one for hierarchical multi-resolution recursion in looped LLMs. Their joint significance lies less in a shared implementation than in a shared methodological tendency: replacing uniformly repeated full-stack computation with structured, staged computation that redistributes model work across time, depth, or resolution (Tsunoo et al., 1 Oct 2025, Yu et al., 12 Feb 2026).