Moonshine v2: Efficient Streaming ASR

• Moonshine v2 is a family of streaming ASR models that uses a sliding-window self-attention mechanism to achieve bounded latency on edge devices.
• It optimizes time-to-first-token by processing variable-length speech segments without zero-padding, ensuring rapid, interactive responses.
• The architecture integrates lightweight audio preprocessing with an ergodic encoder and autoregressive decoder to maintain high transcription accuracy.

Moonshine v2 is a family of streaming automatic speech recognition models for latency-critical edge applications such as live transcription, voice commands, and real-time translation. It is introduced as an ergodic streaming-encoder ASR architecture that replaces the full-attention Transformer encoder used in the original Moonshine family with sliding-window self-attention, with the explicit goal of reducing time-to-first-token and overall interactive delay while preserving strong transcription accuracy (Kudlur et al., 12 Feb 2026). In the broader Moonshine lineage, the original Moonshine models were optimized for live transcription and voice command processing through an encoder-decoder Transformer architecture, Rotary Position Embedding, and training on variable-length speech segments without zero-padding; Moonshine Tiny was reported to achieve a 5× reduction in compute versus Whisper tiny.en on a 10-second speech segment with no increase in word error rate on the standard evaluation datasets used in the paper (Jeffries et al., 2024).

1. Historical placement within the Moonshine model family

The original Moonshine paper presents a family of ASR models optimized for live transcription and voice command processing, with the central efficiency claim that fixed-length, zero-padded inputs make many ASR encoders inefficient at inference. Moonshine addresses this by training on variable-length speech segments without zero-padding, so the encoder does not waste computation on padding tokens at inference. In that formulation, Moonshine is an encoder-decoder Transformer using Rotary Position Embedding instead of traditional absolute position embeddings (Jeffries et al., 2024).

Moonshine v2 retains the live-ASR orientation but shifts the primary optimization target from general encoder efficiency to bounded streaming latency. The motivating diagnosis is that full-attention Transformer encoders remain a strong accuracy baseline because every frame can directly attend to every other frame, but this induces an “encode-the-whole-utterance” latency profile. For streaming use cases, the consequence is that time-to-first-token grows roughly linearly with utterance duration because the encoder must process the entire prefix before any decoder token can be emitted (Kudlur et al., 12 Feb 2026).

This places Moonshine v2 as a second-generation system within the same application domain rather than a wholesale change of task. A plausible implication is that the transition from Moonshine to Moonshine v2 reflects a shift from compute-efficient short-segment inference toward architecture-level control of streaming latency. That interpretation is consistent with the explicit contrast between the original full-attention encoder and the Moonshine v2 sliding-window encoder (Jeffries et al., 2024, Kudlur et al., 12 Feb 2026).

2. Latency model and the motivation for the streaming redesign

Moonshine v2 is introduced for latency-critical speech applications where low time-to-first-token and high transcription accuracy are both required, particularly on resource-constrained edge devices. The paper explicitly frames 250 ms as a practical one-way latency upper bound for interactive voice, which makes encoder behavior central to system design (Kudlur et al., 12 Feb 2026).

In the full-attention encoder regime, attention mixing costs scale as $\mathcal{O}(T^2)$ for a sequence of $T$ frames. The paper gives an illustrative compute estimate

$\mathrm{ops}_{\text{total}(N)} = 6PT + 4dLT^2,$

where $P$ is model parameters, $d$ is hidden dimension, $L$ is number of layers, and $T=50N$ frames for $N$ seconds of audio at 50 Hz. The quadratic term arises from full self-attention, and this produces the unfavorable latency profile for streaming ASR (Kudlur et al., 12 Feb 2026).

Moonshine v2 replaces this with sliding-window self-attention, where each frame attends only to a fixed local neighborhood. With window size $w$, the attention mixing cost becomes $\mathcal{O}(Tw)$, and the corresponding compute estimate is

$T$0

For fixed $T$1, the attention term is linear in sequence length, and the resulting time-to-first-token becomes bounded and largely independent of utterance length (Kudlur et al., 12 Feb 2026).

The significance of this change is not merely asymptotic. The paper’s conceptual timeline emphasizes that encoding overlaps with audio capture, so much of the work is done incrementally rather than after the utterance ends. This suggests that Moonshine v2 is designed around interactive response constraints rather than offline throughput alone (Kudlur et al., 12 Feb 2026).

3. Architecture and the meaning of the “ergodic streaming encoder”

Moonshine v2 is built from four components: an audio preprocessor, an ergodic streaming encoder, an adapter, and an autoregressive decoder (Kudlur et al., 12 Feb 2026).

The audio preprocessor is intentionally lightweight and causal. It segments raw audio into non-overlapping 80-sample windows at 16 kHz, applies cepstral mean and variance normalization, passes values through an $T$2 nonlinearity, and then uses two causal stride-2 convolutions to reduce the frame rate by about $T$3. The output is about 50 feature frames per second, matching Whisper’s feature rate (Kudlur et al., 12 Feb 2026).

The encoder is a Transformer stack with sliding-window self-attention and no positional embeddings at all. Because the encoder has neither absolute nor relative positions, its computation is translation-invariant in time. The paper calls this an ergodic encoder: the same local function is applied everywhere, and structure is inferred from local content rather than explicit time indices (Kudlur et al., 12 Feb 2026).

The attention window is denoted $T$4. Moonshine v2 uses $T$5 in the first two and last two encoder layers, and $T$6 in all intermediate layers. Since features are at 50 Hz, one frame corresponds to 20 ms, so a right-context of four frames corresponds to 80 ms of future context. The paper also notes a conservative bound of 16 frames of extra audio for finalized states in steady state, corresponding to 320 ms. As a result, provisional outputs may be emitted and then refined as future audio arrives (Kudlur et al., 12 Feb 2026).

The adapter restores position awareness where it is needed: it adds a learned positional embedding to encoder outputs and can optionally project features to match decoder dimensionality. This preserves the position-free character of the encoder while allowing a standard sequence-to-sequence decoder interface (Kudlur et al., 12 Feb 2026).

The decoder is a conventional causal Transformer using rotary positional embeddings and cross-attending to encoder features. This decoder choice preserves the autoregressive text-generation paradigm used in the original Moonshine family, but it also imposes an important systems constraint: even when encoder latency is bounded, transcript generation remains serial and token-by-token (Jeffries et al., 2024, Kudlur et al., 12 Feb 2026).

4. Training configuration, model scales, and evaluation protocol

Moonshine v2 is trained on roughly 300K hours of speech data, consisting of about 200K hours from the original Moonshine setup plus an additional 100K hours of internally prepared speech data (Kudlur et al., 12 Feb 2026). The training recipe uses the same tokenizer as the original Moonshine, the Schedule-Free optimizer, an initial learning rate of $T$7, 400K steps, an effective batch size of 512, and training on 8 NVIDIA H100 GPUs (Kudlur et al., 12 Feb 2026).

Three model sizes are reported:

Model	Parameters
Tiny	33.57M
Small	123.36M
Medium	244.93M

The paper states that the encoder is relatively compact, while the decoder is larger because it includes cross-attention and SwiGLU feed-forward blocks (Kudlur et al., 12 Feb 2026).

Evaluation separates accuracy and latency tooling. Accuracy is measured with the Hugging Face Transformers implementation, while latency is measured with a C++/ONNX implementation. Response latency and compute load are reported on an Apple M3 (Kudlur et al., 12 Feb 2026).

The benchmark suite includes LibriSpeech clean, LibriSpeech other, TED-LIUM, VoxPopuli, AMI, Earnings-22, GigaSpeech, and SPGISpeech. This benchmark set overlaps with the corpus context around the original Moonshine family, which was positioned relative to standard ASR datasets such as LibriSpeech, Common Voice, AMI, MLS, GigaSpeech, and People’s Speech (Jeffries et al., 2024, Kudlur et al., 12 Feb 2026).

5. Accuracy, latency, and parameter-efficiency results

Moonshine v2 reports average word error rates of 12.01% for Tiny, 7.84% for Small, and 6.65% for Medium across the standard benchmarks used in the paper (Kudlur et al., 12 Feb 2026). Dataset-level results are reported as follows: LibriSpeech clean 4.49 / 2.49 / 2.08, LibriSpeech other 12.09 / 6.78 / 5.00, TED-LIUM 6.12 / 3.77 / 2.99, VoxPopuli 14.02 / 9.98 / 8.54, AMI 19.03 / 12.54 / 10.68, Earnings-22 20.27 / 13.53 / 11.90, GigaSpeech 13.90 / 10.41 / 9.46, and SPGISpeech 6.16 / 3.19 / 2.58 for Tiny, Small, and Medium respectively (Kudlur et al., 12 Feb 2026).

The paper states that Moonshine v2 achieves state of the art word error rates across standard benchmarks and attains accuracy on-par with models 6x their size while running significantly faster (Kudlur et al., 12 Feb 2026). It further states that the model family lies on a strong accuracy–parameter Pareto frontier, especially in the small-model regime targeted at edge devices (Kudlur et al., 12 Feb 2026).

Latency results are central to the paper’s claims. On the Apple M3, measured real-time response latency between end of speech and transcript return is reported as 27 ms for Moonshine Tiny, 44 ms for Moonshine Base, 50 ms for Moonshine v2 Tiny, 148 ms for Moonshine v2 Small, and 258 ms for Moonshine v2 Medium. Corresponding Whisper baselines are 289 ms for Whisper Tiny, 553 ms for Whisper Base, 1940 ms for Whisper Small, and 11286 ms for Whisper Large v3 (Kudlur et al., 12 Feb 2026).

These measurements support the reported speedups that Moonshine v2 Tiny is 5.8× faster than Whisper Tiny, Moonshine v2 Small is 13.1× faster than Whisper Small, and Moonshine v2 Medium is 43.7× faster than Whisper Large v3 in the live-transcription setting tested on the Apple M3 (Kudlur et al., 12 Feb 2026). The paper also reports lower compute load for Moonshine v2 than Whisper on the same hardware (Kudlur et al., 12 Feb 2026).

A distinct comparison concerns the original Moonshine line. Moonshine Tiny in the original paper achieved a 5× reduction in compute relative to Whisper tiny.en for transcribing a 10-second speech segment, while showing no increase in WER on the standard evaluation datasets used there (Jeffries et al., 2024). Moonshine v2 extends that efficiency narrative by explicitly converting time-to-first-token from a quantity that grows with input duration into one that is approximately constant (Kudlur et al., 12 Feb 2026).

6. Trade-offs, limitations, and deployment significance

The principal practical implication of Moonshine v2 is that high-quality ASR becomes more viable for interactive speech interfaces on resource-constrained devices. The paper explicitly situates the family in live transcription, voice commands, real-time translation, accessibility captioning, and offline or privacy-sensitive on-device speech interfaces (Kudlur et al., 12 Feb 2026). In the original Moonshine framing, the same deployment logic was tied to live transcription, voice commands, and edge, mobile, or embedded scenarios where compute and latency matter (Jeffries et al., 2024).

The architectural trade-offs are also explicit. First, the decoder remains autoregressive, so full transcript generation is still token-by-token even though time-to-first-token is substantially improved (Kudlur et al., 12 Feb 2026). Second, Moonshine v2 is not purely causal everywhere: the first and last encoder layers use a small lookahead, which helps accuracy but adds bounded algorithmic latency (Kudlur et al., 12 Feb 2026). Third, provisional states may change, so the newest emitted text can be revised as more audio arrives (Kudlur et al., 12 Feb 2026). Fourth, the reported work is English-only, although the paper states that the architecture should generalize to other languages (Kudlur et al., 12 Feb 2026). Fifth, the frontend was chosen pragmatically rather than through a full ablation study (Kudlur et al., 12 Feb 2026).

The paper identifies several future directions through contrast. It notes that more streaming-friendly alternatives such as CTC, RNN-T, or TDT could further reduce end-to-end delay, but leaves this for future work (Kudlur et al., 12 Feb 2026). This suggests that the current Moonshine v2 design is a streaming encoder retrofit within an otherwise conventional sequence-to-sequence decoding framework, rather than a fully redesigned streaming recognizer.

Within the Moonshine family as a whole, the distinguishing characteristic of v2 is therefore the replacement of the original full-attention encoder with an ergodic sliding-window encoder. The original Moonshine contribution centered on variable-length training without zero-padding and RoPE-based encoder-decoder efficiency (Jeffries et al., 2024). Moonshine v2 reorients that family toward bounded streaming latency, fixed encoding delay, and strong edge-device responsiveness while preserving competitive word error rates (Kudlur et al., 12 Feb 2026).