---
title: 'Moonshine v2: Efficient Streaming ASR'
url: https://www.emergentmind.com/topics/moonshine-v2
type: topic
---

# Moonshine v2: Efficient Streaming ASR

Moonshine v2 is a family of streaming automatic speech recognition models for latency-critical edge applications such as live transcription, voice commands, and real-time translation. It is introduced as an ergodic streaming-encoder ASR architecture that replaces the full-attention Transformer encoder used in the original Moonshine family with sliding-window self-attention, with the explicit goal of reducing time-to-first-token and overall interactive delay while preserving strong transcription accuracy [2602.12241]. In the broader Moonshine lineage, the original Moonshine models were optimized for live transcription and voice command processing through an encoder-decoder Transformer architecture, Rotary Position Embedding, and training on variable-length speech segments without zero-padding; Moonshine Tiny was reported to achieve a 5× reduction in compute versus Whisper `tiny.en` on a 10-second speech segment with no increase in word error rate on the standard evaluation datasets used in the paper [2410.15608].

## 1. Historical placement within the Moonshine model family

The original Moonshine paper presents a family of ASR models optimized for live transcription and voice command processing, with the central efficiency claim that fixed-length, zero-padded inputs make many ASR encoders inefficient at inference. Moonshine addresses this by training on variable-length speech segments without zero-padding, so the encoder does not waste computation on padding tokens at inference. In that formulation, Moonshine is an encoder-decoder Transformer using Rotary Position Embedding instead of traditional absolute position embeddings [2410.15608].

Moonshine v2 retains the live-ASR orientation but shifts the primary optimization target from general encoder efficiency to bounded streaming latency. The motivating diagnosis is that full-attention Transformer encoders remain a strong accuracy baseline because every frame can directly attend to every other frame, but this induces an “encode-the-whole-utterance” latency profile. For streaming use cases, the consequence is that time-to-first-token grows roughly linearly with utterance duration because the encoder must process the entire prefix before any decoder token can be emitted [2602.12241].

This places Moonshine v2 as a second-generation system within the same application domain rather than a wholesale change of task. A plausible implication is that the transition from Moonshine to Moonshine v2 reflects a shift from compute-efficient short-segment inference toward architecture-level control of streaming latency. That interpretation is consistent with the explicit contrast between the original full-attention encoder and the Moonshine v2 sliding-window encoder [2410.15608] [2602.12241].

## 2. Latency model and the motivation for the streaming redesign

Moonshine v2 is introduced for latency-critical speech applications where low time-to-first-token and high transcription accuracy are both required, particularly on resource-constrained edge devices. The paper explicitly frames **250 ms** as a practical one-way latency upper bound for interactive voice, which makes encoder behavior central to system design [2602.12241].

In the full-attention encoder regime, attention mixing costs scale as $\mathcal{O}(T^2)$ for a sequence of $T$ frames. The paper gives an illustrative compute estimate
$$
\mathrm{ops}_{\text{total}(N)} = 6PT + 4dLT^2,
$$
where $P$ is model parameters, $d$ is hidden dimension, $L$ is number of layers, and $T=50N$ frames for $N$ seconds of audio at 50 Hz. The quadratic term arises from full self-attention, and this produces the unfavorable latency profile for streaming ASR [2602.12241].

Moonshine v2 replaces this with sliding-window self-attention, where each frame attends only to a fixed local neighborhood. With window size $w$, the attention mixing cost becomes $\mathcal{O}(Tw)$, and the corresponding compute estimate is
$$
\mathrm{ops}_{\text{total}(N)} = 6PT + 4dLTw.
$$
For fixed $w$, the attention term is linear in sequence length, and the resulting time-to-first-token becomes bounded and largely independent of utterance length [2602.12241].

The significance of this change is not merely asymptotic. The paper’s conceptual timeline emphasizes that encoding overlaps with audio capture, so much of the work is done incrementally rather than after the utterance ends. This suggests that Moonshine v2 is designed around interactive response constraints rather than offline throughput alone [2602.12241].

## 3. Architecture and the meaning of the “ergodic streaming encoder”

Moonshine v2 is built from four components: an audio preprocessor, an ergodic streaming encoder, an adapter, and an autoregressive decoder [2602.12241].

The audio preprocessor is intentionally lightweight and causal. It segments raw audio into non-overlapping 80-sample windows at 16 kHz, applies cepstral mean and variance normalization, passes values through an $\operatorname{asinh}$ nonlinearity, and then uses two causal stride-2 convolutions to reduce the frame rate by about $4\times$. The output is about 50 feature frames per second, matching Whisper’s feature rate [2602.12241].

The encoder is a Transformer stack with sliding-window self-attention and no positional embeddings at all. Because the encoder has neither absolute nor relative positions, its computation is translation-invariant in time. The paper calls this an ergodic encoder: the same local function is applied everywhere, and structure is inferred from local content rather than explicit time indices [2602.12241].

The attention window is denoted $(w_{\text{left}}, w_{\text{right}})$. Moonshine v2 uses $(16,4)$ in the first two and last two encoder layers, and $(16,0)$ in all intermediate layers. Since features are at 50 Hz, one frame corresponds to 20 ms, so a right-context of four frames corresponds to 80 ms of future context. The paper also notes a conservative bound of 16 frames of extra audio for finalized states in steady state, corresponding to 320 ms. As a result, provisional outputs may be emitted and then refined as future audio arrives [2602.12241].

The adapter restores position awareness where it is needed: it adds a learned positional embedding to encoder outputs and can optionally project features to match decoder dimensionality. This preserves the position-free character of the encoder while allowing a standard sequence-to-sequence decoder interface [2602.12241].

The decoder is a conventional causal Transformer using rotary positional embeddings and cross-attending to encoder features. This decoder choice preserves the autoregressive text-generation paradigm used in the original Moonshine family, but it also imposes an important systems constraint: even when encoder latency is bounded, transcript generation remains serial and token-by-token [2410.15608] [2602.12241].

## 4. Training configuration, model scales, and evaluation protocol

Moonshine v2 is trained on roughly **300K hours** of speech data, consisting of about **200K hours** from the original Moonshine setup plus an additional **100K hours** of internally prepared speech data [2602.12241]. The training recipe uses the same tokenizer as the original Moonshine, the **Schedule-Free** optimizer, an initial learning rate of $2\times 10^{-3}$, **400K steps**, an effective batch size of **512**, and training on **8 NVIDIA H100 GPUs** [2602.12241].

Three model sizes are reported:

| Model | Parameters |
|---|---:|
| Tiny | 33.57M |
| Small | 123.36M |
| Medium | 244.93M |

The paper states that the encoder is relatively compact, while the decoder is larger because it includes cross-attention and SwiGLU feed-forward blocks [2602.12241].

Evaluation separates accuracy and latency tooling. Accuracy is measured with the Hugging Face **Transformers** implementation, while latency is measured with a C++/ONNX implementation. Response latency and compute load are reported on an **Apple M3** [2602.12241].

The benchmark suite includes LibriSpeech clean, LibriSpeech other, TED-LIUM, VoxPopuli, AMI, Earnings-22, GigaSpeech, and SPGISpeech. This benchmark set overlaps with the corpus context around the original Moonshine family, which was positioned relative to standard ASR datasets such as LibriSpeech, Common Voice, AMI, MLS, GigaSpeech, and People’s Speech [2410.15608] [2602.12241].

## 5. Accuracy, latency, and parameter-efficiency results

Moonshine v2 reports average word error rates of **12.01%** for Tiny, **7.84%** for Small, and **6.65%** for Medium across the standard benchmarks used in the paper [2602.12241]. Dataset-level results are reported as follows: LibriSpeech clean **4.49 / 2.49 / 2.08**, LibriSpeech other **12.09 / 6.78 / 5.00**, TED-LIUM **6.12 / 3.77 / 2.99**, VoxPopuli **14.02 / 9.98 / 8.54**, AMI **19.03 / 12.54 / 10.68**, Earnings-22 **20.27 / 13.53 / 11.90**, GigaSpeech **13.90 / 10.41 / 9.46**, and SPGISpeech **6.16 / 3.19 / 2.58** for Tiny, Small, and Medium respectively [2602.12241].

The paper states that Moonshine v2 achieves state of the art word error rates across standard benchmarks and attains accuracy on-par with models **6x their size** while running significantly faster [2602.12241]. It further states that the model family lies on a strong accuracy–parameter Pareto frontier, especially in the small-model regime targeted at edge devices [2602.12241].

Latency results are central to the paper’s claims. On the Apple M3, measured real-time response latency between end of speech and transcript return is reported as **27 ms** for Moonshine Tiny, **44 ms** for Moonshine Base, **50 ms** for Moonshine v2 Tiny, **148 ms** for Moonshine v2 Small, and **258 ms** for Moonshine v2 Medium. Corresponding Whisper baselines are **289 ms** for Whisper Tiny, **553 ms** for Whisper Base, **1940 ms** for Whisper Small, and **11286 ms** for Whisper Large v3 [2602.12241].

These measurements support the reported speedups that Moonshine v2 Tiny is **5.8× faster** than Whisper Tiny, Moonshine v2 Small is **13.1× faster** than Whisper Small, and Moonshine v2 Medium is **43.7× faster** than Whisper Large v3 in the live-transcription setting tested on the Apple M3 [2602.12241]. The paper also reports lower compute load for Moonshine v2 than Whisper on the same hardware [2602.12241].

A distinct comparison concerns the original Moonshine line. Moonshine Tiny in the original paper achieved a **5× reduction in compute** relative to Whisper `tiny.en` for transcribing a **10-second speech segment**, while showing **no increase in WER** on the standard evaluation datasets used there [2410.15608]. Moonshine v2 extends that efficiency narrative by explicitly converting time-to-first-token from a quantity that grows with input duration into one that is approximately constant [2602.12241].

## 6. Trade-offs, limitations, and deployment significance

The principal practical implication of Moonshine v2 is that high-quality ASR becomes more viable for interactive speech interfaces on resource-constrained devices. The paper explicitly situates the family in live transcription, voice commands, real-time translation, accessibility captioning, and offline or privacy-sensitive on-device speech interfaces [2602.12241]. In the original Moonshine framing, the same deployment logic was tied to live transcription, voice commands, and edge, mobile, or embedded scenarios where compute and latency matter [2410.15608].

The architectural trade-offs are also explicit. First, the decoder remains autoregressive, so full transcript generation is still token-by-token even though time-to-first-token is substantially improved [2602.12241]. Second, Moonshine v2 is not purely causal everywhere: the first and last encoder layers use a small lookahead, which helps accuracy but adds bounded algorithmic latency [2602.12241]. Third, provisional states may change, so the newest emitted text can be revised as more audio arrives [2602.12241]. Fourth, the reported work is English-only, although the paper states that the architecture should generalize to other languages [2602.12241]. Fifth, the frontend was chosen pragmatically rather than through a full ablation study [2602.12241].

The paper identifies several future directions through contrast. It notes that more streaming-friendly alternatives such as **CTC**, **RNN-T**, or **TDT** could further reduce end-to-end delay, but leaves this for future work [2602.12241]. This suggests that the current Moonshine v2 design is a streaming encoder retrofit within an otherwise conventional sequence-to-sequence decoding framework, rather than a fully redesigned streaming recognizer.

Within the Moonshine family as a whole, the distinguishing characteristic of v2 is therefore the replacement of the original full-attention encoder with an ergodic sliding-window encoder. The original Moonshine contribution centered on variable-length training without zero-padding and RoPE-based encoder-decoder efficiency [2410.15608]. Moonshine v2 reorients that family toward bounded streaming latency, fixed encoding delay, and strong edge-device responsiveness while preserving competitive word error rates [2602.12241].

Source: https://www.emergentmind.com/topics/moonshine-v2