FLASH Viterbi: Fast, Memory-Efficient Decoding

• FLASH Viterbi is a reformulation of Viterbi decoding that uses non-recursive, parallelizable divide-and-conquer methods to enable memory-efficient processing on resource-constrained systems.
• It splits the approach into an exact FLASH Viterbi decoder and an approximate FLASH-BS variant that leverages dynamic beam search to balance accuracy with reduced memory usage.
• Experimental results show FLASH algorithms achieve significant speedups and memory reductions, such as up to 50.5× FPGA speedup and 69.5× speedup in FLASH-BS with minimal loss in accuracy.

FLASH Viterbi is a reformulation of Viterbi decoding for sequence inference that is explicitly designed for modern data systems, especially edge or resource-constrained deployments where the classical algorithm is memory-intensive and computationally inflexible. The method is presented for Hidden Markov Models, with motivation that also extends to chain-structured models such as CRFs, and it is split into two algorithms: FLASH Viterbi, an exact decoder based on non-recursive divide-and-conquer with pruning and parallelization, and FLASH-BS Viterbi, a dynamic beam-search variant built on a memory-efficient data structure. The stated goals are fast decoding, low memory, adaptability to deployment constraints, and hardware-friendly execution, with software and FPGA implementations reported on synthetic HMMs and speech forced alignment (Deng et al., 22 Oct 2025).

1. Standard formulation and stated design objectives

Given an HMM $\lambda=(\pi,\mathcal{A},\mathcal{B})$, an observation sequence $X=\{x_0,\dots,x_{T-1}\}$, a hidden state set $S=\{s_1,\dots,s_K\}$, and a latent sequence $Q=\{q_0,\dots,q_{T-1}\}$, the decoding objective is

$Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$

The standard Viterbi recursion is written with forward score $\delta_t(j)$ and predecessor pointer $\psi_t(j)$ as

$$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$

and for $t=1,\dots,T-1$,

$$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$

$X=\{x_0,\dots,x_{T-1}\}$0

The final state and traceback are

$X=\{x_0,\dots,x_{T-1}\}$1

The paper states that this classical decoder has time complexity $X=\{x_0,\dots,x_{T-1}\}$2 and space complexity $X=\{x_0,\dots,x_{T-1}\}$3. The time arises from considering all $X=\{x_0,\dots,x_{T-1}\}$4 state transitions per timestep, and the memory from storing dynamic-programming values and backpointers for all $X=\{x_0,\dots,x_{T-1}\}$5 states across all $X=\{x_0,\dots,x_{T-1}\}$6 timesteps. This is the regime that FLASH Viterbi targets: long sequences and large state spaces on systems with limited on-chip memory or limited deployment flexibility. Existing memory-saving methods such as Checkpoint Viterbi and SIEVE/SIEVE-Mp are described as reducing space but often introducing substantial runtime overhead through recomputation, recursion, or graph traversal patterns that are less convenient for parallel or hardware execution (Deng et al., 22 Oct 2025).

2. Non-recursive divide-and-conquer organization

FLASH Viterbi preserves the divide-and-conquer idea of linear-memory Viterbi decoding, but replaces recursive control flow with an explicit task-queue schedule. Each decoding subproblem is represented by a segment tuple $X=\{x_0,\dots,x_{T-1}\}$7, written in the text as $X=\{x_0,\dots,x_{T-1}\}$8, and the original problem starts from $X=\{x_0,\dots,x_{T-1}\}$9. Parent subtasks must be decoded before their children because children need the optimal state identified at the parent’s division point, but subtasks within the same layer have no generation dependency and can therefore be executed in arbitrary order or in parallel (Deng et al., 22 Oct 2025).

The paper states that FLASH performs an initial $S=\{s_1,\dots,s_K\}$0-way partition rather than pure binary bisection, so that a requested parallelism degree $S=\{s_1,\dots,s_K\}$1 can be exploited immediately. After that, threads repeatedly dequeue segments, compute the midpoint

$S=\{s_1,\dots,s_K\}$2

decode the subtask, and enqueue child segments if the segment is still long enough. The architectural significance is that queue-driven execution separates generation order from execution order, making same-layer batching explicit rather than leaving it implicit inside a recursive call stack.

Within a subtask, FLASH does not store the full backpointer table. Instead it uses three arrays: OptProb, containing the best path probability ending at each current state; PreState, containing the previous-state argmax for each state at the current step; and MidState, containing the state at the division-point timestep that lies on the current best path to each current state. The forward dynamic program advances over timesteps $S=\{s_1,\dots,s_K\}$3. Before the midpoint, only scores are propagated. Starting at $S=\{s_1,\dots,s_K\}$4, midpoint identities are propagated by

• $S=\{s_1,\dots,s_K\}$5 when $S=\{s_1,\dots,s_K\}$6,
• $S=\{s_1,\dots,s_K\}$7 when $S=\{s_1,\dots,s_K\}$8.

At the segment endpoint, the midpoint state on the optimal segment path is recovered from the best terminal state’s stored MidState. Hierarchical reconstruction of these midpoint states over all queued segments yields the full path. In the sequential case, this reduces storage from $S=\{s_1,\dots,s_K\}$9 to $Q=\{q_0,\dots,q_{T-1}\}$0 because each active subtask maintains only $Q=\{q_0,\dots,q_{T-1}\}$1 state rather than a full $Q=\{q_0,\dots,q_{T-1}\}$2time backpointer slab (Deng et al., 22 Oct 2025).

3. Pruning, exactness, and asymptotic properties

A naive divide-and-conquer decoder still leaves computational dependencies between sibling subtasks, because the right child would ordinarily depend on dynamic-programming scores at the split boundary. FLASH removes that dependency by pruning all child paths except those consistent with the optimal parent boundary state. Once the parent subtask has identified the optimal boundary state $Q=\{q_0,\dots,q_{T-1}\}$3, a child segment beginning at $Q=\{q_0,\dots,q_{T-1}\}$4 is initialized in log space by

$Q=\{q_0,\dots,q_{T-1}\}$5

This removes dependence on the full predecessor score vector and retains only transitions from the optimal boundary state (Deng et al., 22 Oct 2025).

The paper formalizes correctness through three sets: the full valid path set $Q=\{q_0,\dots,q_{T-1}\}$6, a retained-pruning subset $Q=\{q_0,\dots,q_{T-1}\}$7 that keeps only paths whose state at time $Q=\{q_0,\dots,q_{T-1}\}$8 matches $Q=\{q_0,\dots,q_{T-1}\}$9, and a complete-pruning variant $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$0 that additionally normalizes the start probability to $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$1. The three exactness statements are:

1. $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$2,
2. the optimum inside $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$3 is the original optimum, $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$4,
3. the complete-pruning optimum remains unchanged, $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$5.

The third result is justified by the fact that the retained-pruning and complete-pruning initializations differ only by an additive constant in log space: $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$6

$Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$7

and $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$8 does not affect the $Q^*=\underset{Q}{\arg\max}\, P(Q\mid X,\lambda).$9.

The resulting complexity statements make the exact-versus-adaptive tradeoff explicit. Standard Viterbi is $\delta_t(j)$0 time and $\delta_t(j)$1 space. Checkpoint Viterbi is reported as $\delta_t(j)$2 space. SIEVE-Mp has $\delta_t(j)$3 space and $\delta_t(j)$4 time. FLASH Viterbi remains in the divide-and-conquer family but exposes parallelism degree $\delta_t(j)$5, with stated time complexity

$\delta_t(j)$6

and space complexity

$\delta_t(j)$7

The paper also gives an idealized cycle count interpretation: sequential divide-and-conquer requires $\delta_t(j)$8 decoding cycles, whereas the multithreaded version with degree $\delta_t(j)$9 requires

$\psi_t(j)$0

This is the formal basis for the claim that FLASH Viterbi is adaptive: increasing $\psi_t(j)$1 decreases latency while increasing memory linearly in the number of workers (Deng et al., 22 Oct 2025).

4. FLASH-BS Viterbi: dynamic beam search and heap-based memory reduction

FLASH-BS Viterbi is the approximate variant intended for scenarios in which even $\psi_t(j)$2 memory is too large. The paper contrasts it with static beam-search variants that still compute scores for all $\psi_t(j)$3 states before retaining only the best $\psi_t(j)$4. FLASH-BS instead maintains only the current top-$\psi_t(j)$5 active states while scores are being computed, so discarded states are never materialized in the main state data structure (Deng et al., 22 Oct 2025).

The implementation uses two min-heaps, heap_pre and heap_total, which store the active beam candidates for the previous and current timestep. Each heap entry contains exactly three fields: State, OptProb, and MidState. The root contains the worst candidate currently in the beam. If the heap has fewer than $\psi_t(j)$6 entries, a new candidate is inserted directly; once full, a candidate replaces the root only if it is better. Two separate buffers realize a double-buffering scheme, so the roles of “previous” and “current” heap swap between timesteps without bulk copying.

The variant is explicitly approximate. On the speech forced-alignment task, the relative log-likelihood error is defined as

$\psi_t(j)$7

where $\psi_t(j)$8 is the optimal-path log-likelihood and $\psi_t(j)$9 is the beam-search result. The reported empirical tradeoff is that for $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$0, the relative error stays below $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$1, whereas at $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$2 it rises to $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$3.

Its asymptotic complexity is given as

$$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$4

in time and

$$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$5

in space. Relative to exact FLASH Viterbi’s $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$6 memory, FLASH-BS replaces dependence on the full hidden-state size $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$7 by dependence on beam width $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$8. The paper describes this as decoupling space complexity from the hidden state space size. It also states that, on the speech task, FLASH-BS achieves a 69.5× speedup with only a 0.05% increase in error, which is the principal empirical summary of the approximate regime (Deng et al., 22 Oct 2025).

5. Hardware mapping and empirical evaluation

The hardware design follows the same queue-driven and double-buffered structure as the software algorithm. In the FPGA accelerator, HMM data reside in DDR memory, a FIFO-based task queue emits subtask tuples, and a DDR controller fetches observation segments into an on-chip observation cache. A FINDMAX unit performs the dynamic-programming update using HMM data from DDR and intermediate beam state from on-chip storage. In FLASH-BS, two BRAM-based heap memories, HEAP_1 and HEAP_2, alternate the roles of heap_pre and heap_total (Deng et al., 22 Oct 2025).

The paper attributes hardware friendliness to several specific choices: elimination of recursion and BFS, explicit task-queue control, unified subtask initialization after pruning, double buffering, and pipelining with batched DDR fetches. It also notes a remaining tradeoff: beam search introduces irregular memory access and therefore reduces sequential DDR efficiency relative to exact FLASH hardware.

Resource results are reported on a Xilinx XCZU7EV at 200 MHz against Reduced-Memory Viterbi. At beam width 32K, the FLASH-BS entire decoder uses 198.5 BRAM, 3 DSP, and 42,445 LUT+FF, versus 764 BRAM, 30 DSP, and 35,136 LUT+FF for RM-Viterbi, with 1.835 W versus 2.152 W power. At beam width 512, FLASH-BS uses 32.5 BRAM, 3 DSP, and 41,954 LUT+FF, versus 72 BRAM, 30 DSP, and 34,899 LUT+FF, with nearly equal power. The paper highlights that at 32K beam width FLASH-BS uses only 26.0% of RM-Viterbi’s BRAM at the whole-decoder level and 22.9% at the decoding-unit level, while reducing power by about 15%.

The software evaluation uses synthetic Erdős–Rényi HMMs varying edge probability $$\delta_0(j)=\pi_j\cdot \mathcal{B}_j(x_0), \quad \psi_0(j)=0,$$9, state size $t=1,\dots,T-1$0, and sequence length $t=1,\dots,T-1$1, plus a speech forced-alignment dataset built with HTK from TIMIT, with $t=1,\dots,T-1$2 and $t=1,\dots,T-1$3, on a dual 16-core Xeon 6226R. On the forced-alignment benchmark with $t=1,\dots,T-1$4, sequential C FLASH-BS runs in 14.2 s versus 49.4 s for SIEVE-BS and 38.0 s for SIEVE-BS-Mp, with only $t=1,\dots,T-1$5 B memory versus $t=1,\dots,T-1$6 B and $t=1,\dots,T-1$7 B. Under parallelism 16, FLASH-BS reaches 2.7 s. For exact decoding, C Vanilla uses $t=1,\dots,T-1$8 B versus $t=1,\dots,T-1$9 B for sequential FLASH, while sequential FLASH is slower (385.9 s versus 104.4 s); with $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$0, FLASH reaches 93.6 s, and with $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$1, 71.7 s with memory $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$2 B (Deng et al., 22 Oct 2025).

Additional reported trends are that FLASH and FLASH-BS maintain the lowest memory footprint as $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$3 and $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$4 vary because their storage is independent of sequence length, that FLASH empirically outperforms SIEVE-Mp at equal asymptotic complexity because of non-recursive queue execution and double-buffered memory organization, and that FLASH Viterbi’s FPGA speedup over software grows superlinearly with $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$5, reaching 50.5× at $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$6. The paper also reports Raspberry Pi 5 experiments in which multithreaded FLASH variants remain competitive, with FLASH Viterbi$$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$7 matching or outperforming Vanilla and FLASH-BS widening that advantage (Deng et al., 22 Oct 2025).

6. Position within the broader Viterbi literature

Within the supplied literature, FLASH Viterbi occupies a specific point in the design space: it is an exact low-memory divide-and-conquer decoder with explicit parallel scheduling, and FLASH-BS extends that framework to an approximate beam-search regime. This distinguishes it from other accelerated or memory-reduced Viterbi families.

Temporally abstracted Viterbi methods accelerate exact decoding by pruning trajectory sets over long intervals in systems with strong multiscale dynamics, using abstraction-based bounds rather than midpoint-based divide-and-conquer (Chatterjee et al., 2012). On-line Viterbi methods reduce practical memory by outputting prefixes once traceback paths coalesce, often achieving expected memory much smaller than linear in sequence length, but they do not use FLASH’s explicit segment queue, pruning rule, or FPGA-oriented double buffering (0704.0062). GPU-native exact decoders for WFST lattices reorganize token-passing Viterbi around massive accelerator parallelism, memory-bounded queues, and deferred merging, which is a different target domain and a different execution model from FLASH’s HMM divide-and-conquer structure (Braun et al., 2019). Code-specific fast Viterbi methods can also arise from algebraic structure; for example, the improved decoder for $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$8-partial simplex convolutional codes reduces branch-metric computation from $$\delta_t(j)=\max_{1\le i\le K}\left[\delta_{t-1}(i)\cdot \mathcal{A}_{ij}\right]\cdot \mathcal{B}_j(x_t),$$9 to $X=\{x_0,\dots,x_{T-1}\}$00 through fast Hadamard transforms, but that acceleration is tied to branch-label structure rather than queue scheduling or hardware-adaptive divide-and-conquer (Abreu et al., 2024). At the complexity-theoretic level, unrestricted HMM Viterbi is argued to be difficult to speed up polynomially in general, with lower bounds showing optimality up to subpolynomial factors under APSP- and clique-based assumptions (Backurs et al., 2016). This makes the FLASH strategy notable not as a contradiction to those lower bounds, but as a systems-oriented improvement in a constrained algorithmic regime.

The paper’s own practical recommendation is correspondingly bifurcated. FLASH Viterbi is intended for deployments that require exact decoding quality but cannot afford the $X=\{x_0,\dots,x_{T-1}\}$01 memory footprint of classical Viterbi. FLASH-BS Viterbi is intended for more severe memory or latency constraints, where a user is willing to trade exactness for beam-width-controlled approximation. A plausible implication is that the framework is best understood as a hardware-aware reorganization of space-efficient Viterbi decoding, rather than as a universally faster replacement for every HMM or trellis workload (Deng et al., 22 Oct 2025).