FLASH-BS Viterbi: Dynamic Beam Search Decoding

• The paper introduces FLASH-BS Viterbi, a dynamic beam search variant that decouples memory usage from the full hidden state space by retaining only the top-B candidate paths.
• It employs a heap-based candidate maintenance structure integrated with non-recursive divide-and-conquer and parallel pruning techniques, leading to substantial time and memory efficiency improvements.
• Empirical evaluations report up to an 18.3× speedup and dramatic BRAM savings with minimal decoding error when using appropriately tuned beam widths.

FLASH-BS Viterbi is the dynamic beam search variant of “FLASH Viterbi: Fast and Adaptive Viterbi Decoding for Modern Data Systems” (Deng et al., 22 Oct 2025). It is designed to make Viterbi decoding more space-efficient and more adaptable by keeping only the top-B candidate paths during decoding, rather than maintaining the full K-state dynamic-programming table. Within the FLASH framework, it combines beam search with a memory-efficient data structure, the same non-recursive divide-and-conquer strategy used by FLASH Viterbi, and pruning and parallelization techniques. The paper positions it as the variant that “further decouple[s] space complexity from the hidden state space size,” with explicit attention to resource-constrained data systems and FPGA deployment (Deng et al., 22 Oct 2025).

1. Formal setting and departure from standard Viterbi

The baseline used for comparison is the standard Viterbi recurrence over K hidden states and T timesteps. For each timestep t and state j, it computes the best path probability and predecessor as

$$\delta_t(j) = \max_{1 \leq i \leq K} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right] \cdot \mathcal{B}_j(x_t),$$

$$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$

Backtracking then proceeds through

$$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$

In that formulation, the standard decoder requires storing all states across all timesteps, with space complexity $\mathcal{O}(KT)$ and time complexity $\mathcal{O}(K^2T)$ (Deng et al., 22 Oct 2025). FLASH-BS Viterbi changes this organization at the candidate-set level. It does not keep all $K$ states; instead, it maintains only the top B candidates at each timestep, performs dynamic pruning while probabilities are computed, and stores only the traceback-related information needed for those B candidates.

The central practical consequence is that standard Viterbi’s memory grows with the state-space size K, whereas FLASH-BS keeps memory proportional to B, which is user-chosen. The method is therefore best understood not as a change to the Viterbi objective itself, but as a beam-limited execution model for the same structured decoding task.

2. Place within the FLASH framework

FLASH-BS Viterbi is not an isolated beam-search decoder; it is a specialized variant inside the broader FLASH Viterbi framework (Deng et al., 22 Oct 2025). The main FLASH method combines four components: non-recursive divide-and-conquer task management, pruning and parallelization to remove redundant work and subtask dependencies, a double-buffered memory scheme, and configurable parallelism degree $P$.

For the main FLASH Viterbi method, memory reduction comes from storing only division-point information rather than the full traceback table. The paper states that FLASH Viterbi stores only the transitions “between the division point timestep and the terminal timestep,” using three arrays: OptProb, PreState, and MidState. This reduces space from $\mathcal{O}(KT)$ to $\mathcal{O}(K)$.

FLASH-BS extends that design in two specific ways. First, it restricts the live candidate set by tracking only the top B paths rather than all K states. Second, it introduces a heap-based candidate-maintenance structure for efficient top-B updates. The paper summarizes the method as:

FLASH Viterbi + dynamic beam search + heap-based top-B maintenance

Relative to the main FLASH decoder, FLASH-BS therefore reduces memory further not only by restructuring traceback, but also by shrinking the active hypothesis set itself. This is the sense in which it is presented as the more aggressively memory-saving and more adaptive FLASH variant.

3. Dynamic beam search and heap-based candidate maintenance

The distinguishing algorithmic feature of FLASH-BS is its use of dynamic beam search during probability computation, rather than a static beam-search stage applied after computing all K candidates (Deng et al., 22 Oct 2025). The paper contrasts the two explicitly: static beam search computes and stores all K candidates first and then prunes to B, while dynamic beam search incrementally maintains only the top-B states during the process of calculating path probabilities. As stated in the paper, this “eliminates the need to store full intermediate results for the remaining K-B states.”

The data structure supporting this update pattern is a pair of min-heaps:

• heap_pre: stores the top B candidates from the previous timestep
• heap_total: incrementally builds the top B candidates for the current timestep

Each heap stores three fields:

1. State: the state indices of the top candidates
2. OptProb: the corresponding path probabilities, used as the heap key
3. MidState: the division-point state index along maximum paths, aligned with FLASH Viterbi’s traceback compression

To avoid copying overhead, the implementation allocates two physically separate heap buffers and uses them in a double-buffering manner, alternating their roles between heap_pre and heap_total.

The heap update rule is also specified directly. During decoding, for each candidate transition:

1. if fewer than B-1 elements exist in heap_total, insert directly
2. if exactly B-1 exist, insert and heapify
3. if B elements already exist, compare the new candidate against the heap root and insert only if the new candidate’s OptProb is larger

At the subtask level, FLASH-BS retains FLASH’s divide-and-conquer structure. For a subtask starting at a division point, the update can be simplified to

$$OptProb[i] = \log(\mathcal{A}_{q_{m-1}^*,i}) + \log(\mathcal{B}_{i,x_m}),$$

which the paper highlights because the constant predecessor probability term is removed, enabling independent subtask computation (Deng et al., 22 Oct 2025).

This organization makes FLASH-BS a beam-limited, heap-maintained realization of the FLASH execution model rather than a conventional token-passing beam search.

4. Complexity, traceback compression, and correctness claims

In the parallel setting with parallelism degree $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$0 and beam width $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$1, the paper gives FLASH-BS Viterbi time complexity as

$$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$2

and space complexity as

$$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$3

The overview also describes the variant as reducing space to $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$4 and decoupling it from K (Deng et al., 22 Oct 2025). The same source further characterizes runtime and memory usage as effectively scaling with $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$5 relative to full-state methods.

Traceback compression remains inherited from FLASH. The non-recursive task model uses tuples $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$6, with a queue controlling subtask execution, and MidState stores the division-point state index along maximum paths. This preserves the FLASH strategy of reconstructing path segments without materializing the full $$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$7 dynamic-programming table.

The formal correctness statements reported in the paper concern FLASH pruning rather than beam search specifically, but they are presented as theorems that underpin the framework inherited by FLASH-BS. The claims are:

• the optimal path is in the retained-pruning set:

$$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$8

• the optimal path under retained-pruning equals the divide-and-conquer optimum:

$$\psi_t(j) = \underset{1 \leq i \leq K}{\arg\max} \left[ \delta_{t-1}(i) \cdot \mathcal{A}_{ij} \right].$$9

• the optimal path under complete-pruning also equals the divide-and-conquer optimum:

$$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$0

These theorems are not beam-search-specific, and the paper states that FLASH-BS inherits them through the FLASH framework (Deng et al., 22 Oct 2025). A plausible implication is that the beam component should be viewed primarily as the tunable approximation mechanism, while pruning and divide-and-conquer provide the structural guarantees internal to FLASH.

The adaptive control variables are B and P. The paper states that the beam width can be changed to trade accuracy for resource usage, while P adjusts latency and memory trade-offs. This makes adaptivity an explicit design parameter rather than a byproduct of implementation.

5. Empirical behavior and accuracy–efficiency trade-offs

The paper reports that FLASH-BS outperforms both SIEVE-BS and SIEVE-BS-Mp in decoding time and memory efficiency (Deng et al., 22 Oct 2025). Under sequential execution, against the C implementations of those baselines, FLASH-BS achieves a 3.5× speedup over SIEVE-BS and 2.7× over SIEVE-BS-Mp, with memory reduction of 901.5× over SIEVE-BS and 778.4× over SIEVE-BS-Mp. Against the original Python versions, the reported speedups are 14.7× and 12.9×, with memory reductions of 7129.4× and 6470.4×.

At parallelism degree 16, the reported speedup reaches 18.3× versus SIEVE-BS and 14.1× versus SIEVE-BS-Mp, while memory reductions are 58.2× and 50.3×. The paper attributes these results to four factors: non-recursive divide-and-conquer, double-buffered memory, pruning-parallelization integration, and dynamic beam search’s efficient top-B storage.

A particularly important ablation studies the beam-width parameter on a forced-alignment dataset with $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$1 and $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$2, varying $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$3 from 1024 down to 32. The findings are explicit. When $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$4, relative decoding error stays below 0.05%; when $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$5, error rises to 0.39%. The paper explains this by noting that smaller beams can exclude critical states when transition probabilities are dense or relatively uniform. In the tested speech-recognition setting, FLASH-BS achieved 69.5× speedup with only 0.05% increase in error.

The same experiments state that runtime and memory usage scale proportionally with beam width, effectively to about

$$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$6

of the full-state values. This makes the beam width the principal operating knob for accuracy versus efficiency.

The paper also reports that FLASH variants, including FLASH-BS, are less sensitive to changes in transition density than SIEVE-based beam-search methods. The stated reason is architectural: FLASH uses a standard state-matrix-based approach, whereas SIEVE-BS and SIEVE-BS-Mp use token-passing, which can become costly as density increases. This suggests that FLASH-BS’s efficiency is not tied solely to aggressive pruning; it also depends on the representation of the transition structure.

6. FPGA realization and broader algorithmic context

FLASH-BS Viterbi has a dedicated FPGA accelerator in the paper’s hardware study (Deng et al., 22 Oct 2025). The accelerator includes a TASK QUEUE, a DDR CONTROLLER, an OB_CACHE, a FINDMAX module, storage units for intermediate results, and output writing to OUTPUT_PATH. The beam-search-specific hardware consists of two BRAM-based min-heap structures, HEAP_1 and HEAP_2, which alternate roles as Heap_total and Heap_pre, with search and update operations controlled by a HEAP_OPERATION module.

The paper notes that beam search causes more irregular memory access, so FLASH-BS hardware is somewhat slower than FLASH Viterbi hardware. It nevertheless reports the design as practical and comparable to software performance. Relative to Reduced-Memory Viterbi, FLASH-BS uses substantially less BRAM at beam widths 32K and 512. At beam width 32K, the entire decoder uses only 26.0% of RM-Viterbi’s BRAM, the decoding unit uses only 22.9% of RM-Viterbi’s BRAM, and power is about 15% lower. At beam width 512, it also remains much smaller in BRAM and power.

Within the broader Viterbi literature, FLASH-BS occupies a specific position. General worst-case acceleration of Viterbi is conditionally difficult: the standard dynamic program runs in $$q_{T-1}^* = \underset{1 \leq j \leq K}{\arg\max}\ \delta_{T-1}(j), \quad q_t^* = \psi_{t+1}(q_{t+1}^*) \quad \text{for } \quad (t={T-2},...,0).$$7, and conditional lower bounds argue that this runtime is optimal up to subpolynomial factors in general settings (Backurs et al., 2016). Other accelerations exploit different structure. The temporally abstracted Viterbi algorithm reasons over intervals of time and reports several orders of magnitude speedup over standard Viterbi, as well as significant speedups over CFDP, for problems whose state variables evolve at widely differing rates (Chatterjee et al., 2012). Marginalized beam search methods for HHMMs instead target the outer-state sequence and approximate a marginalized objective that standard Viterbi does not optimize directly (Xu et al., 2023).

FLASH-BS differs from those lines of work in its design target. It remains a beam-search-enabled variant inside a divide-and-conquer Viterbi framework, and its reported contributions center on memory decoupling from the full hidden-state space, dynamic tuning of beam width and parallelism, and hardware-friendly realization for edge deployment. This suggests that FLASH-BS should be understood primarily as a systems-oriented Viterbi operator: adaptive in resource use, explicit in its approximation knob, and engineered around heap-based top-B maintenance rather than temporal abstraction or alternative decoding objectives.