---
title: Bidirectional Beam Search (BiBS)
url: https://www.emergentmind.com/topics/bidirectional-beam-search-bibs
type: topic
---

# Bidirectional Beam Search (BiBS)

Bidirectional Beam Search (BiBS) comprises a family of approximate inference algorithms designed for decoding with bidirectional neural sequence models. By leveraging both past and future context during sequence generation, BiBS methods address intrinsic limitations of unidirectional decoding, particularly when applied to challenging tasks such as fill-in-the-blank generation, abstractive summarization, neural response generation, machine translation, and connectionist temporal classification (CTC) decoding. The BiBS framework encompasses variants that differ in how bidirectional information is combined and exploited during inference, but all aim to improve sequence-level coherence and global optimality relative to conventional left-to-right (L2R) or right-to-left (R2L) beam search.

## 1. Motivation and Foundations

Traditional sequence generation in neural models (e.g., RNN, LSTM, GRU, Transformer) employs unidirectional decoding, in which the model generates each token $y_t$ conditioned only on the prefix $y_{1:t-1}$ and, optionally, a representation of the source input. This sequential generation paradigm limits the decoder’s ability to utilize future information, resulting in characteristic pathologies such as overproduction of initial phrases, repetition, and unbalanced coverage of source content—especially pronounced in long output sequences [1809.06662], [1705.08759], [1902.08955]. Bidirectional sequence models, including bidirectional decoders and language models, offer a richer conditional structure by conditioning each output token on both its prefix and suffix (i.e., $p(y_t \mid y_{1:t-1}, y_{t+1:T}, x)$), but they introduce computational intractability for exact inference due to the exponentially large space of possible output sequences.

BiBS algorithms are motivated by the need for tractable, approximate decoding methods that allow bidirectional models to realize their potential for globally coherent sequence generation without prohibitive computational cost [1705.08759]. They achieve this by various means, such as coordinate descent over prefix and suffix beams, synchronous expansion of L2R and R2L hypotheses, or reranking based on bidirectional scoring criteria.

## 2. Formal Decoding Objectives and Key Variants

The general BiBS decoding objective is to approximate $\hat{Y} = \arg\max_{Y} p(Y \mid X)$ under a model that factorizes conditionally on both left and right context. Two principal mathematical formulations are prevalent:

- **Joint Bidirectional Log-Score Objective** [1809.06662]:
  $$
  S_{\text{fw}}(Y) = \sum_{t=1}^T \log p_{\text{fw}}(y_t | y_{1:t-1}, x), \quad
  S_{\text{bw}}(Y) = \sum_{t=1}^T \log p_{\text{bw}}(y_t | y_{t+1:T}, x)
  $$
  $$
  \hat{Y} = \arg\max_Y[\gamma S_{\text{fw}}(Y) + (1-\gamma) S_{\text{bw}}(Y)]
  $$
  where $\gamma \in [0,1]$ balances forward and backward contributions.

- **Bidirectional RNN Product-of-Conditionals** [1705.08759]:
  $$
  p(Y|X) \propto \prod_{t=1}^T p(y_t|y_{1:t-1}, y_{t+1:T}, X)
  $$
  Approximated via coordinate updates that alternate between left-to-right and right-to-left passes, each pass holding fixed the opposite direction’s beams.

In addition, bidirectional strategies can be classified as:
- **Coordinated Bidirectional Beam Search** (standard BiBS; e.g., [1705.08759]): Iterative forward/backward passes update and coordinate prefix/suffix beams to maximize the approximate joint probability.
- **Synchronous Interactive Decoding** [1902.08955]: L2R and R2L hypotheses are expanded in parallel at each time step, with cross-directional state-sharing.
- **Bidirectional Scoring/Aggregation** (BidiS/BidiA) [2110.03389]: Combines the scores of independent forward and backward beam searches via log-probability addition or similarity-based agreement.
- **Bidirectional LM Fusion for CTC** [2110.03326]: Shallow fusion of forward and backward LMs using a noisy (greedy) future estimate during prefix beam search.

## 3. Algorithmic Instantiations

### 3.1 Bidirectional Beam Search for Bidirectional Decoder Models
The quintessential BiBS for fully bidirectional decoders comprises the following sequence [1809.06662]:
1. **Backward Beam Presearch:** Compute a backward beam of $K$ high-scoring suffixes by running beam search from position $T$ to $1$; cache cumulative backward log-probability arrays for these sequences.
2. **Forward Beam with Bidirectional Scoring:** At each decoding step $t$, extend prefixes in the forward beam by a candidate token $y$, and for each, retrieve the best-matching backward beam suffix and sum their prefix and suffix scores—weighted by $\gamma$.
3. **Beam Maintenance:** Retain the top $K$ prefixes at each step according to the joint score. Decoding proceeds until $T_{\max}$ or until EOS is reached in all beams.

### 3.2 Synchronous Interactive Beam Search
L2R and R2L beams of size $K/2$ are expanded synchronously at each time step, with scoring/attention incorporating the opposite direction’s decoder states (via cross-attention in LSTM/Transformer decoders). At each step, candidates are scored by combining context from both sides, then pruned. Decoding continues until sufficient complete hypotheses are gathered [1902.08955].

### 3.3 Coordinate-Descent BiBS
BiBS as originally proposed for bidirectional models [1705.08759] executes alternating left-to-right and right-to-left passes over the current beam set, at each step updating beam hypotheses using cached log-probabilities from both directions. Candidate sequences are built from the Cartesian product of prefix beams, possible next tokens, and suffix beams.

### 3.4 Bidirectional CTC Decoding
For CTC, BiBS employs a greedy CTC decoding output as a noisy estimate of the future label sequence. During prefix beam search, the current forward LM state is fused with the corresponding backward LM state precomputed from the approximated suffix, with the joint (forward + backward) LM score added to the acoustic model’s log-probabilities [2110.03326].

## 4. Computational Complexity and Resource Analysis

The added complexity of BiBS relative to standard beam search arises primarily from candidate set expansion and the maintenance/coordination of multiple beams:
- **Standard Beam Search:** $O(T \cdot B \cdot |V|)$ candidate scoring per pass.
- **Coordinate-Descent BiBS:** $O(M \cdot T \cdot B^2 \cdot |V|)$ for $M$ rounds, due to cross-beam Cartesian product at each time step [1705.08759].
- **BiBS with Suffix Presearch:** $O(2K|V|T_{\max})$, i.e., roughly $2\times$ the cost of unidirectional search [1809.06662].
- **Synchronous Bidirectional Beam Search:** $O(T \cdot K \cdot |V| \cdot d)$ per pass plus up to $O(T^2 K d^2)$ for full cross-attention in the LSTM case [1902.08955].
- **Bidirectional LM Fusion for CTC:** One backward pass over the greedy sequence (cost $O(T_y \cdot C_{\text{LM}})$) plus standard prefix-beam search, yielding a minor wall-clock overhead [2110.03326].

In all cases, space complexity is dominated by the need to store beam hypotheses and associated RNN/Transformer hidden states, typically $O(B \cdot T)$ per direction.

## 5. Empirical Results and Applications

Empirical evaluations across tasks consistently demonstrate that BiBS and related bidirectional inference strategies yield improved sequence-level metrics over standard unidirectional beam search.

- **Fill-in-the-Blank Image Captioning (COCO, Visual Madlibs):** BiBS achieves higher CIDEr, BLEU-4, and METEOR scores compared to unidirectional and naive bidirectional baselines. For instance, on COCO (r=50%, B=5, M=4): BiBS obtained CIDEr=4.26, BLEU-4=0.408, METEOR=0.368 versus URNN-f+b CIDEr=4.15 [1705.08759].
- **Abstractive Summarization (CNN/DailyMail, DUC-2004):** Bidirectional beam search models outperform state-of-the-art unidirectional models, with best tuning at $\gamma \simeq 0.7$ [1809.06662]. For example, BI‐RNMT produced ROUGE-1/2/L 29.05/10.90/26.05 (DUC-2004) vs. standard RNMT 28.22/10.21/25.14 [1902.08955].
- **Neural Machine Translation (NIST, WMT14):** Synchronous BiBS yielded substantial BLEU gains, e.g. Transformer baseline BLEU 47.19 (MT03-06 avg) versus BIFT 51.11 (Δ+3.92) [1902.08955].
- **Neural Response Generation:** Both BidiS and BidiA strategies on dialog corpora provide nontrivial BLEU-4 and diversity gains, especially at large beam sizes (e.g., Cornell Movie Dialog BLEU-4: VBS 1.30, BidiA 1.42 at B=50) [2110.03389].
- **Speech Recognition (CTC Decoding, Librispeech):** BiBS reduces character error rate at the start of utterances by >20% relative and overall by up to 8% compared to strong unidirectional baselines [2110.03326].

## 6. Model Architectures, Training, and Interaction Mechanisms

- **Model Architecture:** All BiBS frameworks require bidirectional models: either with bidirectional encoder-decoders (forward and backward LSTM, GRU, or self-attention decoders) [1809.06662], independently trained forward/reverse sequence decoders [2110.03389], or forward/backward LMs for LM-augmented tasks (e.g., CTC) [2110.03326].
- **Decoder Initialization:** In architectures with bidirectional encoders and decoders, the forward decoder initializes from the backward encoder’s final state, and vice versa [1809.06662].
- **Score Aggregation:** Forward and backward contributions are linearly combined via a balancing parameter ($\gamma$), or combined post-hoc via reranking [1809.06662], [2110.03389].
- **LM Fusion (CTC context):** Backward LM context is computed from a noisy future obtained with a greedy CTC pass. A future-shifting mechanism and synthetic noise ensure robustness to imperfect future estimates [2110.03326].
- **Cross-Attention Mechanisms:** Synchronous BiBS exploits cross-directional attention between hidden states at each time step, with both LSTM/GRU and Transformer decoders implementing specialized attention/fusion structures [1902.08955].

## 7. Design Principles, Hyperparameter Tuning, and Limitations

- **Beam Width ($K$):** Typical values 4–10; larger beams provide more hypothesis diversity at linear runtime cost [1809.06662], [1902.08955].
- **Score Weighting Parameter ($\gamma$):** Values in $[0.5, 0.7]$ balance past and future; $\gamma \approx 1$ recovers purely forward decoding [1809.06662].
- **Sequence Length Constraint:** $T_{\max}$ controls output length; can be used to enforce compression ratios in summarization [1809.06662].
- **Length Normalization:** Standard exponents ($\alpha$, $\beta$) can correct biases toward very short or very long outputs.
- **Bidirectional LM Training (CTC):** Future-noising with a shift $\tau$ and corruption matching the error profile of the greedy future is essential for robust performance [2110.03326].
- **Scaling Limitations:** All BiBS approaches incur higher computational and memory cost versus standard beam search, e.g., $O(B^2)$ expansion in coordinate-descent variants and need to cache beams from both directions [1705.08759]. Many variants are practical only for moderate beam sizes ($B \leq 10$).
- **Approximation Quality:** BiBS does not recover the exact MAP assignment for the full bidirectional model; all algorithmic forms (coordinate descent, additive scoring, synchronous expansion) are upper-bounded by their respective approximations. However, empirical convergence is rapid (typically 1–4 passes) and consistently yields higher joint log-probabilities and better output coherence [1705.08759].

## Table: Summary of Key BiBS Algorithms

| Algorithm      | Core Mechanism                        | Complexity (per pass)              |
|----------------|--------------------------------------|------------------------------------|
| Coord. BiBS    | Forward/backward alt. beam expansion  | $O(M T B^2 |V|)$                   |
| Synchronous BS | Parallel L2R/R2L with cross-attention | $O(T K |V| [d + attn])$            |
| BidiS/BidiA    | Forward/reverse reranking/agreement   | $O(T B |V|)$ (+$O(B^2)$ for BidiA) |
| CTC BiBS       | Prefix beam + future-augmented BiLM   | $O(2 T_{\text{LM}} + T_x B)$       |

All claims, technical details, and illustrative results are sourced directly from [1809.06662], [1705.08759], [1902.08955], [2110.03389], and [2110.03326].

Source: https://www.emergentmind.com/topics/bidirectional-beam-search-bibs