---
title: Grid Beam Search (GBS)
url: https://www.emergentmind.com/topics/grid-beam-search-gbs
type: topic
---

# Grid Beam Search (GBS)

Grid Beam Search (GBS) is an extension of the classical left-to-right beam search algorithm that enables the incorporation of arbitrary lexical constraints—user-specified words or phrases that must be present exactly once in a generated output sequence. Unlike standard beam search, which seeks to maximize model likelihood without enforcing constraint inclusion, GBS ensures that every output sequence returned satisfies all such constraints, without requiring modification of model parameters or retraining. The algorithm is formulated for general sequence generation models, making it directly applicable in multiple scenarios such as interactive neural machine translation and domain adaptation [1704.07138].

## 1. Motivation and Problem Statement

Conventional beam search aims to identify the highest-probability sequence
\[
\hat{\mathbf y} = \arg\max_{\mathbf y} p_\theta(\mathbf y \mid \mathbf x) = \arg\max_{\mathbf y} \prod_{t=0}^T p_\theta(y_t \mid \mathbf x, y_{<t})
\]
but lacks any mechanism to guarantee that specified lexical elements appear in the generated output. This becomes limiting in use-cases like interactive post-editing and domain-specific translation, where it is often crucial to force the decoder to include a set of constraints $\{\mathbf c_1, \dots, \mathbf c_M\}$ (each being a single- or multi-token word or phrase) in the output. GBS addresses this by constraining the search space, $\mathcal Y(\mathbf c)$, to sequences containing each $\mathbf c_i$ as (contiguous) subsequences exactly once.

## 2. Formalization and Search Structure

GBS frames constrained decoding as the following maximization:
\[
\hat{\mathbf y} = \arg\max_{\mathbf y \in \mathcal Y(\mathbf c)} \log p_\theta(\mathbf y \mid \mathbf x) = \arg\max_{\mathbf y \in \mathcal Y(\mathbf c)} \sum_{t=0}^T \log p_\theta(y_t \mid \mathbf x, y_{<t})
\]
where $\mathcal Y(\mathbf c)$ is the set of possible outputs meeting all lexical constraints. The search is organized in a two-dimensional grid of beams parameterized by $(t, c)$: $t$ is the output timestep, and $c$ tracks the total number of constraint tokens covered. Each cell $\texttt{Grid}[t][c]$ stores up to $k$ best hypotheses with $t$ generated tokens and $c$ covered constraint tokens.

Hypotheses are labeled as either:
- **Open**: permitted to freely generate any token (via `GENERATE`) or to `START` a new, unused constraint.
- **Closed**: currently in the middle of realizing a constraint and required to `CONTINUE` emitting its tokens.

The output space is fully traversed until a hypothesis covering all $N = \sum_i L_i$ constraint tokens (with $L_i$ the length of $\mathbf c_i$) has been generated and EOS has been emitted.

## 3. Algorithmic Operation

The high-level GBS decoding procedure is as follows:

- The grid is initialized such that $\texttt{Grid}[0][0]$ contains the initial (BOS) hypothesis.
- At each timestep $t$ and constraint coverage $c$, candidates are assembled from:
  - Extending open hypotheses in $\texttt{Grid}[t-1][c]$ using `GENERATE`.
  - Starting new constraints from open hypotheses in $\texttt{Grid}[t-1][c-1]` with `START`.
  - Continuing in-progress constraints from closed hypotheses in $\texttt{Grid}[t-1][c-1]` via `CONTINUE`.
- After scoring, only the $k$-best hypotheses are retained in each grid cell.
- Finished hypotheses in $\texttt{Grid}[*][N]$ producing EOS are considered; the best is output.

The pseudocode explicitly implements this control flow, managing open/closed hypothesis status and ensuring each constraint is handled precisely once.

## 4. Computational and Practical Considerations

Computational complexity for GBS is $\mathcal O(T_{\max} \times N \times k)$, compared to $\mathcal O(T_{\max} \times k)$ for unconstrained beam search. In practice, because the total number of constraint tokens $N$ is typically small ($5$–$20$), efficiency is maintained via parallelization across timestep grids and conservative beam sizes (typically $k=5$–$10$).

Additional practical measures include:
- Aggressive hypothesis pruning, or imposing a length cap $T_{\max}$ (set e.g. to $\alpha|x|$, $\alpha\approx1.3$).
- Using subword vocabularies (such as BPE) for robust constraint matching, including previously unseen words.
- Merging overlapping constraints or tokenizing all constraints with the same pre-processing as the main model for alignment.

## 5. Illustrative Example

A toy scenario constrains the output to include “Paris” and “visited” as single-token constraints. At $t=0, c=0$, the initial hypothesis is present. At $t=1$, possible operations include:
- `GENERATE` yielding “He” or “She” ($c=0$),
- `START` “Paris” or “visited” ($c=1$).

As decoding proceeds, it is possible to alternate between generating unconstrained tokens or further constraints, ensuring that all permutations (“She visited Paris”, “Paris was visited”) that include both constraints are considered. The grid structure guarantees exhaustive but efficient exploration of the constrained output space.

## 6. Empirical Results

Two empirical domains highlight GBS benefits:

- **Interactive Post-Editing (Pick–Revise):** Simulated 4-cycle iterative translation, adding one up-to-3-word constraint per cycle, yielded progressive BLEU increases (EN→DE): 18.44 (baseline), 27.64 (+9.20), 36.66 (+9.01), 43.92 (+7.26).
- **Domain Adaptation (Terminology Injection):** Domain-agnostic NMT constrained using automatically extracted terminology pairs achieved BLEU gains for Autodesk IT: EN→DE 26.17→27.99 (+1.82), EN→FR 32.45→35.05 (+2.60), EN→PT 15.41→29.15 (+13.74).

These results confirm large improvements in translation quality for both interactive and zero-shot domain adaptation scenarios, solely by imposing lexical constraints at inference [1704.07138].

## 7. Relationship to Alternative Methods

Standard beam search is incapable of guaranteeing constraint satisfaction—in experiments, BLEU remains unchanged because desired phrasings often do not occur. Prefix-based interactive translation systems can generate outputs consistent with a fixed initial constraint, but can only enforce a single prefix, not multiple or internal constraints. Phrase-based SMT Pick–Revise approaches require phrase tables and explicit alignment, unlike GBS’s token/subword approach that needs no retraining. Soft-constraint and joint attention models require additional training and architectural complexity, whereas GBS operates out-of-the-box atop any pretrained sequence model.

## 8. Extensions, Limitations, and Best Practices

GBS can accommodate discontinuous constraints (such as phrasal verbs with intervening tokens) by filtering valid start/continue points. Subword vocabularies enable the handling of out-of-vocabulary constraint tokens. The principal limitation is linear runtime scaling with $N$; efficient implementation entails capping $T_{\max}$, minimizing $k$, and leveraging beam-level parallelism. Merging overlapping constraints and tokenizing constraints identically to main inputs further enhances efficiency. Early exit upon full constraint coverage and EOS generation is recommended, avoiding unnecessary grid expansion.

In summary, Grid Beam Search represents a straightforward but effective generalization of beam search that tightly integrates arbitrary lexical constraints into output sequences, with demonstrable gains across interactive and domain-adaptation use-cases, and with broad applicability to sequence generation tasks [1704.07138].

Source: https://www.emergentmind.com/topics/grid-beam-search-gbs