Beam Retrieval: Algorithms and Applications

• Beam Retrieval is an interdisciplinary concept combining algorithms and signal processing to reconstruct and optimize beams from noisy or limited measurements.
• It enhances multi-hop retrieval by expanding and scoring search paths over graphs and tree structures, leading to significant improvements in recall and efficiency.
• In physical sciences, beam retrieval recovers phase and amplitude using methods such as modal expansion and diffraction holography to improve system performance.

Beam Retrieval is a broad term encompassing a family of algorithms, architectures, and experimental techniques for recovering, optimizing, or tracking beams—whether physical (optical/electron beams, RF wavefronts) or metaphorical (multi-step retrieval chains, tree branches)—using structured search strategies and measurement processes that reliably locate or reconstruct beams in challenging environments. In scientific instrumentation, beam retrieval refers to phase and amplitude recovery of highly structured coherent beams from limited or noisy measurements. In modern information retrieval, beam retrieval characterizes algorithms that leverage beam search over graphs, trees, or auto-regressive models to efficiently identify relevant chains, paths, or targets from exponentially large hypothesis spaces, with applications in multi-hop question answering, dense passage retrieval, and generative retrieval.

1. Beam Retrieval in Multi-Hop Retrieval and RAG Architectures

Beam retrieval is a central component in advanced multi-hop retrieval for Retrieval-Augmented Generation (RAG) systems, particularly when answers must be constructed across multiple, potentially distant documents. In the Hierarchical Lexical Graph (HLG) framework, beam retrieval combines entity-traced proposition graphs with attention-based scoring (Ghassel et al., 9 Jun 2025). Each statement in the HLG carries embeddings and entity links, and beam expansion is explicitly constrained by the graph's structural tiers. Initialization draws from union of keyword/entity matches and embedding similarity, while each beam candidate is scored by an attention-weighted sum of statement embeddings against the query:

$$\mathbf{e}_{\mathrm{path}(P)} = \sum_{i=1}^n \alpha_i\,\mathbf{e}_{s_i},\quad \alpha_i = \frac{\exp(\cos(\mathbf{e}_Q,\mathbf{e}_{s_i}))}{\sum_{j=1}^n \exp(\cos(\mathbf{e}_Q,\mathbf{e}_{s_j}))}$$

$$\mathrm{Score}_{\mathrm{beam}}(P) = \cos(\mathbf{e}_Q, \mathbf{e}_{\mathrm{path}(P)})$$

Beam search proceeds by expanding from initial states, exploring entity-sharing neighbors, and pruning to the top-B paths at every depth; the process leverages lineage and topic tiers for cross-document multi-hop chaining (see detailed pseudocode in (Ghassel et al., 9 Jun 2025)). This approach yields substantial recall and correctness gains over chunk-based naive RAG, with empirical improvements of 23.1% (Ghassel et al., 9 Jun 2025).

In generative LLM-based retrieval, RICHES unifies retrieval and generation by imposing constrained beam decoding, where beams are constrained to corpus-valid spans using FM-index structures and allowable continuations at each decoding step, integrating unconstrained “thought” tokens with constrained corpus spans to perform retrieval as part of a single auto-regressive beam search (Jain et al., 2024).

2. Beam Retrieval in Tree and Graph-Based Structures

Beam retrieval is critical for extreme-scale retrieval tasks where targets are organized as leaves in large trees, enabling efficient top-k search with logarithmic scaling (Li et al., 2023, Zhuo et al., 2020). In the Joint Tree-based Retrieval (JTR) framework, beam search is used over a learned tree structure with node embeddings optimized via contrastive learning and a maximum-heap property at each node to ensure that relevant targets are retained in the beam (Li et al., 2023). Sibling-based negative sampling drives node embeddings to be easily rank-ordered in each beam expansion. The retrieval pseudocode maintains an active set of nodes (frontiers), selects top-t nodes by score, and expands children recursively until leaves are reached.

Theoretical analysis by (Zhuo et al., 2020) introduced “Bayes optimality under beam search” and demonstrated that standard pseudo-label training fails to guarantee optimal beam-retrieval due to the possibility of early pruning of subtrees containing highly relevant leaves. The authors provide necessary conditions and a beam-aware training algorithm—using optimal pseudo-labels derived from beam-path maxima at each node—to align training with inference and empirically boost recall and F1. Complexity is $O(k \log M)$, preserving log-scaling for million-scale corpora.

In graph-based settings, beam retrieval over proposition graphs enables rich multi-hop semantic path recovery (Wang, 25 Apr 2025). Here, propositions are nodes, and beam search explores paths that maximize average similarity—optionally incorporating coherence terms—between the path embedding and the query embedding. This mechanism is efficient (sub-millisecond per query on GPU), yields SOTA zero-shot retrieval, and demonstrates that beam width critically controls trade-offs between recall and computational cost.

3. Beam Retrieval in Multi-Hop and Multi-Step Question Answering

Beam retrieval is foundational in multi-step neural retrieval frameworks for multi-hop question answering (QA), where relevant evidence must be assembled through a reasoning chain. In the Beam Retrieval architecture (Zhang et al., 2023), retrieval is cast as a left-to-right decoding problem over passage sequences. At each hop, the model scores candidates conditional on both the question and previously chosen passages, expanding and pruning the beam to top B hypotheses. Inference and training jointly optimize the multi-hop chain, leveraging a Transformer encoder and two classification heads to specialize for initial and subsequent hops. Larger beam sizes allow for error recovery and increase negative supervision at training, translating to higher end-to-end QA performance (e.g., SOTA EM and F1 on MuSiQue-Ans, HotpotQA, and 2WikiMultiHopQA). Ablation shows diminishing returns past B=2 (Zhang et al., 2023).

Dense retrieval methods also apply beam search to vector embedding chains. In BeamDR, multi-step chains are iteratively extended by retrieving top-k passages for each refined query embedding, computed as a function of the initial question and prior evidence, with beam search maintaining the highest-scoring paths through the dense representation space (Zhao et al., 2021). This approach substantially improves multi-hop recall versus iterative or single-step approaches for QA benchmarks.

4. Physical Beam Retrieval: Wavefront, Phase, and Mode Recovery

In physical sciences, beam retrieval denotes the recovery of phase and/or amplitude structure of coherent beams from intensity or interferometric data.

In Bessel beams generated by axicons, the phase front is retrieved from intensity measurements at multiple planes by fitting a modal expansion of the incident field (in azimuthal modes), minimizing an objective function comparing measured and predicted intensities across all locations (Miao et al., 2022). Retrieved modal coefficients are then used to program a deformable mirror for closed-loop correction, achieving Strehl ratios up to 0.98 and improving focus quality from $M^2\approx 1.4$ to $1.05$ (Miao et al., 2022).

For electron vortex beams in TEM, diffraction holography enables phase retrieval by forming an interference pattern between the unknown beam and a defocused reference beam at the Fraunhofer plane (Venturi et al., 2017). Fourier-space filtering isolates the cross-term, which upon inverse transform yields the phase difference, and thus the precise OAM structure and wavefront topology of the sample beam.

In SRF cavities, beam phase retrieval is achieved by comparing phases of beam-excited higher-order monopole modes with the fundamental mode, using digital down-conversion and phase extraction for online, sub-$0.1^\circ$ resolution diagnostic measurement of beam arrival time (Shi et al., 2018).

Amplitude-only near-field beam retrieval combines adaptive support discovery (via GP-bandit methods) over DFT-discretized beamspace with subspace-restricted sparse phase retrieval (via SPARTA algorithms), producing reliable channel/beam estimates even in severe SNR and multi-path regimes (Wang et al., 8 Mar 2026). This two-stage procedure yields beamforming gains of over 70% at low SNR.

5. Analysis of Beam Search and Theoretical Properties in Retrieval

Beam search is widely used for efficient search in high-branching spaces, but introduces systematic implications for retrieval optimality. In generative retrieval models based on auto-regressive decoding and constrained output spaces, beam search propagates only the most probable prefixes according to marginal distributions, which may prune branches leading to globally high joint-probability completions. Theoretical lower bounds on the KL divergence between true and step-wise marginal posteriors under corpus constraints show unavoidable errors, even for Bayes-optimal models, due to lack of lookahead and the inherent structure of constrained auto-regressive models (Wu et al., 14 Apr 2025). Modifying the node-scoring to use the max (rather than sum) of descendant probabilities (max-heap trees) can guarantee branch survival for top-$k$ retrieval but complicates training and sacrifices data efficiency.

For learned tree and dense retrieval structures, calibration of scoring functions with respect to the beam search procedure is crucial. Algorithms and negative sampling schemes that enforce the heap/beam-invariance property are beneficial, and the tradeoff between beam width (recall vs. latency), node-embedding capacity, and clustering overlap ($\lambda$) is empirically well characterized (Li et al., 2023, Zhuo et al., 2020).

6. Applications and Empirical Impact

Beam retrieval techniques have enabled major advances across domains:

• In multi-hop QA, end-to-end beam retrieval substantially improves chain recall and enables robust pipelining into supervised or LLM-based answering. The ablation studies and benchmarks show up to 79.31 EM and 90.51 F1 on MuSiQue-Ans (BeamRetrieval, $B=2$), 99.93 EM and 99.96 F1 on 2WikiMultiHopQA, and consistent downstream improvements in LLM few-shot settings (Zhang et al., 2023).
• In dense retrieval benchmarks such as MS MARCO, tree-based beam search methods (e.g., JTR) match or improve top retrieval metrics (MRR@100, nDCG@10) at substantially lower computational cost, with beam width $b=10$ yielding sub-50-ms query times (Li et al., 2023).
• Physical sciences report RMS wavefront error improvements, Strehl ratio gains, and high-fidelity phase/OAM recovery via beam retrieval approaches (Miao et al., 2022, Venturi et al., 2017, Shi et al., 2018).
• In proposition-graph retrieval, explicit beam search recovers multi-hop reasoning chains with near-perfect recall on HotpotQA (97.0%, R@5) and strong F1 scores, without online LLM cost (Wang, 25 Apr 2025).

Beam retrieval thus provides a unifying paradigm for precise, scalable, and fault-tolerant selection and reconstruction in high-dimensional or combinatorial search spaces, across information retrieval and signal processing.