---
title: 'FuseSearch: Hybrid Retrieval & Code Localization'
url: https://www.emergentmind.com/topics/fusesearch
type: topic
---

# FuseSearch: Hybrid Retrieval & Code Localization

FuseSearch refers to two distinct frameworks addressing joint optimization in high-throughput, information-dense search: (1) adaptive parallel code localization in automated software development pipelines [2601.19568] and (2) attribute-vector filtered approximate nearest neighbor (ANN) retrieval via convexified fusion [2509.19767]. Both leverage “fused” objective formulations to overcome speed-quality trade-offs in their domains, optimizing across both retrieval precision and computational or informational efficiency.

## 1. Problem Definitions and Motivations

### Code Localization

FuseSearch for code localization targets the task: Given an issue description $q$ and a large codebase, identify the minimal set of files and functions to modify to resolve the issue. Traditional sequential agents make a single tool call per step (e.g., grep, glob, read_file), leading to information starvation under tight interaction budgets—agents quickly deplete allowed turns without sufficient evidence, causing severe accuracy degradation. Naive parallelism—issuing a fixed number $k$ of tool calls per turn—alleviates context starvation but exhibits a 34.9% redundant invocation rate due to duplicate explorations, squandering compute and introducing noise that harms localization quality [2601.19568].

### Hybrid Attribute-Vector ANN

FuseSearch in hybrid ANN settings addresses filtered nearest neighbor queries, where both content vector similarity and attribute constraints (e.g., category, date) are imposed: For a database $\mathcal{D}$ with each $o_i$ represented by $v(o_i) \in \mathbb{R}^d$ and attributes $f^j(o_i)$, find the top-$k$ objects closest in vector space while satisfying $f^j(o) = F^j_q$ (categorical) or $f^j(o) \in [l,u]$ (range). Current solutions resort to staged filtering or index “hacks” (disjoint attribute and ANN indices), incurring recall/speed overhead and failing under high filter cardinality or low-selectivity queries [2509.19767].

## 2. Formalization of Joint Quality–Efficiency Objectives

### Tool Efficiency and Reward Structures

The code localization FuseSearch defines:

- **Precision, Recall, F1**:
  $$
  P = \frac{|\hat{A} \cap A|}{|\hat{A}|}, \quad
  R = \frac{|\hat{A} \cap A|}{|A|}, \quad
  F_1 = \frac{2PR}{P + R}
  $$
  where $\hat{A}$ is the predicted set, $A$ the ground truth.

- **Per-call Information Gain** $g_i$:
  $$
  g_i =
  \begin{cases}
    \frac{|\mathcal{E}_i \setminus \mathcal{H}|}{|\mathcal{E}_i|} & |\mathcal{E}_i| > 0 \\
    0 & \text{otherwise}
  \end{cases}
  $$
  with $\mathcal{H}$ as the set of code entities previously seen and $\mathcal{E}_i$ the $i$th tool call’s output.

- **Tool Efficiency** $e(\tau) = \frac{1}{k} \sum_{i=1}^k g_i$, $k$ the number of tool calls in trajectory $\tau$.

- **Reward**: FuseSearch employs
  $$
  R(\tau) = \alpha F_1(\tau) + \gamma (F_1(\tau) \cdot e(\tau))
  $$
  with $\alpha = 0.8$, $\gamma = 0.2$, and $F_1$ the weighted sum of file-level and function-level $F_1$.

### Hybrid Attribute-Vector Fusion Objective

The filtered ANN formulation proceeds from a lexicographic multicriteria selection, then relaxes it as:
$$
L(o; \alpha, \beta) = \sum_{j=1}^F \alpha_j \sigma_j(f^j(o), F^j_q) + \beta \rho(v(q), v(o))
$$
where $\sigma_j$ is the attribute comparison (e.g., 0/1 for categorical), $\rho$ the vector distance. The fused nearest neighbor is approximated by minimizing $L(o)$, and filter/vectors are embedded such that fused Euclidean norm reflects this penalized sum.

## 3. Training and Algorithmic Foundations

### Adaptive Parallel Execution for Localization

FuseSearch’s code localization policy is learned in two phases [2601.19568]:

- **Supervised Fine-Tuning (SFT)**: Trajectories are synthesized via a teacher model guiding 2–8 tool calls per turn across ~21K GitHub issue–patch pairs. Joint filtering on $F_1$ and $e$ produces ~6K demonstrations, optimizing cross-entropy over JSON-structured tool predictions.

- **Reinforcement Learning (RL)**: The Group Relative Policy Optimization (GRPO) algorithm maximizes the hybrid reward, regularized toward the SFT policy by a KL term ($\beta = 0.01$) to prevent catastrophic drift.

Adaptive inference policy dynamically determines both the number and type of tool calls; in early turns the policy executes broad parallel exploration, tapering to focused refinement as uncertainty collapses.

### Fused ANN Embedding and Indexing

For hybrid ANN, FuseSearch constructs a fused space embedding $\Psi(v, f; \alpha, \beta)$ via blockwise or neural composition:
$$
\Psi(v, f; \alpha, \beta) = \left[ \frac{v^{(1)} - \alpha f}{\beta}, \ldots, \frac{v^{(d/m)} - \alpha f}{\beta} \right]
$$
For general joint-embedding, neural encoders $E^c$, $E^f$ are concatenated and processed through transformer layers, with a penalized NN loss enforcing margin ordering in the fused objective:
$$
\ell = \max \{0, d_{\text{fuse}}(q, o^+) - d_{\text{fuse}}(q, o^-) + \text{margin}\}
$$
Indices (HNSW, IVF) are built on these fused vectors for both attribute and vector search, with candidate set sizing regulated by theoretical bounds to preserve top-$k$ and recall guarantees.

## 4. Implementation Details

### Code Localization Implementation

- **Backbones**: Qwen3-4B-Instruct, Qwen3-30B-A3B-Instruct.
- **Tools**: grep, glob, and read_file—language-agnostic and read-only.
- **Infrastructure**: SFT utilizes 8 × NVIDIA H20 GPUs; RL utilizes 32 × NVIDIA H20 GPUs leveraging vLLM and RLLM/veRL. Parallel tool invocation is implemented via shell/backend concurrency without inter-call synchronization.

### ANN Indexing Procedures

- **Fused Single-Attribute Index**:
  - Each $o_i$ is encoded as $v_i' = \Psi(v(o_i), f(o_i); \alpha, \beta)$.
  - Insert into HNSW/IVF index.
- **Query-Time**:
  - Query embedded as $\Psi(v(q), F_q; \alpha, \beta)$, retrieve top $k'$ candidates, rerank by $L(o; \alpha, \beta)$ for hybrid objective.

- **Range Filtering**: Queries over attribute intervals are handled by embedding query/attribute pairs as line segments in fused space, with a two-level index structure (angular direction and midpoint) supporting efficient cylinder-based search and precise radius adjustments.

## 5. Empirical Results and Analysis

### Code Localization

On SWE-bench Verified (386 issues, "new file/function" patches excluded):

| Model           | File $F_1$ | Function $F_1$ | Efficiency $e$ | Turns | Time (s) | Tokens (k) |
|-----------------|-----------|---------------|---------------|-------|----------|------------|
| Base, parallel  | 64.50%    | 38.91%        | 59.5%         | 4.24  | 6.12     | 47.9       |
| FuseSearch      | 84.65%    | 56.43%        | 69%           | 4.78  | 5.43     | 30.9       |
| RepoSearcher    | 38.12%    | —             | —             | —     | —        | —          |

FuseSearch demonstrates state-of-the-art performance, improving file-level $F_1$ by +20 percentage points and function-level $F_1$ by +17 points. Search time drops by 93.6%, turns by 67.7%, and token consumption by 68.9% relative to strong baselines.

Ablation studies reveal:
- **SFT alone**: Increases quality but less efficient.
- **RL**: Modest additional gain without prior SFT.
- **Sequential vs Parallel**: Parallelized SFT+RL halves the search cost and token usage against sequential execution.
- **Reward Structure**: Combined $F_1 + F_1·e$ yields the best joint quality and efficiency.

The learned policy exhibits high initial parallel breadth (≈5 calls/turn), transitioning to refinement (≈2 calls/turn).

### Hybrid ANN Search

Benchmarks across SIFT1M, GloVe-1.2M, DEEP, YouTube-Audio, and WIT-Image:

| Benchmark                 | Task            | QPS Improvement | Recall@10 |
|---------------------------|-----------------|-----------------|-----------|
| SIFT1M, single attr       | Ann-Hybrid      | 4.2× over NHQ   | ≈0.95     |
| SIFT1M, multi-attr        | Ann-Hybrid      | 3.2× over NHQ   | ≈0.95     |
| DEEP, range filter        | Ann-Range       | 4–6× over SeRF  | ≈0.95     |
| YouTube-Audio, range      | Ann-Range       | 7–13× ANNS-first| ≈0.95     |

FuseSearch maintains throughput above $10^5$ QPS in multi-attribute scenarios where alternatives drop below $10^3$ [2509.19767].

## 6. Theoretical Guarantees and Parameterization

- **Order-Preserving**: Fused space preserves exact content ranking among objects with matching attributes.
- **Exact-Filter Limit**: As attribute penalty $\alpha \to \infty$ with bounded $\beta$, FuseSearch reduces to strict attribute filtering.
- **Approximation**: Given an ANN engine returning candidates within $(1+\epsilon_{\text{fuse}})$ of optimal fused distance, the same relative error holds for the penalized hybrid objective.
- **Parameter Selection**: To maintain intra-cluster compactness and inter-cluster separation, set $\beta \geq \delta_{\max}/\epsilon_f$ and $\alpha$ per derived bounds, where $\delta_{\max}$ and $\sigma_{\min}$ are maximal content and minimal attribute distances, respectively.

## 7. Limitations and Future Perspectives

- Code localization ground truth comprises a single “golden patch”; supporting multiple valid solutions is an open evaluation challenge.
- Current datasets are biased toward Python; performance on static languages requires further study.
- FuseSearch in code localization settings employs a minimal set of read-only tools; extension to semantic/static analysis (AST, type inference) is plausible for future gains.
- For ANN, the approach assumes sufficient embedding capacity and ANN index scalability; range-filtering and multi-filter heuristics can be further refined for extreme-scale, low-selectivity, or high-dimensional scenarios.

FuseSearch establishes a unified, theoretically principled paradigm for fusing parallel exploration and constraint satisfaction, achieving high retrieval quality and efficiency across both automated software localization [2601.19568] and hybrid similarity search [2509.19767].

Source: https://www.emergentmind.com/topics/fusesearch