---
title: 'PI-SERINI: Minimal BM25 Research Agent'
url: https://www.emergentmind.com/topics/pi-serini
type: topic
---

# PI-SERINI: Minimal BM25 Research Agent

Pi-Serini is a minimal agentic search system designed to systematically evaluate whether properly configured lexical retrieval, specifically BM25, can suffice in “deep-research” pipelines powered by modern large language models (LLMs) [2605.10848]. In contrast to prevailing trends toward dense and reasoning-aware retrievers, Pi-Serini leverages a well-tuned BM25 backend and an explicit agent-tool loop to match or exceed the performance of denser, more complex systems on demanding research benchmarks. By decoupling retrieval, browsing, and reading actions within a controlled agentic loop, Pi-Serini provides insight into the sufficiency of lexical baselines and optimizes the agent–retriever interface for high-evidence recall and answer accuracy.

## 1. Motivation and Foundational Question

The field of deep research systems—often cast as multi-step Retrieval-Augmented Generation (RAG) or ReAct-style agents—has traditionally positioned retriever quality as an upper bound (“hard ceiling”) on answer performance. As such, research commonly advances toward sophisticated retriever architectures: dense embedding retrievers, zero-shot semantic matchers, and even retrievers capable of explicit reasoning. However, as frontier LLMs become increasingly proficient at planning, tool use, and iterative reflection, a critical question arises: is continual innovation in retriever design necessary, or does a well-configured lexical retriever suffice in the context of an LLM-driven agentic loop? Pi-Serini is proposed to disentangle these factors, testing whether previous BM25 baselines were artificially limited due to shallow recall or sub-optimal parameterization rather than true lexical retrieval constraints.

## 2. Architectural Structure and Agentic Loop

At its core, Pi-Serini implements a ReAct-style agentic loop, wherein the LLM alternates between “thinking” (producing reasoning traces) and “acting” (issuing tool calls) until a conclusive answer is produced. The agent interfaces with a Retrieval Controller that exposes three instrumented tools:

- **search(reason, query):** Submits a BM25 query, caching the ranking (up to 1,000 hits) with a unique search_id. Returns the top 5 excerpts initially.
- **read_search_results(reason, search_id, offset, limit):** Enables paginated browsing of cached search results without re-querying the backend.
- **read_document(reason, docid, offset, limit):** Facilitates streaming reads of individual documents in a line-based fashion.

The Retriever Controller maintains logs of four document sets: surfaced ($D_\text{surfaced}$), previewed, opened, and cited—enabling granular measurement of retrieval effectiveness at each stage of evidence access. Pi-Serini operates under a two-stage time-budget regime, defaulting to $T = 300$ seconds per query, with a “submit-now” steer issued at $t = 0.7T$ to prompt answer generation and curtail further tool use. This architecture empowers the LLM to control retrieval depth and context window insertion, moving beyond simplistic “top-$k$ stuffing” strategies and facilitating explicit evidence management.

## 3. BM25 Retrieval Formalism and Tuning

Pi-Serini utilizes Anserini’s BM25 implementation. The BM25 scoring function for document $D$ and query $Q$ is:

$$
\text{Score}(D, Q) = \sum_{t \in Q} \text{IDF}(t) \cdot \frac{f(t, D) \cdot (k_1 + 1)}{f(t, D) + k_1 \cdot (1 - b + b|D|/\text{avgdl})}
$$

where $f(t, D)$ is the frequency of term $t$ in $D$, $|D|$ the document length, $\text{avgdl}$ the average document length, $k_1$ controls term-frequency saturation, $b$ controls document-length normalization, and $\text{IDF}(t) = \log\frac{N-n_t+0.5}{n_t+0.5}$.

Experiments on BrowseComp-Plus (documents averaging $\sim$2,000 tokens; 90th percentile $\sim$14,000 tokens) demonstrated that vanilla BM25 defaults ($k_1=0.9$, $b=0.4$) were inadequate for long-document ranking. Grid search across a 100-query subset established that high parameter values ($k_1 \approx 16\textrm{--}25$, $b \approx 1.0$) are optimal. Pi-Serini adopts $k_1 = 25$, $b = 1$ throughout, yielding substantial improvements in recall and downstream answer performance.

## 4. Retrieval Depth, Evidence Recall, and Agent-Interaction

Pi-Serini systematically explores the impact of retrieval depth—the number of search hits ($k$) cached by the initial search tool. Results indicate that:

- At $k=5$ (a common shallow default), surfaced recall—fraction of evidence docs in $D_\text{surfaced}$—is approximately 70%.
- Increasing to $k=50$ lifts surfaced recall to ~86%; at $k=100$, recall plateaus near this level.
- Maximum depth ($k=1,000$) achieves surfaced recall of 95.8%, nearly matching the oracle level (i.e., BM25 is exposed to almost all required evidence).

However, peak previewed recall (fraction of evidence the agent actually browses) is $74.7\%$ at $k=50$, indicating that increased retrieval depth aids utility only if the agent appropriately explores the available results. Boosting from default shallow settings ($k=5$) to deep settings ($k=1,000$) yields a $+25.3\%$ surfaced-recall gain.

## 5. Benchmarking on BrowseComp-Plus

Evaluation occurs on BrowseComp-Plus, a fixed-corpus deep-research benchmark (830 queries; ~100,000 documents). Each query averages $6.1$ evidence documents and $2.9$ gold documents. The evaluation protocol incorporates:

- **Accuracy:** Judged by gpt-5.3-codex, assessing final exact-answer match with ground truth, allowing trivial rephrasings.
- **Surfaced Recall:** Recall over $D_\text{surfaced}$.
- **Previewed Recall:** Recall over $D_\text{previewed}$.
- **Behavior Recall:** Recall over $D_\text{opened} \cup D_\text{cited}$.
- **Time Budget and Cost:** 300s per query, “submit-now” cut-off at $0.7T$, with per-query cost in USD using standard token pricing.

## 6. Empirical Performance and Ablation Results

Pi-Serini is evaluated across several LLMs, including DeepSeek Flash/Pro, Claude Haiku/Opus, and OpenAI’s GPT-5, 5.2, 5.4, and 5.5. All experiments employ the tuned BM25 at maximum retrieval depth ($k=1,000$). Key results with GPT-5.5:

| Metric                  | Value     |
|-------------------------|-----------|
| Answer accuracy         | 83.1%     |
| Surfaced evidence recall| 94.7%     |
| Previewed recall        | 73.6%     |
| Behavior recall         | 58.9%     |
| Total cost (USD)        | \$291.6   |

Comparative benchmarks show that Pi-Serini outperforms prior dense-retriever agents (e.g., GPT-5 + qwen3-embed-8b achieves 73.0% accuracy and 79.0% surfaced recall at a cost of \$360.7). Controlled ablations demonstrate that tuning BM25 from default ($k_1=0.9$, $b=0.4$) to the Pi-Serini configuration ($k_1=25$, $b=1$) at depth=1,000 boosts answer accuracy from 64.0% to 82.0% (+18.0 pp) and surfaced recall from 84.6% to 95.7% (+11.1 pp). Increasing $k$ from 5 to 1,000 raises surfaced recall by 25.3 pp (70.5% to 95.8%). Pi-Serini also achieves lower per-run costs—3x to 10x less than prior baselines—attributable to time-budget management, cache-efficient ReAct loops (≈80% of tokens from cache), and reduced tool calls per query ($\sim$15–20 vs. $\sim$23).

## 7. Contributions, Implications, and Practical Takeaways

Pi-Serini makes three primary contributions:

- Definition of a minimal search-agent framework that cleanly dissociates retrieval, browsing, and reading processes, with instrumented, paginated tools for expressive evidence management.
- A comprehensive reassessment of BM25’s efficacy on deep-research benchmarks, revealing that prior weaknesses are more attributable to sub-optimal tuning and shallow retrieval than to intrinsic lexical limitations.
- Empirical validation that BM25-based agents can equal or surpass dense-retriever systems on BrowseComp-Plus, while substantially reducing computational cost and highlighting clear optimization levers (parameterization, retrieval depth, tool design).

A plausible implication is that future advances in deep-research systems may derive more from optimizing agent-recognized evidence management and navigation than from further incremental sophistication in retriever architectures themselves. Ensuring proper BM25 tuning and sufficient retrieval depth should be a foundational step before pursuing more complex retrieval solutions.

Source: https://www.emergentmind.com/topics/pi-serini