Paper-Search Systems & Frameworks
- Paper-search is a collection of systems and frameworks that support distributed retrieval, semantic indexing, and decision-making to manage scholarly literature.
- The methodologies combine traditional BM25 ranking with taxonomy-guided and multimodal signals to improve precision and recall in academic search.
- These approaches enable practical applications in systematic reviews, real-time corpus acquisition, and adaptive search workflows through agentic retrieval strategies.
Paper-search denotes the family of systems and evaluation frameworks that support the retrieval, exploration, expansion, validation, and management of scholarly literature. Across the recent literature, it includes distributed search over large publication collections, search-driven corpus acquisition, faceted portals for arXiv, project-based personalization, taxonomy-guided semantic indexing, multimodal page-level search over PDFs, and LLM-based agents that iteratively issue search and expansion calls under explicit or learned control. The problem is therefore treated not only as ranked retrieval, but also as citation-graph traversal, workflow induction, and sequential decision-making under evolving research intent (Bashir et al., 2014, Gollapalli et al., 2016, Springstein et al., 2018, Li et al., 1 Jul 2026).
1. Distributed infrastructure, acquisition pipelines, and early portal design
Early work emphasized scale, distribution, and corpus construction. The Grid-based Academic Publications Search technique organized search as interconnected grid services across Virtual Organizations, with a User Search Interface, Query Search Engine, Query Manager, and per-node Search Service. In the reported experiments on 12 heterogeneous Linux nodes, GAPS reached a speedup of 2.59 at , while maintaining efficiency above 0.25, illustrating a grid-computing approach to searching distributed academic publications (Bashir et al., 2014).
A complementary line of work treated search engines themselves as acquisition infrastructure. The search/crawl framework for automatically acquiring scientific documents interleaved a title-search path and an author-search-to-homepage-to-crawl path. It used quoted paper titles and author names as Bing queries, a RankSVM model for homepage identification, and a Random Forest paper classifier over structural PDF features. The large-scale run obtained approximately 0.665 million research documents using about 0.076 million queries, positioning Web search as an alternative to seed-based crawl methods for building scientific portals (Gollapalli et al., 2016).
Portal-oriented systems then shifted attention from acquisition to interactive discovery. TIB-arXiv extended arXiv Sanity Preserver by removing the original topical restriction, storing all arXiv papers in a unified Elasticsearch back end, and exposing text-based search, visualisation, faceted filtering, and an embedded PDF viewer through a novel user interface (Springstein et al., 2018). Rivendell addressed a different systems problem: concept drift in academic search. Its central mechanism was explicit project context, which grouped queries and relevance judgments by project rather than by undifferentiated user history; in a 199-subject study, project-based search improved relevance by up to 12.8 percent compared to life-time search (Lazebnik et al., 2022).
2. Indexing, ranking, and semantic representations
A recurrent substrate in paper-search is the inverted index with BM25-style scoring, usually augmented by domain-specific structure. TIB-arXiv indexed title, abstract, authors, and categories in Elasticsearch, combined BM25 textual relevance with recency, Twitter mentions, and user collection counts, and exposed weighting presets such as “Relevance” and “Trending.” WisPaper’s Quick Search similarly relied on an inverted index and BM25, with language-aware preprocessing and millisecond-level latency on a 10 M+ document index. SARCH used Apache Solr for BM25 over OCR text, captions, and table summaries, while its hybrid mode merged keyword and embedding retrieval with Reciprocal Rank Fusion (Springstein et al., 2018, Ju et al., 7 Dec 2025, Sinha et al., 7 Nov 2025).
Recent work has pushed beyond flat document representations toward explicit semantic control. TaxoIndex constructs a taxonomy-guided semantic index by extracting core topics and indicative phrases, then uses an add-on indexing network and fusion module to enhance frozen dense retrievers. On CSFCube and DORIS-MAE it improved NDCG@5 and Recall@50, and the reported ablations indicate that both topic- and phrase-level signals contribute materially to retrieval quality, especially in limited-data regimes (Kang et al., 2024).
Other systems impose structure directly on the corpus. MatKB indexes polycrystalline materials synthesis procedures at the paragraph level, stores extracted entity slots such as Material_recipe and Property_temperature in Elasticsearch, and supports slot-constrained bool-must queries over a 9.5 million publication source corpus processed through filtering and NER (Yang et al., 2023). PaperRegister replaces the conventional abstract-only index with a hierarchical register tree whose shallow nodes encode coarse topics and whose deeper nodes encode finer details such as module configuration and training operation. Its online stage uses a 0.6B view recognizer to route queries to the relevant sub-indexes; on flexible-grained search tasks, PaperRegister achieved the strongest Recall@5 across coarse, fine, and very-fine settings, with the advantage increasing as query granularity became finer (Li et al., 14 Aug 2025).
Taken together, these systems show that paper-search ranking is no longer limited to whole-document lexical matching. It increasingly combines lexical, structural, semantic, and user- or corpus-derived signals, with explicit mechanisms for granularity selection and interpretability.
3. Agentic retrieval and sequential search policies
A major recent development is the reformulation of paper-search as an agentic process. PaSa introduced a two-agent architecture consisting of a Crawler and a Selector. The Crawler operates over a paper queue and decides whether to invoke , , or ; the Selector evaluates each final paper against the query and also serves as an auxiliary reward model. PaSa was optimized with reinforcement learning on the synthetic AutoScholarQuery dataset and evaluated on RealScholarQuery, where PaSa-7B achieved Final Recall and outperformed Google-based and GPT-based baselines by large margins in recall (He et al., 17 Jan 2025).
SPAR approached the same task through a modular, training-free multi-agent framework. Its Query Understanding Agent decomposes the initial query, the Retrieval Agent issues multi-source searches, the Judgement Agent scores relevance, the Query Evolver Agent generates follow-up queries conditioned on the current paper cache, and the Reranker Agent combines relevance, authority, and recency. Its RefChain mechanism explicitly couples direct retrieval with one-layer citation expansion, and on SPARBench it achieved , exceeding the listed PaSa result of $0.1041$ on that benchmark (Shi et al., 21 Jul 2025).
PaperScout formalized academic paper search as a Partially Observable Markov Decision Process in which the state is the latent paper pool, the observation is a budgeted summary of expanded and unexpanded papers plus interaction history, and the action is a discrete choice over Search and Expand calls. Its main algorithmic contribution, Proximal Sequence Policy Optimization, moves RL from token-level optimization to turn-level sequence optimization to address the “granularity mismatch” of multi-turn retrieval. On RealScholarQuery, PaperScout with Qwen3-4B and PSPO reported Precision , Recall , , and a higher LLM-score than the no-RL variant and SPAR (Pan et al., 15 Jan 2026).
PaperPilot further made the search process explicit by representing it as an executable DAG over operators such as keyword search, citation expansion, filtering, scoring, reranking, and evidence extraction. The agent induces a workflow, executes it, and then refines the graph itself in response to user feedback rather than merely rewriting text queries. In multi-turn interaction, PaperPilot-9B improved Hit@5 from 58.0 to 77.0, MRR from 47.5 to 59.4, and nDCG@10 from 26.8 to 32.5, while reducing workflow execution errors from 9.5% to 0% (Li et al., 1 Jul 2026).
An adjacent formulation appears in PaperSearchQA, which trains search agents to reason over 16 million biomedical paper abstracts with RLVR. Here the environment appends retrieved papers to the LLM context when the model emits a <search> action, and reward is given only for exact-match answers. Search-R1 with RLVR reached 51.0% on PaperSearchQA and 44.8% on BioASQ, substantially above non-RL baselines such as RAG and PaperQA2 (Burgess et al., 26 Jan 2026). This suggests that paper-search and scientific question answering are converging around shared search-and-reason environments.
4. Multimodal, domain-specific, and workflow-integrated systems
Paper-search is increasingly multimodal. GovScape provides four primary forms of search over 10,015,993 federal government PDFs from the 2020 End of Term crawl, totaling 70,958,487 PDF pages: metadata filtering, exact text search, semantic text search, and visual search over page images. Its pipeline splits PDFs into pages, renders images, extracts text, computes BGE text embeddings and CLIP visual embeddings, and stores them in separate Faiss indexes. The reported preprocessing cost was approximately \$1,500 for 10 million PDFs, corresponding to about 47,000 PDF pages per dollar spent on compute (Deeds et al., 14 Nov 2025).
SARCH addresses a different multimodal setting: scanned archaeological books and reports with variable scan quality. Its ingestion pipeline performs grayscale conversion, bilateral filtering, and Otsu thresholding; uses the Surya toolkit for layout analysis and OCR; applies Qwen2.5-VL-7B-Instruct to table parsing; and classifies images into map, photograph, site layout, or generic figure with zero-shot CLIP. In the reported benchmark, embedding-based retrieval outperformed keyword-only retrieval, achieving 0 and MRR 1, while the hybrid RRF approach remained competitive (Sinha et al., 7 Nov 2025).
Several systems specialize paper-search for particular research workflows. Paperfetcher automates handsearch for systematic reviews through a Python package and Streamlit web app. It queries Crossref journal endpoints over date ranges, supports backward snowballing through Crossref references and forward snowballing through OpenCitations COCI, and exports DOI lists, RIS, CSV/Excel, or pandas DataFrames. The authors note that a typical handsearch across 10 journals with fewer than 5000 papers completes in under 2 minutes on a standard laptop network connection (Pallath et al., 2021). MatKB narrows the search space still further by indexing materials-synthesis paragraphs with semantic slots, allowing direct retrieval of procedures by ingredient, temperature, time, or other extracted entities (Yang et al., 2023).
WisPaper integrates search with management and continuous monitoring. Its architecture combines Scholar Search, a user Library, and AI Feeds in a shared paper database so that any paper found by any module becomes available to the others. Quick Search is keyword-based, while Deep Search uses an LLM-driven agent that decomposes a free-text query into 2–4 Boolean search queries and 1–4 weighted validation criteria, retrieves candidates, assesses each paper against the criteria, and supports collaborative refinement by the user. AI Feeds performs scheduled batch retrieval and semantic filtering to reduce daily category harvests of 50–100 new papers to 3–5 highly relevant papers (Ju et al., 7 Dec 2025).
These systems broaden the meaning of paper-search. The searchable unit may be a paper, a paragraph, a PDF page, an image crop, a table, a DOI, or an evidence-bearing workflow state. The operational objective may be exploratory discovery, systematic review, procedure retrieval, or continuous frontier tracking.
5. Benchmarks, metrics, and reliability
The benchmark literature increasingly treats paper-search as an open-environment evaluation problem rather than a closed retrieval task. ScholarQuest, proposed by Pan et al., was designed specifically for agentic academic search under a shared retrieval backend. It contains 1,111 taxonomy-guided queries spanning over 1,000 computer science topics and four intent types: method-oriented, setting-anchored, scope-controlled, and comparison-based. All systems operate over ScholarBase, which exposes BM25, dense retrieval, hybrid retrieval, metadata lookup, and citation/reference expansion. In the reported results, PaperScout was the best-performing agent, but it still achieved only 0.314 Recall@100 and 0.355 Recall@All, and all agents struggled on scope-controlled queries (Pan et al., 18 Jun 2026).
SPARBench emphasizes expert-annotated relevance. It begins from 50 seed queries, retrieves 198 K candidates from ArXiv, PubMed, OpenAlex, and Semantic Scholar, filters them in two LLM stages, and then relies on expert validation to identify approximately 560 truly relevant papers. Its use of document-level Precision, Recall, and 2 highlights the tension between recall-oriented citation expansion and precision-oriented filtering in agentic systems (Shi et al., 21 Jul 2025).
RealScholarQuery and AutoScholarQuery, introduced alongside PaSa, add another layer of experimental control. AutoScholarQuery contains 33,551 train, 1,000 dev, and 1,000 test synthetic queries derived from related-work sections of major AI conferences, while RealScholarQuery contains 50 fine-grained human queries with expert-reviewed candidate pools. These datasets were used not only for end-to-end comparisons, but also for ablations showing that removing citation expansion or RL training causes substantial recall drops (He et al., 17 Jan 2025).
PaperAsk focuses directly on reliability failures in LLM-mediated scholarly assistance. For paper discovery over 100 topic-plus-date queries from the PaSa benchmark, GPT-4o reached Recall 3, Precision 4, and 5; GPT-5 reached 6, 7, and 8; Gemini 2.5 reached 9, 0, and 1. The study reports that citation retrieval fails in 48–98% of multi-reference queries, section-specific content extraction fails in 72–91% of cases, and topical paper discovery yields 2 scores below 0.32, missing over 60% of relevant literature. Human analysis attributes these failures to uncontrolled expansion of retrieved context and the tendency of models to prioritize semantically relevant text over task instructions. The authors therefore trained lightweight reliability classifiers on PaperAsk data, reporting 96% accuracy and 3–0.97 across tasks (Wu et al., 25 Oct 2025).
A common misconception is that search-enabled LLM interfaces already provide dependable scholarly discovery. The benchmark evidence does not support that conclusion. Current systems show measurable gains over single-shot retrieval, but recall remains limited, constraint handling is fragile, and post-hoc reliability filtering is often necessary.
6. Open problems and research directions
Several problems recur across otherwise different systems. One is concept drift: Rivendell showed that life-time personalization degrades when researchers switch topics, whereas project-partitioned history recovers precision more effectively (Lazebnik et al., 2022). Another is granularity mismatch: PaperScout identified a mismatch between token-level RL and sequence-level multi-turn interaction, and PaperRegister showed that abstract-only indexes are inadequate for flexible-grained queries about module configuration or training operations (Pan et al., 15 Jan 2026, Li et al., 14 Aug 2025).
Constraint preservation is another persistent difficulty. ScholarQuest reports that scope-controlled queries are the hardest class, with mean Recall@100 for PaperScout around 0.18 on that intent type, and PaperAsk documents frequent date mismatch, topic mismatch, and source mismatch in black-box LLM search interfaces (Pan et al., 18 Jun 2026, Wu et al., 25 Oct 2025). This suggests that future paper-search systems will need more explicit negative-constraint handling, stronger intent-conditioned tool policies, and evidence-level reasoning modules rather than relying primarily on snippet semantics.
Coverage and data quality remain limiting factors in open infrastructures. Paperfetcher notes that Crossref deposition lag can hide newly online papers, that COCI updates lag behind Crossref, and that missing abstracts or references create holes in both handsearch and backward chaining (Pallath et al., 2021). GovScape flags pages without text for future OCR and proposes adding olmOCR for scanned pages, while SARCH identifies map text extraction as a bottleneck and recommends specialized text spotting or geolocation gazetteers (Deeds et al., 14 Nov 2025, Sinha et al., 7 Nov 2025).
Multilinguality, modality expansion, and schema induction are active frontiers. WisPaper already supports English and Chinese and plans additional languages, while GovScape proposes multilingual embeddings and extension to mixed Word and HTML corpora (Ju et al., 7 Dec 2025, Deeds et al., 14 Nov 2025). PaperRegister identifies manual schema design as a limitation and proposes automated schema induction, and ScholarQuest recommends symbolic constraint modules, intent-conditioned policies, and hybrid human–LLM validation for critical scopes (Li et al., 14 Aug 2025, Pan et al., 18 Jun 2026).
Overall, the literature indicates that paper-search is moving toward systems that are simultaneously more structured and more adaptive: structured in their indexes, operators, workflows, and evidence traces; adaptive in their use of search, expansion, filtering, reranking, and feedback. A plausible implication is that the next generation of paper-search systems will combine explicit semantic or workflow structure with controllable agentic policies, multimodal retrieval, and benchmarked reliability safeguards rather than treating literature discovery as a single ranked list returned from a single query.
