Paper Circle: Multi-Agent Research Platform

• Paper Circle is an open-source multi-agent framework that automates literature discovery and detailed paper analysis through natural language queries.
• It integrates multi-source retrieval, structured ranking, and the construction of typed knowledge graphs to streamline research workflows.
• The system employs coordinated agents and deterministic artifact generation to optimize retrieval, extraction, and graph-aware question answering.

Paper Circle is an open-source, coder-LLM–orchestrated, multi-agent framework for research discovery and paper analysis that turns natural-language queries into a reproducible workflow for retrieving, scoring, ranking, organizing, and analyzing academic literature. It is organized around two complementary pipelines: a Discovery Pipeline for multi-source literature retrieval and ranking, and an Analysis Pipeline for transforming individual papers into typed knowledge graphs that support graph-aware question answering, coverage verification, and synchronized exports in JSON, CSV, BibTeX, Markdown, and HTML formats (Kumar et al., 7 Apr 2026).

1. Definition and scope

Paper Circle addresses the problem that scientific literature volume makes it difficult to find, assess, organize, and understand relevant work. Its stated goal is to reduce researcher effort by combining multi-source retrieval, structured scoring and ranking, typed paper-level knowledge graphs, and reproducible step-level outputs in a single system (Kumar et al., 7 Apr 2026).

The system is positioned as a joint discovery-and-analysis environment rather than a single search interface or a pure retrieval-augmented generation tool. The Discovery Pipeline retrieves from offline corpora and live APIs, then deduplicates, scores, ranks, and exports candidate papers. The Analysis Pipeline ingests PDFs, performs staged extraction of concepts, methods, experiments, datasets, figures, tables, and equations, and builds a typed graph referred to as a “Paper Mind.” This graph is then used for graph-aware question answering and coverage checks. A plausible implication is that Paper Circle treats literature search and paper understanding as two coupled but separable computational problems: corpus-level selection and document-level structuring (Kumar et al., 7 Apr 2026).

The framework is described as distinctive in combining multi-agent orchestration, multi-source discovery, typed paper knowledge graphs, provenance, coverage verification, graph-aware QA, deterministic runs, and structured exports. This suggests that its principal contribution is systems integration rather than a single isolated retrieval or modeling primitive (Kumar et al., 7 Apr 2026).

2. Multi-agent orchestration and system architecture

Paper Circle uses a multi-agent orchestration baseline built on smolagents, with a CodeAgent (CoA) as the central planner and ToolCallingAgents (ToCA) bound to explicit tools such as arXiv retrieval and PDF parsing. The CodeAgent enforces a minimal-step policy, delegates to specialized agents, attends parallel tool calls, and chains steps through explicit tool input-output parameters. Coordination is described as iterative refinement of an evolving discovery state through “noising and denoising operations,” with step counters, metadata logging, and synchronized artifact regeneration at every stage (Kumar et al., 7 Apr 2026).

The main agent roles can be summarized as follows:

Agent	Function	Outputs
Intent Classification Agent	Parses query constraints and mode	Structured search specification
Paper Search Agent	Executes offline/online retrieval and deduplication	Candidate paper set
Sorting Agent	Reorders by selected criteria or reranking	Ranked list
Analysis Agent	Computes aggregate discovery analytics	Stats and insights
Export Agent	Synchronizes artifact generation	JSON, CSV, BibTeX, Markdown, HTML
Paper Analysis Orchestrator	Coordinates concept/method/experiment/linkage extraction	Typed knowledge graph

The architecture includes a web UI and a FastAPI backend. State is persisted in a central PipelineState, and each step logs agent name, parameters, timestamps, previews of results, and synchronized outputs. Parallelization is used both for online API aggregation and for review-oriented agents through ThreadPoolExecutor. This operational design indicates that reproducibility is treated as a first-class systems property rather than a post hoc export feature (Kumar et al., 7 Apr 2026).

3. Discovery Pipeline

The Discovery Pipeline supports both offline and online retrieval. Offline search uses a local JSON corpus with structured metadata fields such as title, authors, abstract, venue, year, track, keywords, and DOI. It uses BM25 indexing via rank_bm25, with optional semantic similarity from sentence-transformers and optional hybrid retrieval. Online retrieval is executed in parallel against the arXiv API, Semantic Scholar API, OpenAlex, and DBLP, after which all results are normalized into a common paper structure and deduplicated by DOI and normalized titles (Kumar et al., 7 Apr 2026).

Paper Circle scores papers with multiple criteria. The similarity score is defined by TF–IDF cosine over concatenated title and abstract,

$$\text{similarity}(q, p) = \frac{\vec{v}_q \cdot \vec{v}_p}{\|\vec{v}_q\| \cdot \|\vec{v}_p\|}$$

and recency is normalized by the observed year range,

$$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$

The combined score is

$$\text{combined}(p) = w_s \cdot \text{similarity} + w_r \cdot \text{recency} + w_n \cdot \text{novelty} + w_b \cdot \text{bm25}.$$

Novelty is defined as TF–IDF distance from the corpus centroid, and BM25 is normalized to $[0,1]$. Three explicit mode-dependent weightings are reported: Stable $(w_s=0.5, w_r=0.2, w_n=0.1, w_b=0.2)$, Discovery $(0.3,0.1,0.4,0.2)$, and Balanced $(0.3,0.2,0.2,0.3)$ (Kumar et al., 7 Apr 2026).

Diversity-aware postprocessing uses maximal marginal relevance,

$$\text{MMR} = \arg\max_{p \in R \setminus S} \left[ \lambda \cdot \text{sim}(p, q) - (1-\lambda) \cdot \max_{s \in S} \text{sim}(p, s) \right],$$

with $\lambda=0.8$ for Stable, $0.5$ for Discovery, and $$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$0 for Balanced. The pipeline also generates secondary views such as “Hidden Gems,” “Canonical Papers,” and source distribution. This suggests that Paper Circle is designed to balance relevance, novelty, and source diversity explicitly rather than relying on a single retrieval score (Kumar et al., 7 Apr 2026).

At each agent step, the pipeline regenerates synchronized artifacts including papers.json, links.json, stats.json, summary.json, retrieval_metrics.json, CSV, BibTeX, Markdown, and an HTML dashboard. A representative papers.json entry includes fields such as id, title, authors, abstract, venue, year, keywords, doi, arxiv_id, source, pdf_url, and nested scores for similarity, recency, novelty, BM25, and combined score (Kumar et al., 7 Apr 2026).

4. Analysis Pipeline and typed knowledge graphs

The Analysis Pipeline begins with PDF ingestion via PyMuPDF, which extracts metadata, section structure, figures, tables, equations, and local context. It then performs semantic chunking aligned to document structure, with figures, tables, and equations retained as distinct chunks. On top of these chunks, Paper Circle builds a typed graph called MindGraph whose node types are paper, section, concept, method, experiment, dataset, figure, table, and equation, and whose edge types include hierarchy, definition, proposal, usage, evaluation, illustration, dependency, and inter-concept relations (Kumar et al., 7 Apr 2026).

Extraction is staged. The Concept Extractor identifies concept name, description, type, and importance. The Method Extractor targets method-, approach-, architecture-, and algorithm-oriented sections and outputs name, description, category, and key steps. The Experiment Extractor produces setups, datasets, metrics, and results, while the Linkage Agent connects figures and tables to concepts and methods through relations such as illustration or comparison. Every node and edge carries provenance, including source chunk IDs, page numbers, verification status, confidence, and timestamps. This provenance model suggests that Paper Circle’s graph is intended not merely as a semantic summary but as an auditable intermediate representation (Kumar et al., 7 Apr 2026).

Graph-aware question answering is implemented through an EmbeddingStore over text chunks and node descriptions. Retrieval proceeds by selecting top-$$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$1 similar chunks and nodes, then expanding through 1-hop graph neighbors before constructing the answering context. CoverageChecker measures figure coverage, table coverage, section coverage, and equation coverage, and outputs an overall coverage score, lists of unlinked items, and critical issues such as “No figures linked.” A plausible implication is that the system treats incompleteness in extraction as a measurable property rather than an implicit failure mode (Kumar et al., 7 Apr 2026).

The framework also includes review-oriented agents, including a deep analyzer, critic, literature expert, contribution analyzer, reproducibility checker, and summarizer. These are coordinated by an orchestrator that aggregates their outputs into a final structured report. In system terms, Paper Circle therefore extends beyond retrieval and extraction into automated review synthesis, although the paper notes limits in alignment with human evaluations (Kumar et al., 7 Apr 2026).

5. Evaluation and empirical results

Paper Circle is evaluated on both retrieval and review generation. The offline corpus is curated from major CS and ML venues, including ICLR, NeurIPS, ICML, CVPR, IROS, ICRA, AAAI, ACL, ICCV, and EMNLP. The paper reports 21,115 papers processed and 18,613 duplicates removed. Retrieval evaluation uses SemanticBench with 50 queries and RAbench with 500 queries. Reported metrics include Hit Rate, MRR, Recall@K, Precision@K, Success Rate, Time per query, and Steps (Kumar et al., 7 Apr 2026).

The paper defines MRR as

$$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$2

and Recall@K as

$$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$3

On the 50-query SemanticBench benchmark, Qwen3-Coder-30B Inst Q3_K_M achieves Hit Rate 0.80, MRR 0.627, R@1 0.58, R@5 0.66, with time approximately 22.2 s and steps approximately 1.42. The BM25 baseline is reported at Hit Rate 0.78 and MRR 0.541. On RAbench with 500 queries, Qwen3-Coder-30B-A3B-Inst-Q3_K_M achieves Hit Rate 0.98, MRR 0.882, R@1 0.83, R@5 0.93, with time approximately 21.53 s (Kumar et al., 7 Apr 2026).

Configuration ablations attribute strong performance to the full agentic pipeline and structured constraints. The Default (Full Agent) setup on 500 queries reports HR 0.9818, MRR 0.8824, R@1 0.8381, and time approximately 21.54 s. Offline Only degrades to HR 0.92 and MRR 0.6476; No Mentions drops to HR 0.64 and MRR 0.4316. BM25 plus reranking reaches MRR 0.8692 and R@5 0.94 but at approximately 935 s per query, which the paper characterizes as a major latency cost (Kumar et al., 7 Apr 2026).

Real-world deployment statistics are also reported. Across 81 sessions and 78 queries, Paper Circle queried 8.7 sources per run on average, and the fraction of Paper Circle’s retrieved set not retrievable from single sources is reported as 70.9% for arXiv and 80.4% for Semantic Scholar. Median runtime per session is 2.3 minutes. For review generation on 50 ICLR 2024 papers, larger chat-style LLMs produce stronger and more consistent reviews than code-oriented models, but correlation with human ratings remains weak, with $$\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.$$4 (Kumar et al., 7 Apr 2026).

6. Reproducibility, implementation, and limitations

A defining feature of Paper Circle is deterministic, synchronized artifact generation. The central PipelineState records shared state, step counters, timestamps, parameters, previews, and regenerated exports after each operation. Offline searches are deterministic for a fixed corpus, and both sorting and MMR use explicit recorded parameters. In the analysis pipeline, provenance and verification status are propagated to exports, including graph JSON, Markdown, Mermaid, and HTML. This suggests that reproducibility is encoded at both the workflow and representation levels (Kumar et al., 7 Apr 2026).

The implementation stack includes Python and FastAPI on the backend; scikit-learn, NumPy, and pandas for data processing; rank_bm25 and sentence-transformers for retrieval; PyMuPDF for PDF parsing; React, TypeScript, Vite, and TailwindCSS on the frontend; and Supabase for storage. Models include multiple Qwen, DeepSeek-Coder, Granite-code, and gpt-oss variants served through Ollama and fastllm. The repository is publicly available at https://github.com/MAXNORM8650/papercircle, and the web interface is at https://papercircle.vercel.app/ (Kumar et al., 7 Apr 2026).

The paper also identifies several limitations. Review-agent alignment with human judgment is weak, metadata quality across sources is inconsistent, reranking can be prohibitively expensive, and retrieval quality degrades when structured constraints are removed. Mitigations include multi-source aggregation, DOI/title normalization, deterministic offline scoring, explicit logging, coverage verification, and human-in-the-loop operations such as verify_node, edit_node, add/remove_edge, and flag_for_review. A plausible implication is that the framework is intended to support mixed autonomous and supervised scholarly workflows rather than fully unattended literature evaluation (Kumar et al., 7 Apr 2026).

Future work is described in terms of improving orchestration efficiency, reranking cost–accuracy tradeoffs, knowledge graph coverage, QA reliability, and community-oriented “Reading Circle” collaboration and ranking. Within that trajectory, Paper Circle can be understood as a research infrastructure system for computational literature work: it unifies retrieval, ranking, extraction, graph construction, QA, auditing, and export into a single reproducible pipeline (Kumar et al., 7 Apr 2026).