---
title: 'Paper Circle: Multi-Agent Research Platform'
url: https://www.emergentmind.com/topics/paper-circle
type: topic
---

# Paper Circle: Multi-Agent Research Platform

Paper Circle is an open-source, coder-LLM–orchestrated, multi-agent framework for research discovery and paper analysis that turns natural-language queries into a reproducible workflow for retrieving, scoring, ranking, organizing, and analyzing academic literature. It is organized around two complementary pipelines: a Discovery Pipeline for multi-source literature retrieval and ranking, and an Analysis Pipeline for transforming individual papers into typed knowledge graphs that support graph-aware question answering, coverage verification, and synchronized exports in JSON, CSV, BibTeX, Markdown, and HTML formats [2604.06170].

## 1. Definition and scope

Paper Circle addresses the problem that scientific literature volume makes it difficult to find, assess, organize, and understand relevant work. Its stated goal is to reduce researcher effort by combining multi-source retrieval, structured scoring and ranking, typed paper-level knowledge graphs, and reproducible step-level outputs in a single system [2604.06170].

The system is positioned as a joint discovery-and-analysis environment rather than a single search interface or a pure retrieval-augmented generation tool. The Discovery Pipeline retrieves from offline corpora and live APIs, then deduplicates, scores, ranks, and exports candidate papers. The Analysis Pipeline ingests PDFs, performs staged extraction of concepts, methods, experiments, datasets, figures, tables, and equations, and builds a typed graph referred to as a “Paper Mind.” This graph is then used for graph-aware question answering and coverage checks. A plausible implication is that Paper Circle treats literature search and paper understanding as two coupled but separable computational problems: corpus-level selection and document-level structuring [2604.06170].

The framework is described as distinctive in combining multi-agent orchestration, multi-source discovery, typed paper knowledge graphs, provenance, coverage verification, graph-aware QA, deterministic runs, and structured exports. This suggests that its principal contribution is systems integration rather than a single isolated retrieval or modeling primitive [2604.06170].

## 2. Multi-agent orchestration and system architecture

Paper Circle uses a multi-agent orchestration baseline built on **smolagents**, with a **CodeAgent (CoA)** as the central planner and **ToolCallingAgents (ToCA)** bound to explicit tools such as arXiv retrieval and PDF parsing. The CodeAgent enforces a minimal-step policy, delegates to specialized agents, attends parallel tool calls, and chains steps through explicit tool input-output parameters. Coordination is described as iterative refinement of an evolving discovery state through “noising and denoising operations,” with step counters, metadata logging, and synchronized artifact regeneration at every stage [2604.06170].

The main agent roles can be summarized as follows:

| Agent | Function | Outputs |
|---|---|---|
| Intent Classification Agent | Parses query constraints and mode | Structured search specification |
| Paper Search Agent | Executes offline/online retrieval and deduplication | Candidate paper set |
| Sorting Agent | Reorders by selected criteria or reranking | Ranked list |
| Analysis Agent | Computes aggregate discovery analytics | Stats and insights |
| Export Agent | Synchronizes artifact generation | JSON, CSV, BibTeX, Markdown, HTML |
| Paper Analysis Orchestrator | Coordinates concept/method/experiment/linkage extraction | Typed knowledge graph |

The architecture includes a web UI and a FastAPI backend. State is persisted in a central `PipelineState`, and each step logs agent name, parameters, timestamps, previews of results, and synchronized outputs. Parallelization is used both for online API aggregation and for review-oriented agents through `ThreadPoolExecutor`. This operational design indicates that reproducibility is treated as a first-class systems property rather than a post hoc export feature [2604.06170].

## 3. Discovery Pipeline

The Discovery Pipeline supports both offline and online retrieval. Offline search uses a local JSON corpus with structured metadata fields such as title, authors, abstract, venue, year, track, keywords, and DOI. It uses BM25 indexing via `rank_bm25`, with optional semantic similarity from sentence-transformers and optional hybrid retrieval. Online retrieval is executed in parallel against the arXiv API, Semantic Scholar API, OpenAlex, and DBLP, after which all results are normalized into a common paper structure and deduplicated by DOI and normalized titles [2604.06170].

Paper Circle scores papers with multiple criteria. The similarity score is defined by TF–IDF cosine over concatenated title and abstract,
\[
\text{similarity}(q, p) = \frac{\vec{v}_q \cdot \vec{v}_p}{\|\vec{v}_q\| \cdot \|\vec{v}_p\|}
\]
and recency is normalized by the observed year range,
\[
\text{recency}(p) = \frac{\text{year}(p) - \text{year}_{\min}}{\text{year}_{\max} - \text{year}_{\min}}.
\]
The combined score is
\[
\text{combined}(p) = w_s \cdot \text{similarity} + w_r \cdot \text{recency} + w_n \cdot \text{novelty} + w_b \cdot \text{bm25}.
\]
Novelty is defined as TF–IDF distance from the corpus centroid, and BM25 is normalized to \([0,1]\). Three explicit mode-dependent weightings are reported: **Stable** \((w_s=0.5, w_r=0.2, w_n=0.1, w_b=0.2)\), **Discovery** \((0.3,0.1,0.4,0.2)\), and **Balanced** \((0.3,0.2,0.2,0.3)\) [2604.06170].

Diversity-aware postprocessing uses maximal marginal relevance,
\[
\text{MMR} = \arg\max_{p \in R \setminus S} \left[ \lambda \cdot \text{sim}(p, q) - (1-\lambda) \cdot \max_{s \in S} \text{sim}(p, s) \right],
\]
with \(\lambda=0.8\) for Stable, \(0.5\) for Discovery, and \(0.65\) for Balanced. The pipeline also generates secondary views such as “Hidden Gems,” “Canonical Papers,” and source distribution. This suggests that Paper Circle is designed to balance relevance, novelty, and source diversity explicitly rather than relying on a single retrieval score [2604.06170].

At each agent step, the pipeline regenerates synchronized artifacts including `papers.json`, `links.json`, `stats.json`, `summary.json`, `retrieval_metrics.json`, CSV, BibTeX, Markdown, and an HTML dashboard. A representative `papers.json` entry includes fields such as `id`, `title`, `authors`, `abstract`, `venue`, `year`, `keywords`, `doi`, `arxiv_id`, `source`, `pdf_url`, and nested `scores` for similarity, recency, novelty, BM25, and combined score [2604.06170].

## 4. Analysis Pipeline and typed knowledge graphs

The Analysis Pipeline begins with PDF ingestion via PyMuPDF, which extracts metadata, section structure, figures, tables, equations, and local context. It then performs semantic chunking aligned to document structure, with figures, tables, and equations retained as distinct chunks. On top of these chunks, Paper Circle builds a typed graph called **MindGraph** whose node types are **paper, section, concept, method, experiment, dataset, figure, table,** and **equation**, and whose edge types include **hierarchy, definition, proposal, usage, evaluation, illustration, dependency,** and inter-concept relations [2604.06170].

Extraction is staged. The Concept Extractor identifies concept name, description, type, and importance. The Method Extractor targets method-, approach-, architecture-, and algorithm-oriented sections and outputs name, description, category, and key steps. The Experiment Extractor produces setups, datasets, metrics, and results, while the Linkage Agent connects figures and tables to concepts and methods through relations such as illustration or comparison. Every node and edge carries provenance, including source chunk IDs, page numbers, verification status, confidence, and timestamps. This provenance model suggests that Paper Circle’s graph is intended not merely as a semantic summary but as an auditable intermediate representation [2604.06170].

Graph-aware question answering is implemented through an EmbeddingStore over text chunks and node descriptions. Retrieval proceeds by selecting top-\(k\) similar chunks and nodes, then expanding through 1-hop graph neighbors before constructing the answering context. CoverageChecker measures figure coverage, table coverage, section coverage, and equation coverage, and outputs an overall coverage score, lists of unlinked items, and critical issues such as “No figures linked.” A plausible implication is that the system treats incompleteness in extraction as a measurable property rather than an implicit failure mode [2604.06170].

The framework also includes review-oriented agents, including a deep analyzer, critic, literature expert, contribution analyzer, reproducibility checker, and summarizer. These are coordinated by an orchestrator that aggregates their outputs into a final structured report. In system terms, Paper Circle therefore extends beyond retrieval and extraction into automated review synthesis, although the paper notes limits in alignment with human evaluations [2604.06170].

## 5. Evaluation and empirical results

Paper Circle is evaluated on both retrieval and review generation. The offline corpus is curated from major CS and ML venues, including ICLR, NeurIPS, ICML, CVPR, IROS, ICRA, AAAI, ACL, ICCV, and EMNLP. The paper reports **21,115** papers processed and **18,613** duplicates removed. Retrieval evaluation uses **SemanticBench** with **50** queries and **RAbench** with **500** queries. Reported metrics include Hit Rate, MRR, Recall@K, Precision@K, Success Rate, Time per query, and Steps [2604.06170].

The paper defines MRR as
\[
\text{MRR} = \frac{1}{|Q|} \sum_{q \in Q} \frac{1}{\text{rank}_q}
\]
and Recall@K as
\[
\text{Recall@K} = \frac{\#\text{ relevant items retrieved in top } K}{\#\text{ relevant items}}.
\]

On the 50-query SemanticBench benchmark, **Qwen3-Coder-30B Inst Q3\_K\_M** achieves **Hit Rate 0.80**, **MRR 0.627**, **R@1 0.58**, **R@5 0.66**, with time approximately **22.2 s** and steps approximately **1.42**. The **BM25 baseline** is reported at **Hit Rate 0.78** and **MRR 0.541**. On RAbench with 500 queries, **Qwen3-Coder-30B-A3B-Inst-Q3\_K\_M** achieves **Hit Rate 0.98**, **MRR 0.882**, **R@1 0.83**, **R@5 0.93**, with time approximately **21.53 s** [2604.06170].

Configuration ablations attribute strong performance to the full agentic pipeline and structured constraints. The **Default (Full Agent)** setup on 500 queries reports **HR 0.9818**, **MRR 0.8824**, **R@1 0.8381**, and time approximately **21.54 s**. **Offline Only** degrades to **HR 0.92** and **MRR 0.6476**; **No Mentions** drops to **HR 0.64** and **MRR 0.4316**. BM25 plus reranking reaches **MRR 0.8692** and **R@5 0.94** but at approximately **935 s per query**, which the paper characterizes as a major latency cost [2604.06170].

Real-world deployment statistics are also reported. Across **81 sessions** and **78 queries**, Paper Circle queried **8.7 sources per run** on average, and the fraction of Paper Circle’s retrieved set not retrievable from single sources is reported as **70.9%** for arXiv and **80.4%** for Semantic Scholar. Median runtime per session is **2.3 minutes**. For review generation on **50 ICLR 2024 papers**, larger chat-style LLMs produce stronger and more consistent reviews than code-oriented models, but correlation with human ratings remains weak, with \(|r| < 0.25\) [2604.06170].

## 6. Reproducibility, implementation, and limitations

A defining feature of Paper Circle is deterministic, synchronized artifact generation. The central `PipelineState` records shared state, step counters, timestamps, parameters, previews, and regenerated exports after each operation. Offline searches are deterministic for a fixed corpus, and both sorting and MMR use explicit recorded parameters. In the analysis pipeline, provenance and verification status are propagated to exports, including graph JSON, Markdown, Mermaid, and HTML. This suggests that reproducibility is encoded at both the workflow and representation levels [2604.06170].

The implementation stack includes Python and FastAPI on the backend; scikit-learn, NumPy, and pandas for data processing; `rank_bm25` and sentence-transformers for retrieval; PyMuPDF for PDF parsing; React, TypeScript, Vite, and TailwindCSS on the frontend; and Supabase for storage. Models include multiple Qwen, DeepSeek-Coder, Granite-code, and `gpt-oss` variants served through Ollama and fastllm. The repository is publicly available at `https://github.com/MAXNORM8650/papercircle`, and the web interface is at `https://papercircle.vercel.app/` [2604.06170].

The paper also identifies several limitations. Review-agent alignment with human judgment is weak, metadata quality across sources is inconsistent, reranking can be prohibitively expensive, and retrieval quality degrades when structured constraints are removed. Mitigations include multi-source aggregation, DOI/title normalization, deterministic offline scoring, explicit logging, coverage verification, and human-in-the-loop operations such as `verify_node`, `edit_node`, `add/remove_edge`, and `flag_for_review`. A plausible implication is that the framework is intended to support mixed autonomous and supervised scholarly workflows rather than fully unattended literature evaluation [2604.06170].

Future work is described in terms of improving orchestration efficiency, reranking cost–accuracy tradeoffs, knowledge graph coverage, QA reliability, and community-oriented “Reading Circle” collaboration and ranking. Within that trajectory, Paper Circle can be understood as a research infrastructure system for computational literature work: it unifies retrieval, ranking, extraction, graph construction, QA, auditing, and export into a single reproducible pipeline [2604.06170].

Source: https://www.emergentmind.com/topics/paper-circle