ResearchPulse: Multi-Document Inference

• ResearchPulse is an agent-based framework for multi-document scientific inference that reconstructs method–experiment chains from related papers.
• It employs coordinated agents (Plan, Mmap, Lchart) to systematically extract, align, and chronologically order motivations, methods, and experimental data.
• Coupled with ResearchPulse-Bench, it offers citation-aware clustering and visualization tools to capture research evolution and benchmark performance.

ResearchPulse is an agent-based framework for multi-document scientific inference that extracts and aligns motivation, methodology, and experimental results across thematically related papers in order to reconstruct method–experiment chains and broader research development chains (Chen et al., 3 Sep 2025). It is coupled to ResearchPulse-Bench, a citation-aware benchmark of annotated paper clusters, and is designed to address problems that single-paper summarization does not solve, such as temporal method alignment, cross-paper comparison of experimental tables, and structured visualization of how ideas and benchmark performance evolve over time (Chen et al., 3 Sep 2025).

1. Problem formulation and conceptual scope

ResearchPulse formalizes multi-document scientific inference over a collection of related papers

$\mathcal{C} = \{D_1, D_2, \dots, D_N\},$

where each $D_i$ is a scientific paper with sections such as abstract, introduction, methods, experiments, and references (Chen et al., 3 Sep 2025). The task is not merely to summarize these papers jointly, but to extract and align three specific scientific elements across documents: motivations, methods, and experimental results. The intended output is a time-ordered representation of scientific progress rather than a single compressed narrative (Chen et al., 3 Sep 2025).

The framework is instantiated through two complementary subtasks. Method-Tracking aligns motivations and methods across papers into a chronologically ordered chain. Experimental-Analysis aligns benchmark tables, metrics, and models into temporal performance trajectories (Chen et al., 3 Sep 2025). In the paper’s formulation, a method chain is represented as

$$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$

where $m_i$ is the core motivation, $r_i$ is the method description, and $t_i$ is the publication time of $D_i$ (Chen et al., 3 Sep 2025). Experimental chains are similarly represented by aligning extracted tables, model names, metrics, and baseline years: $$\mathcal{E}_{\text{chain}} = \mathrm{Align}\left(\{(T_i,\mathcal{M}_i,\mathcal{E}_i,\mathcal{Y}_i)\}_{i=1}^{N}\right).$$

This formulation addresses several constraints that the paper identifies as central. Single-paper summaries do not explicitly sort methods by publication time, do not map motivations to later methodological refinements, and do not normalize heterogeneous experimental tables across papers (Chen et al., 3 Sep 2025). ResearchPulse therefore treats literature understanding as a structured alignment problem over clusters of related documents rather than as a document-level summarization problem.

2. ResearchPulse-Bench and citation-aware clustering

ResearchPulse-Bench is the benchmark introduced to supervise and evaluate this task (Chen et al., 3 Sep 2025). Its construction begins with seed papers retrieved from Google Scholar using AI-related keywords such as machine learning, computer vision, and NLP, filtered by citation counts, publication time in the last five years, and open-access availability on arXiv or OpenReview (Chen et al., 3 Sep 2025). From each seed paper, the pipeline builds citation graphs using both forward and backward citations, so that clusters reflect not only semantic similarity but also citation lineage and publication chronology (Chen et al., 3 Sep 2025).

Each collected paper is converted from PDF to markdown, segmented into major sections, and then encoded using Sentence-BERT abstract embeddings before K-Means clustering groups papers by semantic similarity (Chen et al., 3 Sep 2025). Citation information is then used to ensure that clusters are both semantically related and citation-aware. This yields clusters intended to represent coherent research threads rather than arbitrary topical neighborhoods (Chen et al., 3 Sep 2025).

Within each cluster, strong LLMs such as DeepSeek-R1 or GPT-4o draft extractions of motivations, methods, experimental tables, model names $\mathcal{M}_i$, evaluation metrics $\mathcal{E}_i$, and baseline years $D_i$0, after which human annotators verify cluster coherence, chronological consistency, table correctness, and visualization fidelity (Chen et al., 3 Sep 2025). The benchmark therefore combines automated draft generation with manual validation rather than relying on raw LLM output alone.

The reported benchmark scale is substantial for long-context scientific inference. For Method-Tracking, the benchmark contains 1,958 train samples and 491 test samples, with average query lengths of 1,161.16 and 1,210.78 tokens and average answer lengths of 344.54 and 355.6 tokens for train and test respectively (Chen et al., 3 Sep 2025). For Experimental-Analysis, it contains 1,550 train samples and 320 test samples, with much longer average query lengths of 14,402.73 and 16,545.45 tokens and average answer lengths of 885.98 and 928 tokens (Chen et al., 3 Sep 2025). At the cluster level, ResearchPulse-Bench includes 100 citation-aware clusters, split into 80 train and 20 test clusters, with cluster sizes ranging from 14 to 38 papers (Chen et al., 3 Sep 2025).

3. Multi-agent architecture

ResearchPulse is organized around three coordinated agents: a Plan Agent, a Mmap-Agent, and an Lchart-Agent (Chen et al., 3 Sep 2025).

Agent	Base model	Primary function
Plan Agent	Qwen-72B	Instruction classification, extraction control, task routing
Mmap-Agent	Qwen2.5-7B-Instruct	Motivation–method extraction and mind-map chain construction
Lchart-Agent	Qwen2.5-Coder-7B-Instruct	Experimental table extraction, alignment, and line-chart code generation

The Plan Agent is the controller. It receives a user instruction $D_i$1, classifies it into a method-tracking intent $D_i$2 or an experimental-analysis intent $D_i$3, invokes an extraction module to parse each document $D_i$4 into structured content $D_i$5, and routes these structured inputs to the appropriate downstream agent (Chen et al., 3 Sep 2025). Formally, the extraction stage is written as

$D_i$6

The Mmap-Agent consumes $D_i$7 and extracts pairs

$D_i$8

where $D_i$9 is the core motivation and $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$0 is the methodological description (Chen et al., 3 Sep 2025). It then orders these pairs by publication time to form the method chain. Its training objective is an autoregressive conditional likelihood: $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$1

The Lchart-Agent consumes $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$2 and extracts structured experimental tuples

$$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$3

where $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$4 is the main table, $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$5 the set of model names, $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$6 the evaluation metrics, and $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$7 the baseline publication years (Chen et al., 3 Sep 2025). It aligns these fields across papers and generates Python plotting code for experimental line charts. Its training objective is

$$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$8

The architecture is explicitly modular. The Plan Agent handles instruction planning and routing, while the two 7B-scale agents specialize in scientific content extraction and structured output generation (Chen et al., 3 Sep 2025). The paper also reports ablations labeled “w/o GPT-4o” and “w/o Compiler,” indicating auxiliary modules that improve output quality and structural validity (Chen et al., 3 Sep 2025).

4. Extraction, alignment, and visualization

In ResearchPulse, method alignment and experiment alignment are parallel but distinct operations (Chen et al., 3 Sep 2025). For method-tracking, the input material is primarily abstract and introduction text, from which the Mmap-Agent extracts problem statements, gaps, goals, and methodological contributions. These are then sorted by $$\mathcal{M}_{\text{chain}} = \mathrm{Sort}_{t_i}\left(\{(m_i, r_i)\}_{i=1}^{N}\right),$$9, producing a chain that makes chronological development explicit (Chen et al., 3 Sep 2025). The alignment mechanism is therefore temporal and thematic rather than graph-theoretic: papers are already grouped in citation-aware topical clusters, and ordering them by publication time exposes the progression of ideas within a research line.

Experimental alignment is more heterogeneous. Papers may report different datasets, metrics, naming conventions, or table layouts, so the Lchart-Agent first identifies the main table $m_i$0, then extracts rows, model names, metrics, and baseline years through citation resolution (Chen et al., 3 Sep 2025). The alignment stage standardizes comparable metrics and orders baselines temporally, yielding a structure that can be rendered as a benchmark trajectory rather than as isolated tables (Chen et al., 3 Sep 2025).

These aligned structures are then visualized in two forms. Motivation–method mind maps use a hierarchical markdown representation in which higher-level nodes correspond to years or sub-problems and child nodes correspond to individual papers annotated by motivation and method summaries (Chen et al., 3 Sep 2025). Experimental line charts place time or baseline year on the $m_i$1-axis and performance metrics on the $m_i$2-axis, with distinct curves corresponding to model or method families (Chen et al., 3 Sep 2025). The plotting stage is explicitly code-generative: $m_i$3 where $m_i$4 is implemented by the Lchart-Agent as Python chart-generation code (Chen et al., 3 Sep 2025).

This design makes ResearchPulse both a scientific extraction system and a visualization system. The visual outputs are not secondary renderings of a summary; they are core artifacts of the task definition. A plausible implication is that the framework is intended for literature review, survey construction, and benchmark tracking as much as for generic document understanding.

5. Evaluation and empirical performance

The evaluation protocol separates textual and visual outputs (Chen et al., 3 Sep 2025). For Method-Tracking, the paper uses BERTScore precision, recall, and F1, METEOR, and a GPT-4o-based GPT-Score with dimensions of fluency, relevance, accuracy, creativity, and overall quality (Chen et al., 3 Sep 2025). For Experimental-Analysis, it evaluates generated charts with Inception Score, FID, KID, CLIP-FID, SSIM, MS-SSIM, PSNR, LPIPS, CMMD, and also measures pass@1, the percentage of generated Python chart scripts that execute without runtime errors (Chen et al., 3 Sep 2025).

On Method-Tracking, Mmap-Agent achieves the highest reported BERTScore F1 at 90.39, with BERTScore precision 90.90, recall 89.89, and METEOR 46.14 (Chen et al., 3 Sep 2025). Its GPT-4o quality score is 85.59, compared with 83.42 for GPT-4o and 84.01 for Claude-3.7-Sonnet (Chen et al., 3 Sep 2025). The corresponding ablations show substantial degradation: removing GPT-4o drops BERTScore F1 to 86.91, removing the Compiler drops it to 83.49, and removing both yields 82.83 (Chen et al., 3 Sep 2025). These results support the paper’s claim that specialized agent coordination and auxiliary validation improve semantic alignment and structural consistency.

On Experimental-Analysis, Lchart-Agent reaches pass@1 = 97.50, FID = 6.73, LPIPS = 8.49, CMMD = 4.06, SSIM = 55.56, PSNR = 12.49, and MS-SSIM = 36.55 (Chen et al., 3 Sep 2025). The paper reports that these values outperform GPT-4o and other baselines on most structural and perceptual metrics while also producing the highest executable-code success rate (Chen et al., 3 Sep 2025). Again, the ablations are informative: removing the Compiler reduces pass@1 to 90.63 and raises FID to 8.02, while removing both GPT-4o and the Compiler drops pass@1 to 90.00 and SSIM to 48.79 (Chen et al., 3 Sep 2025).

Across both subtasks, the study emphasizes that these gains are obtained with 7B-scale agents rather than very large monolithic models (Chen et al., 3 Sep 2025). The baselines include Qwen2.5-7B-Instruct, Qwen2.5-Coder-7B-Instruct, InternLM3-8B-Instruct, Llama-3.1-8B-Instruct, CodeLlama-7B-hf, GPT-4o, Claude-3.7-Sonnet, and Gemini-1.5 Pro (Chen et al., 3 Sep 2025). The empirical pattern therefore supports the claim that task-specific fine-tuning and role specialization can outperform larger general-purpose systems on structured scientific inference.

6. Limitations, failure modes, and position in research tooling

The paper reports several recurring failure modes (Chen et al., 3 Sep 2025). In method-tracking, the system can produce misaligned attribution, such as assigning “message passing between graph nodes” to LeCun’s 1989 CNN paper rather than to later GNN models (Chen et al., 3 Sep 2025). It can also omit central components of complex methods, for example by reducing “layer-aware semantic adaptive quantization” to a partial description that leaves out dynamic precision assignment or semantic-aware evaluation metrics (Chen et al., 3 Sep 2025). In experimental-analysis, the reported errors include incorrect Top-1 accuracy values, trend misreadings, and partial contextualization of baseline models (Chen et al., 3 Sep 2025).

The current domain coverage is also narrow. ResearchPulse-Bench focuses on AI-related literature, specifically ML, CV, and NLP clusters, so generalization to biology, physics, or other scientific domains remains untested (Chen et al., 3 Sep 2025). The paper further notes challenges from noisy tables, ambiguous or multi-layered motivations, and the lack of an explicit graph-reasoning component beyond citation-aware clustering and chronological ordering (Chen et al., 3 Sep 2025). Future directions include more explicit graph-based reasoning, broader domain coverage, greater robustness to noisy tables, and tighter integration with deep research agents that could use method–experiment chains to generate hypotheses or propose baselines (Chen et al., 3 Sep 2025).

Within the broader landscape of research tools, ResearchPulse occupies a distinct position. ResearchStudio-Reel treats the “last mile” of research dissemination as a composition of skills, using one shared extractor and editable generators for posters, videos, and blogs (Xiao et al., 5 Jul 2026). PolicyPulse is a GPT-4-based interactive system that synthesizes public experiences from Reddit into themes, subtopics, and quotes for policy researchers (Wang et al., 29 May 2025). ResearchPulse differs from both by centering not on dissemination artifacts or public-opinion synthesis, but on cross-document reconstruction of scientific development chains (Chen et al., 3 Sep 2025). This suggests a broader trend: agent-based systems are increasingly being organized around structured intermediate representations rather than single-shot summaries, with ResearchPulse contributing the specific case of temporally aligned, citation-aware scientific inference.