Multi-Document Scientific Inference

• Multi-document scientific inference is a structured process of retrieving, aligning, and reasoning over evidence distributed across research papers and modalities.
• It leverages techniques like knowledge graph integration, PICO extraction, and multimodal cues to reconcile conflicting data and extract key scientific elements.
• Recent approaches integrate retrieval, formal reasoning, and cross-document synthesis to generate coherent summaries and support complex evidence-based analysis.

Multi-document scientific inference is the end-to-end process by which a system retrieves, aligns, and reasons over dispersed evidence spread across multiple scientific papers and modalities, including text, tables, figures, formulas, and algorithms (Xiong et al., 13 Apr 2026). In the recent literature, the term covers several closely related formulations: generating related-work paragraphs from cited abstracts, synthesizing findings across medical studies, answering multi-hop questions over multimodal document collections, inferring cross-document coreference and hierarchy among technical concepts, and reconstructing motivation–method–experiment chains across research clusters (Lu et al., 2020, DeYoung et al., 2021, Park et al., 20 Aug 2025, Forer et al., 2024, Chen et al., 3 Sep 2025). A recurring premise is that scientific reasoning rarely reduces to single-document compression: key evidence is distributed across papers, technical terminology varies across authors and venues, and contradictory or only partially overlapping findings must often be reconciled before a coherent claim can be produced (DeYoung et al., 2021, Wang et al., 2022).

1. Conceptual scope and distinctions

Multi-document scientific inference is defined in several papers as a structured, cross-document reasoning problem rather than a purely stylistic summarization problem. ResearchPulse formalizes it as the task of extracting core scientific elements—motivation, method, experiment, and results—from a set of related papers and aligning these elements over time to reconstruct development chains and benchmark trajectories (Chen et al., 3 Sep 2025). PaperScope frames it as a multi-source, multi-modal, multi-step reasoning problem grounded in realistic research settings, with explicit dependence on visual anchors such as charts, figures, algorithm blocks, and formulas (Xiong et al., 13 Apr 2026). DocHop-QA describes it as question answering that requires integrating dispersed, complementary evidence residing in multiple scientific documents and formats, including textual passages, tables, and layout-aware signals (Park et al., 20 Aug 2025).

Several works distinguish this setting from adjacent tasks. ResearchPulse states that it goes beyond single-paper summarization, multi-document summarization, knowledge graph construction, and citation-based synthesis because it emphasizes structured scientific logic and temporal alignment rather than a flattened thematic overview (Chen et al., 3 Sep 2025). MS2 makes a related point in the biomedical setting: in medicine, “summarization” is not just compression—it is inference across multiple documents, often under contradictory evidence and heterogeneous study designs (DeYoung et al., 2021). M3SciQA further sharpens the distinction by requiring a model to interpret a figure or table in an anchor paper, rank the correct cited paper among the anchor’s references, and then answer a detailed question from the referenced paper (Li et al., 2024).

This body of work also rejects the assumption that scientific evidence is predominantly textual. PaperScope intentionally places key evidentiary cues in visual modalities and reports that text-only ablations exhibit a dramatic performance drop, confirming that multi-modal integration is required (Xiong et al., 13 Apr 2026). DocHop-QA likewise integrates tables and structural layout cues, while M3SciQA stores tables as images to preserve format and evaluates figure- and table-grounded citation retrieval (Park et al., 20 Aug 2025, Li et al., 2024). A plausible implication is that multi-document scientific inference is increasingly treated as a joint problem of retrieval, alignment, grounding, and reasoning rather than as a downstream text generation task alone.

2. Task formulations and reasoning regimes

A major formulation is multi-document scientific summarization. KGSum treats Multi-Document Scientific Summarization as generating a coherent, concise summary for a cluster of topic-relevant scientific papers, with a unified knowledge graph $G$ built from all documents in the cluster (Wang et al., 2022). Its formal objective is

$y^* = \arg\max_y P(y \mid D, G),$

and the model factorizes generation through an intermediate graph-descriptive target, KGtext:

$$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$

This formulation makes cross-paper relationships explicit during both encoding and decoding (Wang et al., 2022).

In biomedical evidence synthesis, MS2 defines two complementary tasks. The first is texts-to-text summarization: given a review background statement and all cited study abstracts, generate the review’s overall findings statement. The second is table-to-table structured inference: given PICO-derived intervention/outcome pairs, infer an Evidence Inference direction label among increases, no_change, and decreases (DeYoung et al., 2021). The paper reports that only 53.8% of study-level EI classifications match the review-level EI classification, making contradiction handling a central concern rather than an edge case (DeYoung et al., 2021).

Question answering introduces a different decomposition. AIM-SciQA operationalizes inter-document multi-hop QA as a two-step chain: identify a target paper and then integrate facts across source and target documents (Lee et al., 15 Mar 2026). DocHop-QA broadens this into four tasks: BBox Entity Index Extraction, XML Entity Index Extraction, Structured Generative Answering, and Generative Text Extraction, spanning multi-label prediction and free-form generation over multiple documents and modalities (Park et al., 20 Aug 2025). M3SciQA explicitly separates the workflow into a visual-context retrieval stage and a reference-based QA stage, reflecting the way researchers move from an anchor figure or table to the cited literature (Li et al., 2024).

Controlled reasoning benchmarks make the decomposition even more explicit. SciR separates what must be extracted from realistic scientific discourse from what must be reasoned over once the relevant premises have been found (Beckmann et al., 11 Jun 2026). It instantiates three paradigmatic forms of inference—deduction, induction, and causal abduction—and varies two difficulty axes independently: principled inference complexity and premise obfuscation (Beckmann et al., 11 Jun 2026). This suggests that “multi-document scientific inference” can denote not only a family of downstream applications, but also an experimental framework for disentangling extraction difficulty from formal reasoning difficulty.

3. Representations, alignment structures, and model architectures

Knowledge-graph-centric modeling is one prominent strategy. KGSum constructs a unified knowledge graph from all documents in the input corpus using DYGIE++ for within-document entities, relations, and coreference clusters (Wang et al., 2022). The graph uses six entity types—Task, Method, Metric, Material, Generic, OtherScientificTerm—and seven relation types—Compare, Used-for, Feature-of, Hyponym-of, Evaluate-for, Part-of, Conjunction (Wang et al., 2022). Per-paper graphs are fused into a single graph so that shared entities align different documents, and Levi transformation converts labeled edges into an unlabeled directed graph by treating relations as nodes (Wang et al., 2022). Two graph-based encoding modules then propagate information: a Graph Updater over the corpus-level graph and an Entity-Sentence Updater over an entity–sentence heterogeneous graph, followed by a two-stage decoder that first generates KGtext and then the final summary (Wang et al., 2022).

Clinical structure provides another alignment substrate. MS2 marks Population, Intervention, and Outcome spans with special tokens and augments them with Evidence Inference direction labels and supporting sentences (DeYoung et al., 2021). Inputs may include <background>, <study>, PICO tags, and <evidence> spans, while downstream evaluation uses both lexical overlap and polarity-sensitive EI metrics (DeYoung et al., 2021). This representation is designed to expose contradictions and to reason over intervention–outcome tuples rather than over undifferentiated text (DeYoung et al., 2021).

Cross-document concept reasoning motivates a different representation again. SciCo-Radar addresses Hierarchical Cross-Document Coreference Resolution by retrieving full-text scientific literature to generate context-dependent singleton definitions $D(m)$ for individual mentions and relational definitions $R(m_i,m_j)$ for mention pairs (Forer et al., 2024). Pairwise relations are then classified into four classes—co-reference, $m_1 \rightarrow m_2$, $m_2 \rightarrow m_1$, and none—while an efficient re-ranking stage reduces the cost of relational-definition generation from $O(|M|^2)$ to $O(|M|\cdot K)$ (Forer et al., 2024). In this setting, definition generation is not ancillary metadata; it is the main mechanism for disambiguating long-tail technical concepts and inferring hierarchy across papers (Forer et al., 2024).

Agentic systems extend alignment from symbolic structures to workflow decomposition. ResearchPulse uses a Plan Agent for task decomposition, a Mmap-Agent for motivation–method extraction, and an Lchart-Agent for experimental extraction, normalization, and chart generation (Chen et al., 3 Sep 2025). PaperScope provides Local FileSearch and Local FileVisit tools inside a unified multi-modal environment, while its benchmark construction relies on a heterogeneous knowledge graph whose nodes include title, research background, classification tags, key contributions, methodology, datasets, results, metrics, formulas, algorithms, figures, tables, and limitations (Xiong et al., 13 Apr 2026). AIM-SciQA, by contrast, constructs cross-document relations through section-level aggregations of question embeddings, embedding cosine similarity, and citation guidance when available (Lee et al., 15 Mar 2026). Across these works, the common design pattern is that cross-paper reasoning depends on an intermediate alignment layer—knowledge graphs, PICO tuples, definition graphs, citation-aware clusters, or question-aligned sections—rather than on raw concatenation alone.

4. Benchmark and dataset ecosystem

The benchmark landscape is heterogeneous because the field spans summarization, evidence synthesis, QA, controllable reasoning, and research-development reconstruction. The following resources are repeatedly used to operationalize multi-document scientific inference.

Benchmark	Primary formulation	Reported scale
Multi-XScience (Lu et al., 2020)	related-work paragraph generation from a query abstract and cited-paper abstracts	30,369 / 5,066 / 5,093 examples; average 4.42 references; average input length 778.08 tokens; average target length 116.44
MS2 (DeYoung et al., 2021)	medical review finding generation and PICO+EI structured inference	over 470k documents and 20k summaries; median 17 studies per review
ResearchPulse-Bench (Chen et al., 3 Sep 2025)	method-tracking and experimental-analysis over citation-aware clusters	100 clusters; papers per cluster average approximately 24.93
M3SciQA (Li et al., 2024)	multi-modal retrieval plus reference-based QA over citation clusters	1,452 expert-annotated questions; 70 paper clusters; 3,066 papers
DocHop-QA (Park et al., 20 Aug 2025)	multimodal, multi-document, multi-hop QA	11,379 QA instances; 9,250 unique documents
PaperScope (Xiong et al., 13 Apr 2026)	agentic deep research across multi-modal paper collections	2,400 QA pairs; built on a knowledge graph of over 2,000 AI papers

These datasets stress different properties of the problem. Multi-XScience is explicitly “extreme summarization”: its targets are short, highly abstractive related-work paragraphs, and the dataset reports very high target novelty—42.33% novel unigrams, 81.75% novel bigrams, 94.57% novel trigrams, and 97.62% novel 4-grams (Lu et al., 2020). MS2 is organized around systematic-review style evidence synthesis and contradiction analysis in medicine (DeYoung et al., 2021). M3SciQA and DocHop-QA require multimodal reading, but M3SciQA is citation-cluster grounded whereas DocHop-QA does not rely on explicitly hyperlinked documents and instead uses semantic similarity and layout-aware evidence synthesis (Li et al., 2024, Park et al., 20 Aug 2025). PaperScope evaluates full agentic pipelines over large local corpora, while SciR is a generator rather than a fixed train/dev/test benchmark and uses controllable rendering to preserve mechanistic ground truth under multi-document scientific discourse (Xiong et al., 13 Apr 2026, Beckmann et al., 11 Jun 2026).

A further distinction concerns evidence density and dependency depth. PaperScope reports that 1,400 of its questions require 5+ supporting documents, underscoring deep cross-paper synthesis (Xiong et al., 13 Apr 2026). AIM-SciQA, in contrast, fixes hop length at two by design—first locate the target paper, then integrate source–target evidence—so that retrieval and reasoning can be evaluated separately under Realistic and Oracle conditions (Lee et al., 15 Mar 2026). This suggests that current datasets probe different slices of the broader inference problem: some emphasize abstraction, some contradiction, some multimodal grounding, and some formal reasoning under controlled obfuscation.

5. Evaluation protocols and empirical findings

Evaluation has moved beyond ROUGE-only comparison. KGSum is still assessed with ROUGE-1/2/L F1 on Multi-XScience, where it reports $35.77 / 7.51 / 31.43$ and outperforms Pointer-Generator, MGSum, and BART (Wang et al., 2022). Its human evaluation also reports significantly better ratings across informativeness, fluency, and succinctness, with overall rating $y^* = \arg\max_y P(y \mid D, G),$0 versus $y^* = \arg\max_y P(y \mid D, G),$1 for Pointer-Generator, $y^* = \arg\max_y P(y \mid D, G),$2 for MGSum, and $y^* = \arg\max_y P(y \mid D, G),$3 for GraphSum, while ablations show monotonic degradation as KG-centric components are removed (Wang et al., 2022). These results support the claim that explicit entity and relation modeling benefits cross-paper summarization.

Medical evidence synthesis requires metrics that preserve directionality. MS2 supplements ROUGE with EI macro-F1 and $y^* = \arg\max_y P(y \mid D, G),$4, a Jensen–Shannon-distance-based consistency metric over intervention/outcome tuples (DeYoung et al., 2021). On the test set, BART achieves ROUGE-1 $y^* = \arg\max_y P(y \mid D, G),$5, ROUGE-2 $y^* = \arg\max_y P(y \mid D, G),$6, ROUGE-L $y^* = \arg\max_y P(y \mid D, G),$7, $y^* = \arg\max_y P(y \mid D, G),$8, and EI F1 $y^* = \arg\max_y P(y \mid D, G),$9, while human evaluation finds that 67/133 generated summaries match the gold primary direction after excluding summaries lacking an efficacy statement (DeYoung et al., 2021). The paper further reports that only 53.8% of study-level EI directions match the review-level direction, showing that lexical fluency and clinical-faithful aggregation are not equivalent (DeYoung et al., 2021).

Multimodal retrieval and cross-document QA remain substantially below expert performance. In M3SciQA, human expert MRR in the visual retrieval stage is $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$0, whereas GPT-4o reaches approximately $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$1 on the full test table; in the second stage, human expert accuracy is $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$2 and the best LLM, Command R+, reaches $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$3 (Li et al., 2024). GPT-4o also shows a split failure profile in which correct visual reasoning does not guarantee correct ranking of the cited paper, and expanding the reference set beyond top-3 papers degrades QA accuracy due to noise (Li et al., 2024). DocHop-QA reports similarly uneven performance across subtasks: the best XML Entity Index Extraction result is F1 $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$4 with BigBird (per-entity) + CLIP, while Qwen2.5-VL, zero-shot, image-only achieves BLEU $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$5, ROUGE-1 $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$6, ROUGE-2 $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$7, ROUGE-L $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$8, and ROUGE-LSum $$P(S \mid D) = P_{\theta_{D\rightarrow T}}(T \mid D)\cdot P_{\theta_{(D,T)\rightarrow S}}(S \mid D, T).$$9 in Generative Text Extraction (Park et al., 20 Aug 2025).

Agentic and structured systems reveal a mixed picture. ResearchPulse reports that Mmap-Agent achieves METEOR $D(m)$0 and BERTScore F1 $D(m)$1, while Lchart-Agent reaches Pass@1 $D(m)$2, FID $D(m)$3, and CLIP-FID $D(m)$4 in experimental-analysis chart generation (Chen et al., 3 Sep 2025). PaperScope is more pessimistic: across 16 agents, Grok 4 leads with overall score $D(m)$5, OpenAI o3 Deep Research scores $D(m)$6, and text-only ablation causes an average $D(m)$7 drop (Xiong et al., 13 Apr 2026). Error analysis in PaperScope attributes 37.5% of failures to extraction errors, 22.5% to reasoning errors, 20% to retrieval failures, 15% to hallucinations, and 5% to context limits (Xiong et al., 13 Apr 2026).

Benchmarks with formal ground truth make the extraction–inference split measurable. SciR shows that both principled inference complexity and premise obfuscation hurt every model, and that their effects compound: for GPT-4o CoT, average chance-normalized accuracy falls from $D(m)$8 to $D(m)$9 under harder inference alone, to $R(m_i,m_j)$0 under obfuscation alone, and to $R(m_i,m_j)$1 when both are combined (Beckmann et al., 11 Jun 2026). Even neuro-symbolic pipelines degrade sharply under obfuscation, indicating that scientific rendering stresses premise extraction rather than only end-to-end language modeling (Beckmann et al., 11 Jun 2026). SciCo-Radar provides a comparable message for relation induction: on the full SciCo test set, coreference CoNLL F1 rises from $R(m_i,m_j)$2 with the Longformer baseline to $R(m_i,m_j)$3 with relational definitions, while hierarchy F1 rises from $R(m_i,m_j)$4 to $R(m_i,m_j)$5 (Forer et al., 2024).

6. Limitations, misconceptions, and research directions

A central misconception rejected across the literature is that multi-document scientific inference is solved by stronger summarization alone. MS2 explicitly centers contradictions; M3SciQA shows that correct visual interpretation does not ensure correct paper retrieval; PaperScope and SciR show that retrieval, extraction, and formal reasoning remain separable bottlenecks (DeYoung et al., 2021, Li et al., 2024, Xiong et al., 13 Apr 2026, Beckmann et al., 11 Jun 2026). A second misconception is that text-only modeling is sufficient. PaperScope, DocHop-QA, and M3SciQA all design tasks that require figures, tables, or layout-aware cues, and report substantial penalties when multimodal grounding is weakened (Xiong et al., 13 Apr 2026, Park et al., 20 Aug 2025, Li et al., 2024).

Current limitations are also recurrent. KGSum notes reliance on information extraction quality, degradation from greedy KGtext decoding, and the absence of explicit contradiction or uncertainty modeling (Wang et al., 2022). MS2 warns that without strong factuality guarantees, deployment in high-stakes clinical settings risks plausible but incorrect summaries and that quality assessment, risk of bias, and study-level metadata should be integrated (DeYoung et al., 2021). ResearchPulse identifies PDF parsing artifacts, method naming ambiguity, and limited domain coverage outside AI-related fields (Chen et al., 3 Sep 2025). SciCo-Radar highlights the cost of relational-definition generation and the absence of global DAG constraints during hierarchy inference (Forer et al., 2024). AIM-SciQA currently excludes multimodal content such as figures and tables, while DocHop-QA reports very low spatial containment accuracy despite higher overlap scores, indicating weak precise grounding (Lee et al., 15 Mar 2026, Park et al., 20 Aug 2025).

The proposed directions are correspondingly varied. KGSum suggests contradiction-aware relations, claim verification modules, uncertainty annotations, external domain ontologies, constrained decoding, and reinforcement learning with factual consistency rewards (Wang et al., 2022). MS2 proposes better factuality metrics, contradiction detection and modeling, citation- and evidence-grounded generation, stronger PICO extraction, and risk-of-bias features (DeYoung et al., 2021). PaperScope emphasizes graph-aware retrieval, stronger models for chart/table reading, formula parsing, algorithm understanding, robust tool invocation, and memory and planning for long contexts (Xiong et al., 13 Apr 2026). ResearchPulse proposes formal alignment models, schema enrichment beyond AI, robust multi-modal PDF parsers, and human-in-the-loop validation (Chen et al., 3 Sep 2025). SciR recommends multi-pass extraction, retrieval-augmented parsing, and self-consistency over multiple formalizations to improve the formalization step under obfuscation (Beckmann et al., 11 Jun 2026).

Taken together, these directions indicate that the field is converging on a richer notion of scientific inference: not only producing fluent outputs, but grounding claims in aligned evidence across documents, modalities, and structured scientific objects; not only ranking relevant papers, but preserving provenance, polarity, hierarchy, and uncertainty; and not only answering benchmark questions, but supporting the broader research workflows of evidence synthesis, literature review, claim verification, and research development analysis (Xiong et al., 13 Apr 2026, Chen et al., 3 Sep 2025, Forer et al., 2024).