Multi-Turn Agentic Scientific Literature Search via Workflow Induction

Abstract: Scientific literature search often requires more than retrieving papers from a single query: users' intents are underspecified, preference-dependent, and evolve through interaction. Existing search agents typically rely on fixed pipelines or implicit language-only reasoning, making their search strategies difficult to control, inspect, and refine. We introduce PaperPilot, a multi-turn literature search agent that frames scientific search as workflow induction. Given an anchor paper and a user query, PaperPilot constructs an executable DAG of paper-search operators, including keyword search, citation expansion, filtering, scoring, reranking, and evidence extraction. User feedback is then used to refine both the query and the workflow itself. We train PaperPilot with supervised workflow imitation and preference optimization over controlled workflow corruptions. Experiments show that PaperPilot-9B improves over the base Qwen3.5-9B toolset agent under multi-turn interaction, increasing Hit@5 from 58.0 to 77.0, MRR from 47.5 to 59.4, and nDCG@10 from 26.8 to 32.5, while reducing workflow execution errors from 9.5% to 0%. These results show that explicit, editable search workflows provide an effective and controllable interface for aligning literature search agents with complex scientific intent.

• The paper demonstrates that explicit, editable DAG workflows driven by multi-turn agent interactions significantly improve retrieval performance and cost efficiency.
• The paper outlines a two-stage training paradigm combining workflow imitation and preference optimization to enhance stability and accuracy of search results.
• The paper validates PaperPilot against baselines, reporting improved metrics such as Hit@5, Recall@50, and a complete elimination of workflow execution errors.

Multi-Turn Agentic Scientific Literature Search via Workflow Induction

Motivation and Problem Definition

Automating scientific literature search presents substantive challenges due to the implicit, evolving, and multi-faceted nature of user intent. Traditional systems typically execute fixed retrieval pipelines or rely exclusively on black-box LLM reasoning, which severely constrains controllability and user-driven refinement. "Multi-Turn Agentic Scientific Literature Search via Workflow Induction" (2607.00597) introduces PaperPilot, a framework casting literature search as the real-time induction and editing of explicit, executable DAG-structured search workflows. This approach moves away from opaque LLM-based query augmentation, allowing both the system and the end-user to interactively clarify, inspect, and optimize not just queries, but the entire search and post-processing pipeline.

Figure 1: PaperPilot leverages multi-turn feedback to clarify user intent and iteratively refine the search workflow, contrasting with single-pass retrieval approaches.

System Architecture and Workflow Induction

PaperPilot operates over a library of typed paper-search operators—encompassing primitives such as keyword search, citation graph expansion (including successor and predecessor traversals), deduplication, filtering, scoring, reranking, and evidence extraction. Each search episode is cast as constructing and executing a DAG over these operators, grounded on an anchor paper and an underspecified user query. The central innovation is the system’s iterative refinement over multi-turn human-agent interaction—explicitly revising both operation types (e.g., swapping citation expansion direction) and parameterization (e.g., filter predicates, rerank strategies) in response to structured or free-form feedback.

Figure 2: Given an anchor paper and a user-defined search objective, PaperPilot induces a DAG workflow from the operator toolset, executes it, and iteratively refines the workflow via multi-turn user interaction.

Critically, this approach yields high interpretability and controllability—key requisites for technical literature search where user preferences span nuanced axes such as baseline selection, recency, citation directionality, and application domain.

Training Paradigm

The system's 9B parameter model (PaperPilot-9B) is tuned in a two-stage regime:

1. Supervised Workflow Imitation: High-quality search workflow trajectories are distilled from a teacher model across five canonical search directions. Valid turns—where the gold standard paper is surfaced in top candidates—form the core supervised data.
2. Preference Optimization: Preference pairs are constructed by introducing realistic workflow corruptions (e.g., dropped nodes, wrong operator usage, misapplied filters) and optimizing the model to distinguish optimal workflows from flawed variants via DPO objectives.

This construction ensures that the model both faithfully induces executable, parameterized workflows and internalizes structural failure modes, directly addressing the challenges associated with editing-in-place rather than simply replanning from scratch.

Experimental Results and Findings

PaperPilot-9B is evaluated against both baseline tool-augmented LLMs and large-scale proprietary systems using a hold-out set with controlled simulation of multi-turn user feedback. End-to-end experiments employ anchor/intent pairs spanning diverse search requirements (predecessor, successor, sibling, benchmark, survey). Retrieval performance is assessed across standard information retrieval metrics (Hit@K, Recall@K, MRR, nDCG@K) and workflow-execution stability.

Retrieval and Cost Efficiency

• Under multi-turn interaction, PaperPilot-9B increases Hit@5 from 58.0 to 77.0, Recall@50 from 34.8 to 40.0, MRR from 47.5 to 59.4, and nDCG@10 from 26.8 to 32.5 over its base Qwen3.5-9B toolset-prompted agent, while reducing workflow execution errors from 9.5% to 0%.
• These gains are achieved with a cost two orders of magnitude lower than proprietary systems such as OpenAI DeepResearch (PaperPilot-9B: \$0.018/case vs DeepResearch: \$6.09/case).
• Compared to GPT-5.4 Web Search, PaperPilot-9B matches or outperforms on top-hit metrics, with superior cost-effectiveness.

  Figure 3: PaperPilot-9B significantly improves retrieval quality across Hit@5, Recall@50, MRR, and nDCG@10, while eliminating workflow execution errors relative to its base agent.

Workflow Induction and Refinement

Dedicated ablations on workflow construction and editing demonstrate that PaperPilot-9B achieves substantially higher workflow similarity (TF-IDF cosine) to ground truth compared to Qwen3.5-9B, with robust local-edit consistency, especially for add-node and remove-node refinement operations.

Figure 4: Workflow-generation similarity: PaperPilot-9B achieves higher TF-IDF similarity to reference workflows than base Qwen3.5-9B.

Figure 5: Workflow-editing stability: PaperPilot-9B exhibits superior local-edit consistency in add-node and remove-node scenarios, evidencing robustness in multi-turn refinement.

Multi-Turn Dynamics

Analysis of agent behaviors highlights that early dialog turns focus on clarification and constraint identification, while later turns are dominated by direct workflow refinement and output finalization. Multi-turn feedback is most beneficial when systematically mapped to explicit workflow edits as opposed to mere query text concatenation.

Figure 6: Distribution of agent actions reveals a staged interaction pattern—clarification in early turns, followed by structured workflow editing.

Figure 7: Retrieval quality improves with increasing multi-turn refinement for systems equipped to translate user feedback into workflow edits.

Sensitivity to Candidate Pool Scale

A scaling analysis reveals that retrieval effectiveness peaks for candidate pool sizes in the $K=8$–$10$ range. Larger pools introduce distractors and degrade precision, indicating diminishing returns from brute-force expansion without targeted workflow curation.

Figure 8: Sensitivity analysis: Retrieval performance is maximized at moderate candidate pool sizes; excessive expansion reduces effectiveness.

Human Study

In human-in-the-loop evaluations covering diverse research domains and user profiles, PaperPilot yields the highest success rate (74.7%), with favorable user satisfaction (QSS 4.2/5) and conviction that multi-turn clarification materially contributes to retrieval alignment.

Implications and Future Directions

Strong empirical evidence is provided that making literature search workflows explicit, editable, and operator-typed is highly beneficial for aligning retrieval outputs to complex scientific search intent. Unlike pipeline or implicit LLM approaches, PaperPilot’s agentic workflow induction closes the loop between user preference and symbolic control, yielding not only higher retrieval accuracy and stability but also practical cost advantages.

Practically, this design paradigm enables transparent debugging, reproducibility, and extension—opening latent customization and audit trails for research discovery tooling. Theoretically, framing agentic search as interactive workflow induction situates scientific retrieval in the space of interpretable, type-checked programming, creating links with the broader literature in program synthesis, automated planning, and operator-centric agent architectures.

Nonetheless, workflow coverage is bounded by the symbolic operator repertoire, and reliance on teacher-generated supervision imposes data bias risks. Extending PaperPilot to incorporate richer operator ontologies, learning new primitives on-demand, and open-domain cross-disciplinary evaluation remain essential future directions.

Conclusion

PaperPilot demonstrates that multi-turn workflow induction is a technically robust and cost-efficient paradigm for scientific literature search, outperforming static and black-box LLM-based agents on both retrieval metrics and workflow stability (2607.00597). The explicit, editable workflow abstraction is shown to be decisive in aligning to evolving, fine-grained user intent. This work points toward a new programmatic interface for LLM agents in research discovery—one grounded in the composition, execution, and continuous refinement of transparent, type-safe search workflows.