PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams

Abstract: Scientific paper recommendation is typically evaluated as static ranking over a fixed candidate set, yet real scientific reading unfolds as a daily, longitudinal process in which interests shift and feedback accumulates. We introduce PaperFlow, a framework that organizes it into three coupled stages: Profiling, which constructs and maintains a structured, inspectable scholarly profile from heterogeneous cold-start evidence; Recommending, which ranks each date-specific paper stream through multi-signal aggregation under a fixed display budget; and Adapting, which updates user state from semantically distinct feedback signals and models interest drift across days. We further define a longitudinal user-day benchmark that fixes users, dates, candidate pools, visible inputs, and hidden simulated relevance labels under a shared temporal information boundary. The benchmark contains 24 simulated research users, 50 daily paper streams, 1,200 user-day episodes, 20,727 unique papers, and 497,448 episode-paper records. We additionally specify a blind human-evaluation protocol to validate alignment between automatic metrics and expert judgments. Experiments against five scientific recommendation baselines show that PaperFlow achieves the strongest oracle-based ranking, the highest behavioral alignment with simulated reading selections, and the best blind human-evaluation score.

• The paper introduces a dynamic recommendation framework that profiles, recommends, and adapts to daily scientific paper streams using multi-signal ranking and drift modeling.
• It demonstrates superior performance over baselines with notable gains in gNDCG, behavioral alignment, and rapid adaptation to evolving research interests.
• Its integrated approach of editable profiles and continuous adaptation establishes a reproducible benchmark for longitudinal and feedback-driven paper recommendation.

PaperFlow: Profiling, Recommending, and Adapting Across Daily Paper Streams

Introduction and Motivation

Scientific paper recommendation is a canonical information-retrieval problem with significant emphasis on personal relevance in large and fast-evolving literatures. Traditional approaches predominantly operate in static settings, framing the problem as a single-shot ranking task given a fixed candidate pool and a relatively static user profile. This procedure is misaligned with real-world scientific reading, which is inherently longitudinal: researchers interact daily with dynamic paper streams, select a sparse subset for deep reading, and evolve their research interests over time due to feedback and drift.

PaperFlow addresses this fundamental gap, proposing an integrated framework that models the scientific reading process as a continuous, adaptive loop encompassing three tightly coupled stages: (1) Profiling, which builds and maintains an editable, structured scholarly profile from diverse cold-start evidence; (2) Recommending, which ranks each day's candidate pool through multi-signal aggregation; and (3) Adapting, which updates user models via feedback and manages interest drift. The system is benchmarked over a temporally frozen, reproducible dataset with detailed logging and both automatic and human-centric evaluation protocols.

Figure 1: Motivation and paradigm comparison between traditional scientific paper recommendation and PaperFlow.

Framework Overview and Methodological Details

PaperFlow operationalizes a closed-loop reading workflow. At the core, the system maintains a structured scholarly profile, decomposing user interest into long-term directions, topic weights, explicit author/institution preferences, must-read rules, and dynamically updated fields representing behavioral state and drift adaptation. Cold-start construction integrates profile text, URLs, representative papers or PDFs, and explicit preferences, leveraging LLM-assisted canonicalization to populate all structured fields.

Figure 2: Overview of the PaperFlow dynamic personalized scientific reading loop.

Each day, the system ranks a date-frozen candidate pool via a hierarchical scoring architecture weighted across semantic-content similarity, topic alignment, prior quality signals, explicit rules, recent behavioral feedback, and drift status. The display is constrained to Top-20 recommendations, modeled to reflect realistic screen real estate and selection behavior.

The adaptation mechanism decouples short-term behavioral signals from persistent interests, distinguishing transient exploration from substantive migration. A finely-grained drift model governs how sustained evidence for a new research direction impacts the profile, employing evidence-based windows and explicit bounded updates to prevent brittle oscillations or premature anchoring.

Benchmark Design and Evaluation Protocol

Crucially, PaperFlow introduces a benchmark that rigorously reproduces user-day interaction episodes rather than mere user-paper pairs. This longitudinal setup uses 24 simulated researcher personas spanning scientific domains, each interacting across 50 daily paper streams, yielding 1,200 user-day episodes over 20,727 unique papers and nearly half a million episode-paper records. The benchmark strictly enforces temporal isolation: candidate sets, user state, and only prior feedback are available during ranking, preventing future information leakage.

Figure 3: Construction pipeline of the PaperFlow benchmark. Daily paper streams and simulated researcher profiles are converted into date-frozen user-day episodes, providing clean method inputs alongside hidden pseudo-oracle labels, reading-report records, and drift diagnostics.

PaperFlow's evaluations are multi-faceted:

• Oracle-based ranking metrics (gNDCG@20, Useful@k, OracleRecall@20, etc.) measuring latent ranking quality under pseudo-oracle relevance labels.
• Behavioral consistency (SelectedNDCG@20), comparing ranked outputs to simulated user selections.
• Blind human evaluation, sampling Top-20 lists in a method-agnostic protocol with high annotator agreement.
• Specialized drift and adaptation metrics with diagnostic breakdowns for episodes featuring profile migration.
• Token-cost analysis for efficiency studies across LLM backbones.

  Figure 4: Overview of PaperFlow evaluation metrics. The figure groups metrics by oracle-based ranking quality, behavior alignment, interest-drift adaptation, report and model quality, human validation, and real-user pilot evaluation.

Empirical Results and Analysis

Main Results

PaperFlow achieves the highest aggregate quality, outpacing relevant baselines including Scholar Inbox, OMRC-MR, and various citation/entity-enhanced models. For instance, PaperFlow yields a gNDCG@20 of 50.65 vs. 39.00 for Scholar Inbox, and a HumanEval score of 65.56 vs. 55.56, indicating both robust pseudo-oracle and human preference alignment.

A salient distinction is observed in behavioral metrics: the SelectedNDCG@20 for PaperFlow is 70.88, over double the best-performing baseline. This substantial gain illustrates the advantage of unified state management and behavioral adaptation in anticipating what a simulated researcher will actually select for further reading.

Ablation and Drift Analysis

Ablation studies delineate the contributions of profile updating, drift modeling, explicit preferences, and reading signals. Notably, while simplified or fixed-profile variants can marginally improve static ranking (oracle-aligned gNDCG@20), they underperform on behavioral alignment. This reveals a static-dynamic trade-off intrinsic to temporally aware recommendation.

Figure 5: Interest-drift analysis. Cell color is normalized within each metric, with darker green indicating better performance. PostDrift metrics are computed on the post-drift window; NewTopicR, OldTopicR, AdaptDelay, DriftAuto, and AdaptHuman summarize adaptation-oriented behavior.

Interest-drift analysis on controlled migration episodes confirms that PaperFlow is superior in adaptation—exposing new-topic papers more quickly, aggressively downweighting obsolete directions, and recovering more rapidly after drift events compared to ablated or static baselines. AdaptationHumanScore and DriftAutoScore are both maximized in the full method, validated by blind annotation.

LLM Backbone Study

Modularizing the LLM component, PaperFlow evaluates closed and open-access backbones. Quality (ModelAutoScore, ModelHumanScore) and cost vary materially across models, though high correlation (Pearson r > 0.96) exists between automatic and human scores, validating the utility of automatic metrics.

Figure 6: Automatic--human metric alignment (ModelAutoScore vs. ModelHumanScore).

Token-cost analysis indicates efficiency is backbone-dependent and not necessarily aligned with quality, guiding model selection in production settings.

Case Studies

To complement aggregate metrics, detailed case analyses are conducted:

• Full alignment: Dense, high-quality recommendations for NLP users when profile and pool match tightly.
• Realized drift: Multistep profile migration adaptively exposing new-topic papers without premature anchoring.
• Behavior consistency: Strong agreement between behavioral signals and recommendation rank, surpassing static label congruence.

  Figure 7: Representative PaperFlow case study.

The visualization of simulated researcher profiles (Figure 8), and typical NLP and drift adaptation cases (Figures 10, 11) substantiate the robustness of the method under domain change, profile complexity, and evolving user interests.

Figure 8: The 24 simulated researcher profiles.

Figure 9: Successful recommendation case for an NLP user. The episode shows that the user's NLP/LLM/information-extraction profile aligns with the daily candidate pool, producing a dense Top-20 list with useful papers near the top.

Figure 10: Interest-drift case from GUI/Web agents to multimodal reasoning. Repeated evidence for a new direction leads PaperFlow to lock a multimodal-reasoning anchor and reweight later recommendations away from stale web-automation interests.

Figure 11: Behavior-consistency case with a high-SelectedNDCG list. Selected papers appear near the front of the Top-20, showing that behavior-based agreement can complement static oracle relevance in longitudinal recommendation evaluation.

Practical and Theoretical Implications

Practically, PaperFlow enables more transparent and adaptive reading pipelines for researchers navigating subfield evolution, literature deluge, and personalized knowledge acquisition. The deployment of structured, editable profiles, integrated reading reports, and temporally contextualized adaptation mechanisms sets a new bar for system support in scholarly ecosystems.

Theoretically, the benchmark formalizes paper recommendation as a temporal, input-frozen, feedback-coupled sequence—a setting that exposes limitations in one-shot or information-leaky approaches, and creates clear evaluation criteria for behavioral alignment and drift adaptation. The explicit demonstration of a static-dynamic trade-off, tunable via system configuration, catalyzes future research on alignment between automatic, behavioral, and human-centric relevance.

Future Directions

Potential research avenues include scaling human-in-the-loop evaluation beyond small pilot studies, extension to larger-scale and heterogeneous real-user logs, richer modeling of domain adaptation, and integration with agentic or conversational systems for higher-order feedback. Dynamically incorporating multiple collaborative profiles and integrating with the broader academic online ecosystem remains an open challenge.

Conclusion

PaperFlow establishes a systematic, temporally-sensitive framework for paper recommendation, coupling structured profiling, daily, multi-signal ranking, feedback-driven model adaptation, and precise modeling of interest drift. By publicizing protocols, benchmarks, and code, it drives both methodological innovation and replication in adaptive scientific information support. The empirical gains in behavioral, drift, and human-aligned metrics underscore the necessity of closed-loop, adaptive models for longitudinal scholarly recommendation (2606.07454).