Asta Interaction Dataset Overview

• Asta Interaction Dataset is a comprehensive, publicly available benchmark that captures anonymized logs of user queries and interactions with AI research tools.
• It includes structured data from PaperFinder and ScholarQA, detailing query intents, complexity metrics, and behavioral navigation patterns.
• The dataset supports empirical analysis of research workflows, guiding improved design and evaluation for AI-powered research assistants.

The Asta Interaction Dataset (AID) is a large-scale, publicly available resource characterizing real-world user interactions with AI-powered scientific research tools. It consists of anonymized logs of queries and user interface actions collected across two deployed modules within a retrieval-augmented, LLM–based research assistant. The dataset enables detailed empirical analysis of how researchers engage with AI-driven literature discovery and scientific question answering, and provides the first taxonomized, data-rich benchmark for understanding and evaluating such workflows (Haddad et al., 26 Feb 2026).

1. Dataset Composition and Schema

AID comprises 258,935 user queries and 432,059 clickstream interactions captured through two core interfaces:

• PaperFinder (PF): Literature discovery, supporting exploratory and targeted paper retrieval.
• ScholarQA (SQA): Scientific question answering, supporting natural language research queries and multi-section summarization.

Data are distributed as six anonymized Parquet tables:

File	Scope	Key Fields
optin_queries_anonymized.parquet	All queries	thread_id, query_ts, tool, query
section_expansions_anonymized.parquet	SQA only	thread_id, section_expand_ts, section_id
s2_link_clicks_anonymized.parquet	All tools (evidence)	thread_id, s2_link_click_ts, corpus_id, tool
report_section_titles_anonymized.parquet	SQA only	thread_id, section_idx, section_title
report_corpus_ids_anonymized.parquet	SQA only	thread_id, corpus_id
pf_shown_results_anonymized.parquet	PF only	thread_id, query_ts, result_position, corpus_id

No user identifiers are released; queries containing personally identifiable information were filtered out by LLM-based content screening (<1% of data). Each session is identified by a salted SHA-256 thread hash and demarcated using a 45-minute inactivity threshold (Haddad et al., 26 Feb 2026).

2. Query Intent Taxonomy and Classification

AID introduces a 16-category query intent taxonomy specific to research assistants. Categories, which are not mutually exclusive, include:

• Broad Topic Exploration
• Specific Factual Retrieval
• Concept Definition and Explanation
• Comparative Analysis
• Causal/Relational Inquiry
• Methodological/Procedural Guidelines
• Tool/Resource Discussion
• Research Gap Analysis
• Citation/Evidence Finding
• Specific Paper Retrieval
• Ideation
• Application Inquiry
• Data Interpretation Support
• Content Generation Experiment
• Academic Document Drafting
• Focused Academic Synthesis

Intents are assigned through heuristic pseudocode using lexical cues, structural query features, and entity detection. For instance, queries like "GLP-1 and diabetes" are classified as Broad Topic Exploration, while requests such as "Write a full Materials and Methods section" are mapped to Academic Document Drafting. The taxonomy enables stratified sampling and intent-aware analysis (Haddad et al., 26 Feb 2026).

3. Query Structure, Complexity, and Metrics

AID systematically quantifies query length, constraint, entity, and relation densities. For each query $i$:

• Word count $L_i$
• Constraint count $K_i$
• Entity count $E_i$
• Relation count $R_i$
• Overall complexity score: $C_i = K_i + E_i + R_i$

Statistical summaries by interface:

Metric	PF	SQA	S2
Mean constraints/query	0.60	0.82	0.15
Mean entities/query	4.00	5.14	2.25
Mean relations/query	2.17	2.68	1.20
Mean length (words)	17.0	36.96	5.35

These distributions reflect that users engage in complex, multi-entity and multi-constraint interactions, particularly within SQA. Complexity metrics enable longitudinal studies on how researchers adapt their formulation strategies (Haddad et al., 26 Feb 2026).

4. Engagement Patterns and Behavioral Analysis

AID reveals several unique engagement behaviors:

• Artifact Persistence and Revisit Patterns: 50.5% of SQA and 42.1% of PF users revisit prior outputs, often hours later (median 4–6 h). Only 14–19% of sessions involved near-duplicate queries, indicating that previously generated content is treated as a persistent research artifact.
• Nonlinear Navigation: SQA section navigation is non-sequential—43% of users skip introductions, 52.4% expand sections out-of-order, and frequent backward jumps occur.
• Citation Engagement: Deep evidence engagement is quantified by per-thread citation click counts and revisit counts; aggregate engagement depth score is computed as a weighted sum of these behaviors.
• Evolving Query Complexity: With experience (defined by query index per user/thread), researchers shift from Broad Topic Exploration (dropping from 61.2% to 53.5%) toward queries that include more entities ($+15\%$) and more constraints (mean rises from 0.55 to 0.66), though keyword-style queries persist ($\sim$45–55%) even at advanced stages (Haddad et al., 26 Feb 2026).

These artifacts enable the design and testing of assistant features structured around real engagement flows.

5. Data Availability, Anonymization, and Licensing

The dataset is available under the CC BY-4.0 license and can be accessed at https://github.com/allenai/asta-interaction-dataset. All PII is removed via LLM-based filtering, all user and session identifiers are anonymized or omitted, and Parquet serialization is used for efficient analysis. Licensing facilitates open, reproducible research for both behavioral studies and algorithmic evaluation (Haddad et al., 26 Feb 2026).

6. Implications for Research Assistant Design and Evaluation

The structural and behavioral statistics in AID inform several assistant design recommendations:

• Query-Formulation Support: Prompts for intent clarification and query builder interfaces are suggested to reduce user errors and guide filter application.
• Content Navigation and Consumption: Nonlinear navigation patterns imply the necessity of persistent artifact support, section-level TOC, and “TL;DR” toggles in interface design.
• Reliability and Latency: Graceful fallback mechanisms and error recovery guidance can mitigate latency and failure, particularly for inexperienced users.

Evaluation protocols proposed include intent-stratified benchmarks (sampling by query category to test retrieval/citation accuracy and behavior reproduction), longitudinal user simulation (measuring progression in query complexity and engagement depth), and artifact persistence tests (tracking report revisits and content refreshes over time). These methods leverage the taxonomy and behavioral diversity evident in AID (Haddad et al., 26 Feb 2026).

7. Research Significance and Future Directions

AID provides an empirical, taxonomized foundation for the study and benchmarking of AI-powered research assistants, supporting both behavioral science and technical system assessment. Its scale, session structure, and granular intent annotation distinguish it from generic search logs and facilitate new evaluations—longitudinal adaptation, intent stratification, artifact persistence, and multi-faceted engagement metrics (Haddad et al., 26 Feb 2026).

A plausible implication is that future research assistants for scientific workflows should prioritize artifact persistence, enhance clarity in query formulation support, and tailor interface affordances for non-linear, high-complexity exploration, as reflected in actual researcher usage patterns.