---
title: Asta Interaction Dataset Overview
url: https://www.emergentmind.com/topics/asta-interaction-dataset
type: topic
---

# Asta Interaction Dataset Overview

The Asta Interaction Dataset (AID) is a large-scale, publicly available resource characterizing real-world user interactions with AI-powered scientific research tools. It consists of anonymized logs of queries and user interface actions collected across two deployed modules within a retrieval-augmented, large language model–based research assistant. The dataset enables detailed empirical analysis of how researchers engage with AI-driven literature discovery and scientific question answering, and provides the first taxonomized, data-rich benchmark for understanding and evaluating such workflows [2602.23335].

## 1. Dataset Composition and Schema

AID comprises 258,935 user queries and 432,059 clickstream interactions captured through two core interfaces:

- **PaperFinder (PF):** Literature discovery, supporting exploratory and targeted paper retrieval.
- **ScholarQA (SQA):** Scientific question answering, supporting natural language research queries and multi-section summarization.

Data are distributed as six anonymized Parquet tables:

| File                              | Scope                  | Key Fields                                            |
|------------------------------------|------------------------|-------------------------------------------------------|
| optin_queries_anonymized.parquet   | All queries            | `thread_id`, `query_ts`, `tool`, `query`              |
| section_expansions_anonymized.parquet | SQA only               | `thread_id`, `section_expand_ts`, `section_id`        |
| s2_link_clicks_anonymized.parquet  | All tools (evidence)   | `thread_id`, `s2_link_click_ts`, `corpus_id`, `tool`  |
| report_section_titles_anonymized.parquet | SQA only               | `thread_id`, `section_idx`, `section_title`           |
| report_corpus_ids_anonymized.parquet | SQA only               | `thread_id`, `corpus_id`                              |
| pf_shown_results_anonymized.parquet | PF only                | `thread_id`, `query_ts`, `result_position`, `corpus_id`|

No user identifiers are released; queries containing personally identifiable information were filtered out by LLM-based content screening (<1% of data). Each session is identified by a salted SHA-256 thread hash and demarcated using a 45-minute inactivity threshold [2602.23335].

## 2. Query Intent Taxonomy and Classification

AID introduces a 16-category query intent taxonomy specific to research assistants. Categories, which are not mutually exclusive, include:

- Broad Topic Exploration
- Specific Factual Retrieval
- Concept Definition and Explanation
- Comparative Analysis
- Causal/Relational Inquiry
- Methodological/Procedural Guidelines
- Tool/Resource Discussion
- Research Gap Analysis
- Citation/Evidence Finding
- Specific Paper Retrieval
- Ideation
- Application Inquiry
- Data Interpretation Support
- Content Generation Experiment
- Academic Document Drafting
- Focused Academic Synthesis

Intents are assigned through heuristic pseudocode using lexical cues, structural query features, and entity detection. For instance, queries like "GLP-1 and diabetes" are classified as Broad Topic Exploration, while requests such as "Write a full Materials and Methods section" are mapped to Academic Document Drafting. The taxonomy enables stratified sampling and intent-aware analysis [2602.23335].

## 3. Query Structure, Complexity, and Metrics

AID systematically quantifies query length, constraint, entity, and relation densities. For each query $i$:

- Word count $L_i$
- Constraint count $K_i$
- Entity count $E_i$
- Relation count $R_i$
- Overall complexity score: $C_i = K_i + E_i + R_i$

Statistical summaries by interface:

| Metric                    | PF      | SQA      | S2      |
|---------------------------|---------|----------|---------|
| Mean constraints/query    | 0.60    | 0.82     | 0.15    |
| Mean entities/query       | 4.00    | 5.14     | 2.25    |
| Mean relations/query      | 2.17    | 2.68     | 1.20    |
| Mean length (words)       | 17.0    | 36.96    | 5.35    |

These distributions reflect that users engage in complex, multi-entity and multi-constraint interactions, particularly within SQA. Complexity metrics enable longitudinal studies on how researchers adapt their formulation strategies [2602.23335].

## 4. Engagement Patterns and Behavioral Analysis

AID reveals several unique engagement behaviors:

- **Artifact Persistence and Revisit Patterns:** 50.5% of SQA and 42.1% of PF users revisit prior outputs, often hours later (median 4–6 h). Only 14–19% of sessions involved near-duplicate queries, indicating that previously generated content is treated as a persistent research artifact.
- **Nonlinear Navigation:** SQA section navigation is non-sequential—43% of users skip introductions, 52.4% expand sections out-of-order, and frequent backward jumps occur.
- **Citation Engagement:** Deep evidence engagement is quantified by per-thread citation click counts and revisit counts; aggregate engagement depth score is computed as a weighted sum of these behaviors.
- **Evolving Query Complexity:** With experience (defined by query index per user/thread), researchers shift from Broad Topic Exploration (dropping from 61.2% to 53.5%) toward queries that include more entities ($+15\%$) and more constraints (mean rises from 0.55 to 0.66), though keyword-style queries persist ($\sim$45–55%) even at advanced stages [2602.23335].

These artifacts enable the design and testing of assistant features structured around real engagement flows.

## 5. Data Availability, Anonymization, and Licensing

The dataset is available under the CC BY-4.0 license and can be accessed at https://github.com/allenai/asta-interaction-dataset. All PII is removed via LLM-based filtering, all user and session identifiers are anonymized or omitted, and Parquet serialization is used for efficient analysis. Licensing facilitates open, reproducible research for both behavioral studies and algorithmic evaluation [2602.23335].

## 6. Implications for Research Assistant Design and Evaluation

The structural and behavioral statistics in AID inform several assistant design recommendations:

- **Query-Formulation Support:** Prompts for intent clarification and query builder interfaces are suggested to reduce user errors and guide filter application.
- **Content Navigation and Consumption:** Nonlinear navigation patterns imply the necessity of persistent artifact support, section-level TOC, and “TL;DR” toggles in interface design.
- **Reliability and Latency:** Graceful fallback mechanisms and error recovery guidance can mitigate latency and failure, particularly for inexperienced users.

Evaluation protocols proposed include intent-stratified benchmarks (sampling by query category to test retrieval/citation accuracy and behavior reproduction), longitudinal user simulation (measuring progression in query complexity and engagement depth), and artifact persistence tests (tracking report revisits and content refreshes over time). These methods leverage the taxonomy and behavioral diversity evident in AID [2602.23335].

## 7. Research Significance and Future Directions

AID provides an empirical, taxonomized foundation for the study and benchmarking of AI-powered research assistants, supporting both behavioral science and technical system assessment. Its scale, session structure, and granular intent annotation distinguish it from generic search logs and facilitate new evaluations—longitudinal adaptation, intent stratification, artifact persistence, and multi-faceted engagement metrics [2602.23335]. 

*A plausible implication is that future research assistants for scientific workflows should prioritize artifact persistence, enhance clarity in query formulation support, and tailor interface affordances for non-linear, high-complexity exploration, as reflected in actual researcher usage patterns.*

Source: https://www.emergentmind.com/topics/asta-interaction-dataset