DataMind-12K: Large-Scale Data Analysis Corpus

• DataMind-12K is a curated, large-scale dataset of 11,707 multi-turn data-analytic trajectories that pair natural-language queries with executable, reasoning-rich code.
• It addresses key limitations of open-source analytic agents by enabling robust training through recursive composition and a blend of supervised fine-tuning and reinforcement learning.
• The dataset spans diverse domains and file formats, supporting the development of agents that rival proprietary systems on complex multi-hop analytic tasks.

DataMind-12K is a curated, large-scale corpus of 11,707 multi-turn, stepwise data-analysis trajectories, each pairing a natural-language data-analytic query with its corresponding executable, reasoning-rich code solution. Developed as part of the DataMind pipeline, DataMind-12K is designed to address the key limitations of open-source data-analytic agents—including insufficient training resources, challenges in stable multi-turn code generation, and the lack of standardized protocols for integrating supervised fine-tuning (SFT) and reinforcement learning (RL). The dataset spans diverse domains, file formats, and task categories, and supports robust training for generalist, code-producing, data-analytic agents that can match or exceed the performance of leading proprietary systems (Qiao et al., 29 Sep 2025).

1. Motivation and Problem Scope

DataMind-12K responds to the difficulty of training open-source agentic systems to perform complex, real-world data analysis—a field involving heterogeneous data files (CSV, Excel, SQL databases) and requiring workflows from exploratory data cleaning to inferential analytics and predictive modeling. Existing open-source models have demonstrated limited capabilities with multi-turn code tasks and often require labor-intensive prompt engineering. The construction and release of DataMind-12K aims to:

• Provide a high-quality, step-annotated data corpus (>11k trajectories) spanning a spectrum of domains and file formats.
• Enable the creation of agents capable of both simple and long-horizon (“multi-hop”) data-analytic tasks through recursive composition mechanisms.
• Serve as a foundation for supervised and RL-based training, as demonstrated by the DataMind-14B and DataMind-7B models, which surpass proprietary agents such as GPT-5 and DeepSeek-V3.1 on standard data analysis benchmarks (Qiao et al., 29 Sep 2025).

2. Composition, Coverage, and Statistics

Each trajectory in DataMind-12K is structured as $$y = (\tau_0, \alpha_0, o_0, \ldots, \tau_T, \alpha_T, o_T, <$$answer$>\ldots)$, where $\tau_t$ denotes agent thought, $\alpha_t$ the code action, and $o_t$ the resulting execution feedback.

2.1 File Format and Domain Distribution

The dataset is composed of the following source files:

Format	Trajectories	Percentage
CSV	6,672	57.0%
Excel	1,098	9.4%
SQLite	3,937	33.6%
Total	11,707	100%

Input files (3,400 CSV, 560 XLSX from Kaggle; 1,954 SQLite from BIRD and OmniSQL) cover at least ten broad application areas—including finance, healthcare, biology, social science, retail, e-commerce, manufacturing, marketing, sports analytics, and environmental science. While per-domain counts are not published, this diversity supports agents generalizing to heterogeneous data contexts.

2.2 Task Category Breakdown

Tasks are structured under an 18-way fine-grained taxonomy, grouped into four high-level classes with approximate proportions:

• Basic Descriptive & Aggregation: 32% (aggregation, counting, ranking, comparison, descriptive analysis)
• Statistical & Inferential: 28% (statistical analysis, correlation, distribution analysis, anomaly detection, causal analysis)
• Advanced Reasoning: 25% (multi-hop numerical reasoning, time-based calculation, fact checking, impact analysis, domain-specific inference)
• Preprocessing & Feature Engineering: 15% (comprehensive data preprocessing, feature engineering, arithmetic calculation)

Each fine-grained type is present in at least 500 trajectories, reducing overfitting and reinforcing agent generalization (Qiao et al., 29 Sep 2025).

3. Task Taxonomy and Recursive Composition Protocol

DataMind-12K’s fine-grained task taxonomy ensures broad coverage of analytic styles and complexity. To further increase difficulty and realism, the dataset leverages a recursive easy-to-hard composition mechanism:

Formally, for level $k$, a task template pool $T^{(k)}$ yields next-level tasks via

$$T^{(k+1)} = \bigl\{\text{Compose}(q_i, q_j)\mid q_i \in T^{(k)},\, q_j\in T^{(1)}\bigr\}$$

where Compose$(\cdot,\cdot)$ connects the output of one subtask to the input of the next (tabular or SQL). Generation proceeds as follows:

• Metadata (headers, types, samples) is extracted for each file.
• Base-level queries $Q^{(1)}$ are generated via few-shot prompting of DeepSeek-V3.
• For $>\ldots)$0, two tasks $>\ldots)$1 (from $>\ldots)$2, $>\ldots)$3) are combined—e.g., "Given the result of $>\ldots)$4, generate a type-$>\ldots)$5 follow-up that requires multi-step reasoning."
• Up to five levels are combined, producing chains of two to five steps, and requiring comprehension of intermediate results and multi-hop reasoning.

This approach yields composite queries requiring sequential aggregation, logical filtering, temporal reasoning, and visualization in a single trajectory.

4. Data Synthesis, Filtering, and Quality Control

The DataMind pipeline for corpus creation employs both expert LLM-based synthesis and rigorous filtering:

4.1 Knowledge-Augmented Trajectory Sampling

• Each query $>\ldots)$6 is coupled with a hand-crafted high-level workflow prompt (e.g., “Step 1: inspect columns; Step 2: filter by condition; Step 3: compute stats; Step 4: interpret results”).
• N=3 independent trajectories $>\ldots)$7 are sampled from an expert model (DeepSeek-V3.1).
• A self-consistency judge ($>\ldots)$8 GPT-4o-mini) validates that all three trajectories converge on the same final answer $>\ldots)$9: $\tau_t$0
• For $\tau_t$1, the judge’s chain-of-thought $\tau_t$2 is fed back for trajectory refinement, maximizing both answer correctness and diversity of reasoning (Qiao et al., 29 Sep 2025).

4.2 Rule-Based Filtering

Further filters remove any trajectory that:

• Violates the ReAct format (missing > /<code>/<answer> tags), > > - Produces a final answer exceeding 1,024 tokens, > > - Contains garbled or mixed-language outputs. > > After filtering, 11,707 trajectories remain in DataMind-12K. > > ## 5. Training Protocols and Rollout Framework > > DataMind-12K supports a dynamically blended SFT+RL training protocol, underpinned by robust, memory-efficient rollout practices: > > ### 5.1 Training Objectives > > - SFT Loss: > > > $\tau_t$3 > > excluding tokens that repeat execution feedback. > > - RL Loss (DAPO algorithm): > > > $\tau_t$4 > > with $\tau_t$5 the policy importance ratio, $\tau_t$6 the normalized judge reward. > > - Combined Objective: > > > $\tau_t$7 > > annealing $\tau_t$8 from 0.9 to 0.05 via cosine decay. Training includes a “cold-start” SFT phase for initial stability. > > ### 5.2 Rollout Techniques > > - Asynchronous Execution: Model token generation and code execution run in parallel threads to optimize resource use. > > - Chunked Code Maintenance: Only current code snippets are stored in memory, concatenated at runtime to emulate seamless notebook execution without incurring prohibitive memory costs. > > - Sandboxing and Security: Strict CPU/memory/time limits are enforced in code containers, and code invoking unsafe APIs is filtered out. > > - Reward Shaping: The judge model assigns binary format and answer rewards, length-normalized penalties, and a final multi-criteria reward: $\tau_t$9 > > This framework supports effective, scalable training for data-analytic agents, with dropout of degenerate trajectories. > > ## 6. Sample Trajectories and Annotation Schema > > DataMind-12K stores each trajectory as a JSON object with fields including file path, query (q), description (d), trajectory steps (ordered <think>/<code>/<interpreter>), and the final answer. > > Example (CSV): > > > - Query: “Which product category in sales.csv had the highest average revenue in 2021?” > > - Sequence: Load file → inspect columns → filter year → group and average by category → identify maximum → output answer (“Electronics”). > > Example (SQLite): > > > - Query: “In company.db, find the top‐3 departments by total headcount where average salary exceeds $80,000.” > > - Sequence: Inspect schema → join tables → filter on salary → group by department → sort and output top-3 as result. > > These examples illustrate the fine-grained, multi-turn, reasoning-plus-code format that characterizes DataMind-12K. > > ## 7. Access, Licensing, and Usage > > DataMind-12K will be released at https://github.com/zjunlp/DataMind under the Apache 2.0 license. Each trajectory is a line-delimited JSON file; code to reproduce the full synthesis and training pipeline will also be provided. > > - Citation requirement: “Scaling Generalist Data-Analytic Agents” (Qiao et al., 2025) (Qiao et al., 29 Sep 2025). > > - Prohibited uses: deployment in automated decision systems without human oversight; all users must attribute appropriately. > > DataMind-12K unifies a comprehensive taxonomy, recursive composition, knowledge-augmented expert sampling, stringent filtering, and a blended SFT+RL training framework. The resource enables open-source agents to match the analytic versatility and robustness previously associated only with proprietary, closed-source solutions.