DataFlow: An LLM-Driven Framework for Unified Data Preparation and Workflow Automation in the Era of Data-Centric AI

Abstract: The rapidly growing demand for high-quality data in LLMs has intensified the need for scalable, reliable, and semantically rich data preparation pipelines. However, current practices remain dominated by ad-hoc scripts and loosely specified workflows, which lack principled abstractions, hinder reproducibility, and offer limited support for model-in-the-loop data generation. To address these challenges, we present DataFlow, a unified and extensible LLM-driven data preparation framework. DataFlow is designed with system-level abstractions that enable modular, reusable, and composable data transformations, and provides a PyTorch-style pipeline construction API for building debuggable and optimizable dataflows. The framework consists of nearly 200 reusable operators and six domain-general pipelines spanning text, mathematical reasoning, code, Text-to-SQL, agentic RAG, and large-scale knowledge extraction. To further improve usability, we introduce DataFlow-Agent, which automatically translates natural-language specifications into executable pipelines via operator synthesis, pipeline planning, and iterative verification. Across six representative use cases, DataFlow consistently improves downstream LLM performance. Our math, code, and text pipelines outperform curated human datasets and specialized synthetic baselines, achieving up to +3\% execution accuracy in Text-to-SQL over SynSQL, +7\% average improvements on code benchmarks, and 1--3 point gains on MATH, GSM8K, and AIME. Moreover, a unified 10K-sample dataset produced by DataFlow enables base models to surpass counterparts trained on 1M Infinity-Instruct data. These results demonstrate that DataFlow provides a practical and high-performance substrate for reliable, reproducible, and scalable LLM data preparation, and establishes a system-level foundation for future data-centric AI development.

• The paper introduces a scalable, programmable framework that leverages LLMs to transform data preparation into a unified, reproducible workflow.
• It employs a modular operator library and agentic automation, significantly improving semantic quality and efficiency in tasks like text, code, and Text-to-SQL.
• Empirical results show measurable gains (e.g., 1–3% in math benchmarks, 7% in code generation) and set new standards for data-centric AI research.

DataFlow: Systematizing LLM-Centric Data Preparation and Workflow Automation

Motivation and Context

The proliferation of LLMs has transformed data preparation into a semantically rich, model-in-the-loop process, requiring scalable and reproducible workflows across domains such as text, code, mathematical reasoning, and structured knowledge. Prevailing practices—ad-hoc scripts, configuration-centric frameworks, and limited support for generative transformations—exhibit major deficiencies: fragmented protocols, poor extensibility, minimal agentic automation, and limited semantic control. Existing systems (e.g., NeMo Curator, Data-Juicer) focus on extraction and filtering, lacking unified abstractions and principled synthesis-orchestration capabilities. The DataFlow framework directly addresses these limitations by formalizing data preparation as a programmable, modular substrate, prioritizing LLM-driven synthesis, semantic refinement, and agentic workflow construction (2512.16676).

System Architecture and Abstractions

DataFlow is architected as a layered system with explicit separation of concerns and extensibility (Figure 1). At its core, a unified execution engine integrates global storage, operator libraries, prompt template abstractions, and backend-agnostic LLM serving. Above this engine, Control Layer APIs (CLI and DataFlow-Agent) expose programmable, scriptable, and agent-driven pipeline orchestration, while a modular extension ecosystem supports domain-specific workflows. The collective design enables reproducible, task-aligned dataset production for consumption by downstream LLM applications.

Figure 1: DataFlow's high-level architecture, integrating backend abstractions, reusable pipelines, control layers, and an extensible operator ecosystem.

Operators interact with the canonical tabular dataset maintained in global storage using a read-transform-write paradigm, enabling deterministic, composable data transformations (Figure 2).

Figure 2: Operator execution via run(), encapsulating read-transform-write interaction with Global Storage.

Operators are flexibly bound to dataset columns via key-based input-output mapping, supporting arbitrary compositionality without preprocessing (Figure 3). Workflows are assembled as pipelines using a PyTorch-style API, supporting stepwise execution, compilation, and resumption for efficient development and debugging (Figure 4).

Figure 3: Flexible operator binding via key mapping, enabling agnostic composition across heterogeneous datasets.

Figure 4: Pipeline API illustration, featuring modular backend instantiation, operator configuration, and composite execution.

Operator and Pipeline Ecosystem

The framework incorporates nearly 200 modular operators spanning four functional categories: generation, evaluation, filtering, and refinement. Core and domain-specialized operators are organized across modality and functional dimensions, supporting text, code, math, SQL, agentic multi-hop reasoning, and large-scale knowledge extraction. The operator library embodies reusable primitives for generate–evaluate–filter–refine workflow design, balancing extensibility with conceptual compactness. Sample evolution trends demonstrate how pipeline stages modulate dataset cardinality and semantic quality (Figure 5).

Figure 5: Sample count evolution across operator stages, contrasting filtering/preparation dynamics across pipelines.

Agentic Automation and Natural-Language Programming

A distinguishing component of DataFlow is the DataFlow-Agent, built atop LangGraph for multi-agent graph orchestration (Figure 6). The agent ensemble translates natural language task specifications into executable DAG pipelines via intent decomposition, data routing, operator retrieval/synthesis, pipeline assembly, and verification. Notably, the agentic subsystem autonomously synthesizes and debugs new operator code in response to under-specified or novel user intent, surpassing the capabilities of prior configuration-driven agentic approaches.

Figure 6: Multi-agent orchestration architecture for natural-language-driven pipeline construction and verification.

Domain-Specific Pipeline Example: Text-to-SQL

Text-to-SQL pipelines exemplify the unified operator composition and prompt template abstraction enabled by DataFlow (Figure 7). The SQL Generation and SQL Refinement pipelines integrate bespoke operators for SQL synthesis, augmentation, question generation, CoT trace construction, comprehensive filtering, and detailed schema classification. Backend database management and prompt template specialization ensure portability across relational systems.

Figure 7: End-to-end Text-to-SQL pipeline integrating modular operators for query synthesis, augmentation, reasoning, and validation.

Empirical Results and Contradictory Claims

Extensive experiments across six representative domains validate DataFlow's efficacy. On mathematical benchmarks, DataFlow-synthesized training data yields 1–3 point gains over high-quality synthetic baselines (e.g., MATH, GSM8K, AIME). In code generation, DataFlow pipelines deliver up to 7% average improvements over leading public benchmarks. For Text-to-SQL, models trained on DataFlow corpora surpass the SynSQL baseline by over +3% in execution accuracy using an order-of-magnitude fewer samples. AgenticRAG experiments show DataFlow-constructed datasets matching or exceeding human-supervised multi-hop benchmarks in cross-dataset generalization. Notably, unified multi-domain datasets produced by DataFlow (10K samples) enable base models to outperform configurations trained with 1M Infinity-Instruct instances, challenging assumptions on scale versus semantic fidelity.

Practical and Theoretical Implications

Practically, DataFlow establishes a standardized protocol for LLM data preparation, reconciling the trade-off between modular reuse and domain customization. The PyTorch-like API and plugin ecosystem enable scalable, community-driven operator and pipeline development. Automated agentic orchestration materially reduces engineering overhead and accelerates iterative development. The system facilitates reproducibility, controlled ablation, and systematic comparison across workflows—capabilities critical for robust LLM benchmarking and transfer learning. Theoretically, DataFlow introduces a compositional substrate for model-in-the-loop synthesis, formalizing operator and prompt template abstractions for rich semantic control. The agentic layer anticipates future directions in language-driven program synthesis, dynamic workflow adaptation, and autonomous data engineering.

Speculation on Future AI Evolution

DataFlow's unified abstraction is extensible to multimodal regimes (vision, table, graph) and industrial verticals (AI4Science, enterprise data engineering). Expansion toward domain-specialized variants (e.g., DataFlow-Table, DataFlow-Graph, DataFlow-Multimodal) will facilitate richer cross-modal data synthesis and curation. Further integration of learning-based operator selection, metric-driven pipeline optimization, and reinforcement learning-guided workflow design is plausible. As data-centric paradigm priorities intensify, systematized, LLM-orchestrated data engineering substrates such as DataFlow will be instrumental for compositional generalization, continual adaptation, and self-evolving model/data co-design.

Conclusion

DataFlow (2512.16676) presents a system-level framework for LLM-centric data preparation, combining modular operator composition, agentic pipeline orchestration, and backend-agnostic deployment. Empirical analyses demonstrate contradictory evidence against scale-based data efficiency assumptions, highlighting the primacy of semantic quality, verification, and targeted synthesis. The methodology formalizes a reproducible, extensible, and programmable foundation for data-centric AI research and engineering, supporting scalable workflow automation beyond current extraction- and filter-centric conventions. DataFlow stands as a robust substrate for community protocol standardization, multi-domain benchmarking, and future advances in agentic and language-driven data engineering.