Simula: Seedless Synthetic Data Framework

• Simula is a seedless, reasoning-driven framework for generating synthetic datasets with transparent auditability and fine-grained control over quality, diversity, and complexity.
• The framework employs a modular three-stage pipeline including taxonomy construction, controlled sampling, and critic refinement to ensure comprehensive data coverage and integrity.
• Empirical results show that Simula enhances taxonomic coverage, broadens complexity spectrums, and improves downstream task accuracies across various domains.

Simula is a reasoning-driven, seedless framework for synthetic data generation and evaluation designed to address the limitations of prompt-based, evolutionary, or seed-reliant methods. It provides a transparent, auditable, agentic pipeline for constructing datasets where users can specify desired characteristics and allocate resources with high granularity. Simula's architecture is notable for its modular, explainable control over coverage, diversity, quality, and complexity, enabling scalable synthetic data solutions especially in domains plagued by data scarcity, privacy, or quality concerns (Davidson et al., 31 Mar 2026).

1. Objectives and Defining Characteristics

Simula is engineered to generate specialized datasets at scale without requiring human-curated seed examples. Fundamental design objectives include full transparency and auditability of data provenance, fine-grained user control over quality, diversity, and complexity, and adaptability to improvements in underlying LLM reasoning. The framework positions mechanism design—focusing on how rather than what data is generated—as a fundamental research dimension independent of post-hoc filtering or data curation (Davidson et al., 31 Mar 2026).

2. Pipeline Architecture and Agentic Workflow

Simula operates via a three-stage agentic pipeline:

1. Factor Disentanglement and Taxonomy Construction: The user supplies high-level instructions $y$ (and optionally, a sample $S$). An LLM proposes high-level factors $f_0, ..., f_k$, which are expanded into hierarchical taxonomies $\mathbb{T} = \{ T_0, ..., T_k \}$. Expansion employs three agentic steps per factor:
   • Best-of-N proposals for generating diverse candidate child nodes,
   • Critic refinement cycles for completeness and specificity,
   • Level-wise planning for sibling granularity consistency.

Taxonomy expansion is implemented as shown in Algorithm 1 (LaTeX), supporting breadth-first generation and explicit critique-refinement loops.

1. Controlled Sampling and Meta-Prompt Generation: The Mixer Module defines sampling strategies to combine and weight taxonomy nodes, effecting trade-offs between coverage and diversity (e.g., children's vs. adult content). For each sampled mix $M$, the LLM generates $K$ candidate meta-prompts; one is randomly selected and, with probability $c$, passed to a Complexifier Module for edge-case enrichment.
2. Agentic Refinement and Critic Filtering: The LLM generates an initial data point $d = \mathrm{Generate}(p)$. The critic loop (single-critic for open-ended, dual-critic for verifiable tasks) verifies requirements per taxonomy. If rejected, an LLM-based Refiner edits $d$ according to critic feedback, iterating until acceptance or maximum retries. Accepted data points populate the synthetic dataset $\mathcal{D}$ (Davidson et al., 31 Mar 2026).

3. Reasoning-Based Mechanisms and Control Processes

Simula operationalizes LLM agency throughout the pipeline, facilitating:

• Generator-Critic Taxonomy Expansion: Enables explainable, breadth-first coverage of conceptual spaces with white-box provenance.
• Mixer-Guided Sampling: Provides explicit control of global coverage versus local diversity, supporting logical groupings and targeted weighting.
• Meta-Prompt Complexification: Allows stochastic control over complexity via the complexity ratio $S$0.
• Critic-Refinement Loop: Guarantees enforceability of taxonomic requirements via explicit, auditable LLM reasoning and editing.

Each process is designed for traceability and resource allocation, shaping the dataset along axes of interest independently and supporting future improvements as LLM reasoning advances (Davidson et al., 31 Mar 2026).

4. Customization and Enforced Data Properties

Users specify dataset characteristics via high-level natural language descriptions $S$1, with optional concrete samples $S$2 and factor depths $S$3. The modular pipeline enforces properties as follows:

• Taxonomic factors explicitly encode axes of variation.
• Mixers determine global coverage logic and weighting over taxonomy nodes.
• Meta-prompts guarantee semantic traceability of each example.
• The critic infrastructure ensures that outputs satisfy requirements, refining or rejecting as needed with explanations captured for auditability (Davidson et al., 31 Mar 2026).

5. Evaluation Metrics: Intrinsic and Downstream

Simula distinguishes several axes for evaluation:

• Intrinsic Diversity:
  • Global Diversity: $S$4.
  • Local Diversity: Mean pairwise embedding distance among k-nearest neighbors.
  • Taxonomic Coverage: For each level $S$5, $S$6.
• Intrinsic Complexity:
  • Calibrated attribute scoring via LLM reasoning.
  • Conversion to Elo scores through pairwise comparison, yielding a global complexity ranking.
• Downstream Task Metrics:
  • Train a student model via LoRA on synthetic data with varying $S$7.
  • Report accuracy $S$8 on held-out benchmarks (e.g., CTI-MCQ, CTI-RCM, LEXam, GSM8K, and Global MMLU) (Davidson et al., 31 Mar 2026).

6. Principles for Scalable and Transparent Synthetic Data Generation

Simula prescribes several core guidelines:

1. Treat mechanism design as a primary research axis.
2. Emphasize a seedless, reasoning-first methodology for robust improvement.
3. Decouple and control coverage, diversity, complexity, and quality at each pipeline phase for targeted allocation.
4. Maintain transparency and full auditability, ensuring provenance via taxonomy-driven traces and critic explanations.
5. Employ multi-faceted evaluation: embedding-based diversity, taxonomy-based coverage, complexity scores, and downstream benchmarks.
6. Recognize context-dependence: optimal configurations are domain-, model-, and use-case-specific (Davidson et al., 31 Mar 2026).

7. Empirical Performance Across Domains

Empirical results demonstrate Simula’s impact:

• Substantial gains in niche domains (CTI-MCQ, CTI-RCM, LEXam): up to 15–20 point accuracy improvements over baseline synthetic data, especially under data-scarce conditions ($S$9k–$f_0, ..., f_k$0k examples).
• Superior scaling in popular benchmarks (GSM8K, Global MMLU): joint global+local diversification with critic refinement shows steeper accuracy improvements as synthetic dataset size grows up to $f_0, ..., f_k$1k.
• Notable gains in multilingual regimes: critic-based rejection increases non-English MMLU accuracy (e.g., Korean, Nepali) by $f_0, ..., f_k$2–$f_0, ..., f_k$3 points per complexity tier.
• Intrinsic metric analysis: Simula variants approximately double taxonomic coverage compared to real datasets, with a broader complexity spectrum, enabling stratified analysis of model gaps by complexity level (Davidson et al., 31 Mar 2026).

Simula introduces a transparent, reasoning-driven, and agentic pipeline for synthetic data generation and evaluation. Its architecture and methodology enable rigorous, controllable, and explainable dataset construction, substantially outperforming legacy synthetic generation systems across domains and evaluation settings (Davidson et al., 31 Mar 2026).