Simula Framework: Reasoning-Driven Synthetic Data

• Simula Framework is a reasoning-driven, seedless synthetic data generation tool that creates audit-traceable datasets using modular, agentic pipelines.
• It integrates explicit conceptual taxonomies and iterative critique loops to ensure fine-grained control, transparency, and scalability in specialized AI applications.
• Simula has been validated in real-world deployments, notably in enterprise security, where it improves downstream model calibration and operational efficiency.

Simula is a reasoning-driven, seedless synthetic data generation and evaluation framework designed to address data scarcity and privacy challenges in specialized AI applications. By leveraging modular, agentic pipelines and explicit conceptual taxonomies, Simula enables fine-grained control, transparency, and scalability in constructing synthetic datasets without requiring pre-labeled seed data or intensive domain expertise. Its methodology has been rigorously validated in high-throughput, real-world settings such as enterprise security, demonstrating strong performance in both intrinsic data properties and downstream model calibration (Momeni et al., 9 Dec 2025, Davidson et al., 31 Mar 2026).

1. Motivation and Objectives

Simula was developed in response to the limitations of traditional data collection and prior synthetic data methods, which frequently depend on seed examples, manual prompts, or evolutionary algorithms. Such methods typically suffer from poor scalability, low transparency, and limited coverage control, especially in domains with privacy-sensitive or rare data distributions. Simula’s core objectives are:

• Seedless, reasoning-first generation: Constructing each synthetic data point “from first principles” through explicit agentic reasoning steps rather than by sampling from seed corpora or pre-existing pools.
• Fine-grained, explainable control: Allowing dynamic specification of diversity, complexity, and intended coverage by integrating model-built taxonomies and scenario planners.
• Transparency and auditability: Ensuring that every synthetic example is traceable to its generating plan and can be audited at each sampling and critique step.
• Scalability: Linear scaling with dataset size, with the added benefit that improved model reasoning directly lifts data quality and diversity.

These principles differentiate Simula from conventional synthetic generation systems and provide a foundation for robust, explainable, and scalable data curation (Davidson et al., 31 Mar 2026).

2. System Architecture and Agentic Workflow

Simula operates in a multi-phase agentic pipeline involving both human and automated agents, typically instantiated as LLMs or multi-modal models (M₃). The architecture consists of the following principal components:

Phase	Function	Key Agent Roles
Taxonomy Generation	Disentangles the conceptual space and builds hierarchical	ProposeFactors, Taxonomy Generator
	coverage via semantic taxonomies for each factor
Data Synthesis	Samples strategies and taxonomy nodes, generates scenario	Strategy Planner, Scenario Generator,
	prompts, injects complexity, synthesizes labeled examples	Sample Synthesizer, Quality Critic
Evaluation & Coverage	Computes coverage, diversity, and difficulty; enables audit	Embedded Critic, Coverage Analyzer

Detailed Flow:

1. Taxonomy Generation: The model, via the “ProposeFactors” agent, extracts key factors from high-level task instructions $y$ and uses these to generate semantic taxonomies $\{\mathcal{T}_i\}$ with breadth-first expansion and critic refinement.
2. Data Synthesis:
   • Global Sampling: Strategy mixes $M$ are sampled—combinations of taxonomy leaves that define the desired distributional coverage.
   • Local Scenario Generation: For each mix, multiple “meta-prompts” ($K$ candidates) are produced; one is randomly selected.
   • Complexification: With a user-set fraction $c$, prompts are made more challenging before synthesis.
   • Critic Refine Loop: Examples are iteratively proposed, critiqued, and refined up to $R$ times, enforcing label correctness and plausibility before acceptance into the dataset.
3. Evaluation: Coverage metrics (level-ratio), embedding-based diversity, and calibrated difficulty (via pairwise Elo) are computed. All samples retain an audit trail linking them to taxonomy and agentic decisions (Davidson et al., 31 Mar 2026, Momeni et al., 9 Dec 2025).

3. Formal Algorithms and Statistical Formulation

Taxonomy and Data Generation

Let $S$ denote strategy sets (e.g., “malicious,” “benign”), and for each $s \in S$ let $T_s$ be its taxonomy leaves. Sampling proceeds with uniform distributions: $p(s) = 1/|S|,\, p(\ell \mid s) = 1/|T_s|$.

Each labeled example $\{\mathcal{T}_i\}$0 is created by:

$\{\mathcal{T}_i\}$1

and is only accepted if

$\{\mathcal{T}_i\}$2

Pseudocode for the Seedless Generation Loop (Momeni et al., 9 Dec 2025, Davidson et al., 31 Mar 2026): $K$2

ML Integration and Loss

Training uses a combination of Simula-generated synthetic data $\{\mathcal{T}_i\}$3 and a time-weighted analyst feedback set $\{\mathcal{T}_i\}$4. Each sample $\{\mathcal{T}_i\}$5 receives weight $\{\mathcal{T}_i\}$6 based on origin and recency. For $\{\mathcal{T}_i\}$7 as the binary label and $\{\mathcal{T}_i\}$8 as the classifier:

$\{\mathcal{T}_i\}$9

The decision threshold $M$0 is calibrated to maximize F1-score while controlling the number of human-reviewed incidents.

4. Quality Control, Coverage, and Evaluation

Simula introduces systematic, multi-axis evaluation to ensure dataset quality and effectiveness:

• Embedding-based Diversity: Global and local diversity assessed via average cosine distances in embedding space (locality via $M$1-NN).
• Taxonomic (Level-Ratio) Coverage: Proportion of unique taxonomy nodes (at each level $M$2) present in the generated set, enabling actionable diagnostics and gap-filling.
• Complexity Scoring (CAST): Calibrated batch scoring and pairwise Elo ratings quantify sample difficulty; complexity curriculum can then be constructed.
• Critic Verification (“Double Critic”): Each proposed example passes through two critics (one for each possible label), empirically boosting acceptance precision (e.g., from $M$3 to $M$4 on hard MATH questions) (Davidson et al., 31 Mar 2026).

Downstream model calibration is evaluated by fine-tuning student models (e.g., Gemma 3 4B) on synthetic datasets and comparing accuracy on held-out, real test sets (e.g., CTI-MCQ, GSM8K, Global MMLU).

5. Enterprise Security Case Study and Empirical Results

Simula’s deployment in an enterprise security context demonstrates its capacity to generate diverse, high-quality training corpora and support self-sustaining active learning pipelines (Momeni et al., 9 Dec 2025):

• Operational Pipeline: Raw log streams ($M$5 events/day) are filtered by high-recall YARA rules, reducing candidate events by up to $M$6. Only $M$7 escalations per day reach analysts, with ticketing precision climbing from 0.7 to above 0.9 throughout the deployment.
• Synthetic Data Impact: Increasing $M$8 from 100 to 20,000 raises the maximum F1-score from 0.83 to 0.95, surpassing manually curated datasets.
• Active Learning: Outperforms fixed models—within one year, the active system discovers 10 unique true positives and eliminates 77 false positives not caught by static models.
• Minimal Analyst Overhead: Simula enables analysts without extensive ML expertise to guide data plan construction and model tuning.

6. Mechanism Design and Practical Guidelines

Key insights for applying Simula across domains (Davidson et al., 31 Mar 2026):

• Global and local diversity should be synthesized by combining broad taxonomic coverage with scenario multiplicity ($M$9 meta-prompts per mix).
• Complexity should be explicitly managed with a tunable complexity budget $K$0 and used to scaffold a difficulty curriculum for downstream models.
• Resource allocation can be balanced by tuning the critic budget $K$1 and scenario count against available compute.
• Continuous coverage diagnostics via taxonomy tracking guides where to adjust sampling or strategy.
• Auditability is maintained by logging taxonomy nodes and prompt trails for each sample.

7. Implications, Recommendations, and Broader Impact

Simula provides a blueprint for seedless, audit-traceable, and controllable synthetic data generation applicable to any domain with limited or sensitive ground truth. Practitioners are advised to:

• Start with 5–10 domain-specific factors, expanding semantic taxonomies to depth 2–3.
• Regularly monitor level-ratio coverage and update strategies to expand underrepresented areas.
• Leverage complexity and audit logs to calibrate student model curricula and ensure robust evaluation.

The system improves as foundation models advance and is well-suited for regulated or data-constrained environments. Live results indicate that Simula’s approach generalizes across structured security logs, math word problems, legal exams, and other specialized datasets, with diversity, complexity, and critic components all contributing additively to both data and model quality (Davidson et al., 31 Mar 2026, Momeni et al., 9 Dec 2025).