YuLan-OneSim: LLM-based Social Simulation

• YuLan-OneSim is a language-model-driven social simulation system that automates computational social science experiments using natural language and a rich template library.
• It employs an auto-programming pipeline and a distributed agent platform to execute scalable, high-fidelity simulations with up to 100k concurrent agents and robust event-driven communication.
• The system integrates a VR²T fine-tuning loop and an AI social researcher module to enable end-to-end study design, execution, and reporting without the need for specialized coding skills.

YuLan-OneSim is a large-scale, language-model-driven social simulation system designed to automate and accelerate the entire cycle of computational social science research. It enables the code-free construction, execution, and analysis of high-fidelity social experiments over arbitrary scenarios as specified in natural language, powered by an extensible library of scenario templates, a scalable, distributed agent platform, and a closed-loop LLM fine-tuning framework. YuLan-OneSim underlies advanced platforms such as S-Researcher, supporting canonical research workflows in induction, deduction, and abduction, and further realizes an “AI social researcher” capable of end-to-end study design, execution, and reporting without specialized programming expertise (Wang et al., 12 May 2025, Wang et al., 2 Apr 2026).

1. System Foundations and Design Objectives

YuLan-OneSim is architected to provide:

• Generality: Supports arbitrary experimental designs specified in natural language via an auto-programming pipeline (Overview–Design Concepts–Details, ODD protocol), translating textual descriptions to executable simulations. This is underpinned by a scenario library of over 50 templates covering eight domains (economics, sociology, politics, psychology, organization, demographics, law, and communication), and a domain-agnostic behavior-graph formalism $\mathcal{G} = \{\mathcal{N}, \mathcal{E}\}$ representing agent actions and interactions.
• Scalability: Delivers population-scale simulations with up to 100,000 concurrent agents by leveraging an asynchronous, event-driven master–worker architecture, gRPC communication with message batching, and topology-aware co-location of interacting agents. Empirical results show linear-to-sublinear scaling, e.g., $\sim45$ s/round at 1k agents, $\sim450$ s/round at 10k agents, and $\sim3,538$–$6,026$ s/round at 100k agents, with throughput exceeding 49 events/s (Wang et al., 2 Apr 2026, Wang et al., 12 May 2025).
• Reliability: Maintains scientific trustworthiness of simulated behavior via a feedback-driven, multi-agent LLM fine-tuning cycle (Verifier–Reasoner–Refiner–Tuner, VR²T) and supports continuous improvement under both supervised (SFT) and reinforcement learning (e.g., DPO, PPO-style) objectives. Automated “LLM-as-Judge” quality assessments are conducted over simulation rounds.

2. Scenario Construction and Auto-Programming Pipeline

Scenarios are constructed in a multi-stage pipeline that translates natural-language descriptions to executable experiments:

1. ODD Formalization: LLM “detailer” agents interactively extract and structure scenario intent into an ODD document, querying the user as needed for clarifications.
2. Behavior Graph Construction: The system constructs $\mathcal{G} = \{\mathcal{N}, \mathcal{E}\}$, in which nodes $n_i$ denote distinct actions/events and edges $e_{ij}$ represent transitions or triggers, annotated with variable schemas for parameter passing.
3. Graph Validation: Both static analysis (checking for connectivity, variable constraints, dead-ends) and semantic validation (LLM “verifier” agents check action sequence plausibility) precede code generation.
4. Code Generation and Refinement: Breadth-first graph traversal applies code templates (Python/C++), while LLM-based "C-Refine" and "G-Refine" components iteratively repair logical errors. Compilation/unit testing automate detection of remaining faults.
5. Prompt Engineering Templates: Encapsulate structured prompts for ODD sections, graph-node type definitions, and code-repair operations.

The default scenario repository provides detailed, editable templates (including ODD specifications, agent graphs, and data schemas) spanning 50 topics within eight domains. Scenario generation achieves high efficiency—mean generation time is $\sim359$ s ($\sim301$ tokens/s), scenario behavior graphs rate $\sim45$0/5, and code quality $\sim45$1/5 as evaluated in the benchmark (Wang et al., 12 May 2025).

3. Distributed Simulation Architecture

YuLan-OneSim leverages a robust, distributed framework optimized for high agent volumes and rapid event throughput. Key architectural features include:

• Master–Worker Design: A master node manages the global state and distributes workload to workers. Each worker hosts a shard of agents (up to 1,000 per worker in 100k-agent runs).
• Event Bus and Messaging: An asynchronous, event-driven bus handles all agent communications (event objects contain sender, recipient, type, and payload metadata). Cross-node communication utilizes gRPC with batching and peer-to-peer routing.
• Agent Components: At runtime, agents are instantiated from LLMs (code generation by GPT-4o; inference by Qwen2.5-7B-Instruct via vLLM), and each agent comprises a profile (public/private/dynamic attributes), customizable memory module, planning (Chain-of-Thought, BDI, or ToM-based), and an action interface mapping state and plan to outgoing events.
• Concurrency and Fault Tolerance: Worker failures are automatically detected; the master node re-assigns agent partitions to available nodes, enabled by heartbeat monitoring and conceptual checkpoint/recovery support.

Scaling laws indicate $\sim45$2 for time per round with $\sim45$3 agents and $\sim45$4 workers (Wang et al., 12 May 2025). Empirical evaluation demonstrates stable throughput and the ability to match reference empirical datasets in real-world cases (e.g., Brazilian real estate).

4. Feedback-Driven Fine-Tuning and VR²T Cycle

YuLan-OneSim integrates a dynamic refinement mechanism for its backbone LLMs via the following closed loop:

• Verifier: Grades the quality of each LLM prompt–response pair ($\sim45$5).
• Reasoner: Generates diagnostic explanations ($\sim45$6).
• Refiner: If $\sim45$7, issues a minimal correction ($\sim45$8).
• Tuner: Collects corrected pairs for fine-tuning via SFT or RL objectives:
  • Supervised Fine-Tuning: $\sim45$9
  • RL (e.g., PPO): $\sim450$0

Iterating over simulation–fine-tune cycles, improvements of $\sim450$1–$\sim450$2\% per round (SFT) and $\sim450$3\% to $\sim450$4\% (DPO, depending on model size) are achieved before plateau (Wang et al., 2 Apr 2026). This feedback mechanism significantly raises the fidelity of agent behavior over time, while robust error breakdown shows that most issues are logical (∼70%) rather than syntactic.

5. AI Social Researcher: End-to-End Automation

YuLan-OneSim enables a fully autonomous “AI social researcher” module, orchestrating the complete research workflow:

• Experiment Design: Inspiration Agent generates candidate questions; Evaluation Agent scores them by feasibility, complexity, and insight; ODD-Generation Agent produces a structured protocol.
• Simulation and Analysis: The system runs YuLan-OneSim with the produced ODD, Data Analysis Agent processes outputs and produces visualizations and metrics.
• Reporting: Outline Writing Agent generates report structure; Report Writing Agent produces full LaTeX documents; Reviewer Agent critiques drafts and drives iterative revisions.

Empirical evaluation shows that this end-to-end loop achieves scenario design scores of relevance $\sim450$5, feasibility $\sim450$6, and overall report quality of $\sim450$7 (all on a five-point scale) (Wang et al., 12 May 2025). The entire process—from topic selection to final PDF—proceeds with minimal human intervention.

6. Canonical Social Science Reasoning Modes and Empirical Validation

YuLan-OneSim operationalizes three canonical research paradigms in computational social science:

• Induction: Scenario-generated, large-scale simulations to discover emergent regularities; outputs include macro-level patterns and candidate theories. Case: Axlerod’s cultural dissemination—local convergence increases by $\sim450$8, global diversity declines by $\sim450$9, reproducing the coexistence of local assimilation and global polarization.
• Deduction: Explicit hypothesis encoding for comparative model testing; system ranks competing hypotheses against empirical benchmarks (e.g., Spearman $\sim3,538$0, RMSE, regression coefficients). Case: Teacher attention allocation, finding communicative ability to be the dominant factor as in corresponding survey data.
• Abduction: Mechanism-guessing via counterfactuals and effect-size regressions. Case: Public goods games—simulation captures leader anchoring ($\sim3,538$1), with close alignment to parallel human experiments (Pearson $\sim3,538$2).

Twelve-dimension expert evaluation yields overall averages: Inductive=$\sim3,538$3, Abductive=$\sim3,538$4, Deductive=$\sim3,538$5 (Wang et al., 2 Apr 2026).

7. Limitations, Challenges, and Prospective Developments

Persistent challenges include LLM hallucination and logical errors in auto-generated code (addressable via enhanced static analysis and automated code-repair loops), reduced behavioral heterogeneity (20–300$\sim3,538$6 lower variance versus human subjects), and under-sensitivity to intent/distal-outcome effects.

Proposed future enhancements aim to:

• Integrate automated literature retrieval/citation for deeper theoretical grounding.
• Expand scenario libraries into domains of emotion and culture.
• Develop calibration methods to modulate agent heterogeneity against human benchmarks.
• Incorporate systematic robustness and sensitivity analyses across LLM backbones (Wang et al., 12 May 2025, Wang et al., 2 Apr 2026).

Summary Table: YuLan-OneSim Core Features

Feature	Implementation Overview	Quantitative Metrics/Empirical Results
Scenario Construction	ODD protocol, NL→code auto-generation	50+ templates, $\sim3,538$7/5 scenario quality
Distributed Simulation	Master–Worker, event-driven, gRPC	$\sim3,538$8k agents, $\sim3,538$9 events/s
Feedback-Driven Fine-Tuning	VR²T loop with SFT/DPO, LLM-as-Judge	$6,026$019.8–27.4\% DPO gain, error $6,026$1 30\% syntax
AI Social Researcher Automation	Inspiration/Evaluation/Reporting LLM agents	End-to-end loop, report quality $6,026$2/5
Canonical Research Modes	Induction/Deduction/Abduction workflows	Case studies validate model with survey/alignment

YuLan-OneSim establishes a technically rigorous foundation for automated, large-scale, LLM-based social experimentation and human–AI collaborative inquiry, supporting scenario construction, distributed agent simulation, continuous LLM improvement, and comprehensive reporting from a single platform (Wang et al., 2 Apr 2026, Wang et al., 12 May 2025).