LLM-Sim: Integrating LLMs with Simulators
- LLM-Sim is a simulation framework where language models bridge high-level reasoning and structured simulator execution.
- It integrates diverse roles such as planners, multi-agent participants, and system designers through explicit control interfaces.
- Applications span robotics, social and legal simulations, and infrastructure co-simulation with rigorous evaluation metrics.
Searching arXiv for papers relevant to “LLM-Sim” and the cited systems. Searching for "LLM-Sim" and related frameworks. LLM-Sim denotes a family of systems in which LLMs are coupled to simulators, simulated institutions, or simulation infrastructure so that the model acts as a planner, code generator, role-conditioned agent, reward or parameter designer, evaluator, or runtime orchestrator. Recent arXiv work uses the label explicitly in robotics and social simulation and motivates the same pattern in education, law, scientific computing, sim-to-real transfer, and LLM-serving co-simulation (Lu et al., 3 Jan 2026, Lee et al., 13 Apr 2026, Nasim et al., 10 Mar 2025, Ye et al., 28 May 2026, Yue et al., 8 Feb 2025, Cho et al., 26 Feb 2026).
1. Conceptual scope and emergence
From 2024 to 2026, the term appears across at least three related senses. In one sense, an LLM is placed inside a simulated world or institution and interacts as an agent, as in educational classrooms, legal proceedings, or social networks. In a second sense, an LLM is coupled to an external simulator as a planner or optimizer, turning natural language or structured feedback into executable simulation actions or parameter updates. In a third sense, the simulator targets the LLM system itself, as in hardware/software co-simulation for LLM inference serving (Ye et al., 28 May 2026, Yue et al., 8 Feb 2025, Zhang et al., 24 Aug 2025, Chen et al., 8 Jun 2026, Chen et al., 28 May 2026, Cho et al., 2024).
Taken together, these systems suggest that LLM-Sim is best understood as an umbrella label for simulation frameworks in which LLMs mediate between high-level reasoning and a structured execution environment. The shared core is not a single architecture but a recurring pattern: a symbolic or natural-language interface, explicit environment dynamics, bounded action or role spaces, and an evaluation layer that judges either outcome quality, process fidelity, or both.
| Mode of integration | Representative systems | Primary function |
|---|---|---|
| LLM as planner or code generator | EduSim-LLM, FoamPilot | Natural language to simulator actions or case edits |
| LLM as multi-agent participant | AgentSchool, MASER, SimCourt, Civil Court, Law in Silico | Role-conditioned interaction in institutional or social processes |
| LLM as simulator designer or optimizer | DrEureka, Battery-Sim-Agent | Reward, randomization, or parameter updates in closed loop |
| LLM as infrastructure subject or orchestrator | LLMServingSim, LLMServingSim 2.0 | Runtime-driven serving-system co-simulation |
| LLM as world-building component | Genie Sim PanoRecon | Background generation for embodied simulation |
| LLM-Sim evaluation layer | SLALOM | Process-fidelity validation via longitudinal metrics |
2. Recurrent architectures and formal interfaces
A defining architectural pattern is the use of constrained interfaces between the LLM and the environment. EduSim-LLM formalizes the control path as
with the LLM responsible for generating Python control code over a fixed action library and the backend responsible for deterministic execution in CoppeliaSim (Lu et al., 3 Jan 2026). Battery-Sim-Agent uses an analogous closed loop, but the interface is parameter rather than action oriented: where the LLM proposes structured updates and the simulator returns rich residuals and features (Chen et al., 28 May 2026). DrEureka similarly casts simulation design as
with the LLM proposing reward functions and domain-randomization distributions rather than direct policies (Ma et al., 2024).
A second recurrent pattern is explicit role separation. MASER instantiates Client, Lawyer, and Supervisor agents, with the Supervisor operating turn-by-turn and sentence-level to check profile alignment, agenda compliance, and distractor behavior (Yue et al., 8 Feb 2025). SimCourt uses 5 courtroom roles and 5 core trial stages, while the civil-court framework uses judge, plaintiff, and defendant agents organized through a five-stage civil trial procedure (Zhang et al., 24 Aug 2025, Chen et al., 8 Jun 2026). AgentSchool generalizes this to a partially observable multi-agent state transition process with student, teacher, and organizational agents, making the simulator’s state explicit rather than reducing everything to prompted dialogue (Ye et al., 28 May 2026).
A third pattern is hierarchical context management. Short-term interaction history, long-term memory summaries, planning modules, and retrieval layers recur across domains. SimCourt combines short-term memory, long-term memory, strategy construction, and reflection after each stage (Zhang et al., 24 Aug 2025). Civil Court simulation uses stage summaries as long-term memory and FAISS-based statute retrieval over more than 17,000 Chinese statutes and provisions (Chen et al., 8 Jun 2026). FoamPilot couples GPT-4o to shell, Python, and retrieval tools in a LangGraph loop, using a fused source-code index over .H and .C files for code insight (Xu et al., 2024). LLMServingSim 2.0 translates serving decisions into operator-level execution graphs and embeds them into a single runtime loop with network, memory, and power modeling (Cho et al., 26 Feb 2026).
3. Robotics and embodied environments
In robotics-oriented LLM-Sim, the LLM commonly operates above a fixed simulator API. EduSim-LLM is organized as a four-module pipeline—Natural Language Interface, LLM-based Instruction Planner, Simulation Control Backend, and User Interaction Frontend—and uses LangChain plus LLMs to generate Python code over a predefined function library for multiple KUKA YouBot-like robots in CoppeliaSim (Lu et al., 3 Jan 2026). The control library includes locomotion, manipulation, and perception primitives such as moveToXY, rotateToBeta, presetFold, closeGripper, and capturePhoto, while low-level physics and kinematics remain in the simulator. The paper distinguishes direct control, which resembles short-horizon teleoperation through natural language, from autonomous control, in which a single instruction is decomposed into a full multi-step, multi-robot script.
Genie Sim PanoRecon occupies a different layer of the embodied stack. It is not itself a planner, but a feed-forward Gaussian-splatting pipeline that converts one 360° panorama into a globally consistent 3D Gaussian scene for robotic manipulation simulation, and it is integrated into the LLM-driven Genie Sim platform as a scalable background generator (Li et al., 8 Apr 2026). Its pipeline fuses DA360 and DepthPro depth estimates in panoramic space, projects the fused result to six non-overlapping cube-map faces, applies training-free depth injection into SHARP, and merges the resulting Gaussians with depth-aware consistency constraints. This makes LLM-specified or LLM-conditioned world generation practical at the background-asset level.
DrEureka moves the LLM one level upstream again: the model does not control the robot in deployment, but designs the simulator that trains the robot. It automatically constructs reward functions and domain randomization distributions, then relies on standard RL for policy learning (Ma et al., 2024). On quadruped locomotion, dexterous manipulation, and the walking-globe task, the LLM proposes reward terms for stability, smoothness, and torque regularization and then, using the Reward-Aware Physics Prior, proposes physically plausible randomization ranges for parameters such as friction, payload mass, gravity, or motor strength. This establishes a distinct LLM-Sim variant in which the model edits the training environment rather than acting inside it.
These examples show three embodied roles for the LLM: planner over a safe API, world-builder for simulator assets, and designer of reward and physics variation. A plausible implication is that “embodiment” in LLM-Sim now spans the full chain from scene construction to policy training to interactive execution.
4. Social, educational, and legal institutional simulation
The most expansive use of LLM-Sim appears in institutional and social domains, where the simulator is not primarily geometric but procedural. AgentSchool models education as a partially observable multi-agent state transition process. Student agents have weighted subject knowledge graphs, thinking-workflow pools, explicit misconceptions, and episodic memory; teacher agents plan, scaffold, and reflect along the Zone of Proximal Development; and a configurable scenery generator situates activity across formal and informal learning fields (Ye et al., 28 May 2026). The simulator further decouples interaction scale, temporal granularity, and simulation duration, making education a long-horizon stateful environment rather than a classroom-themed role-play prompt.
Social influence and generative social-science simulators push this institutional logic into networked populations. “Simulating Influence Dynamics with LLM Agents” couples Deffuant-style bounded confidence dynamics with two strategic LLM broadcasters, Red and Blue, which alternate in generating persuasive misinformation and factual rebuttals for a directed social network of Green agents (Nasim et al., 10 Mar 2025). Law in Silico extends the same idea to a legal society: individuals are sampled from socio-demographic distributions and interact with LLM-based legislature, judiciary, and enforcement agents under explicit legal rules and sanctions, producing both macro-level crime trends and micro-level labor–firm conflicts (Wang et al., 28 Oct 2025). Civil Court Simulation with LLMs and SimCourt both operationalize courtroom procedure as staged, role-bound interaction, but the former targets Chinese civil cases and emphasizes claims, liability allocation, and multi-item adjudication, while the latter targets Chinese criminal trials and replicates all 5 core stages of a Chinese trial with 5 courtroom roles (Chen et al., 8 Jun 2026, Zhang et al., 24 Aug 2025).
MASER adds a data-generation and evaluation perspective to legal LLM-Sim. It grounds Client, Lawyer, and Supervisor agents in real judgment documents, enforces role behavior via supervision, and uses the resulting multi-turn legal scenarios to fine-tune legal models and to build the Multi-stage Interactive Legal Evaluation benchmark (Yue et al., 8 Feb 2025). In this form, simulation is simultaneously a synthetic-data engine, an evaluation environment, and a procedural regularizer.
Across these systems, the common substrate is explicit institutionality: roles are asymmetric, procedures are ordered, and memory is indispensable. This suggests that one of the strongest contemporary uses of LLM-Sim is not open-ended conversation but the emulation of norm-governed processes.
5. Scientific, engineering, and infrastructure co-simulation
A parallel line of work couples LLMs to scientific simulators and engineering workflows rather than to social institutions. FoamPilot is a proof-of-concept LLM agent for FireFOAM/OpenFOAM that provides code insight, case configuration, and job execution functionality through shell, Python, and RAG tools (Xu et al., 2024). It supports navigation of the FireFOAM source tree, natural-language modification of simulation cases, and serial or HPC execution through SLURM-oriented prompting. Here the simulator is not a virtual society but a CFD codebase and its surrounding execution environment.
Battery-Sim-Agent is the first framework to deploy an LLM agent in a closed loop with a high-fidelity battery simulator for inverse parameter estimation (Chen et al., 28 May 2026). Built on PyBaMM and the Doyle–Fuller–Newman model, it provides the model with residuals, feature descriptors, and curve overlays, and receives JSON-formatted parameter updates in return. The warm-up phase builds an internal sensitivity map, after which the agent iteratively narrows discrepancy between simulated and observed voltage, current, capacity, and long-horizon degradation behavior. This is a scientific LLM-Sim pattern in which reasoning replaces black-box optimization.
DrEureka, though focused on robotics, belongs to the same engineering family because it uses the LLM to modify reward functions and domain randomization rather than to act inside the trained controller (Ma et al., 2024). The simulator remains the locus of optimization, and the LLM supplies structured design hypotheses.
LLMServingSim and LLMServingSim 2.0 represent a further shift: the simulated system is the LLM-serving infrastructure itself. LLMServingSim models dynamic autoregressive serving at iteration granularity, reuses computation across decoder blocks, and reports less than 14.7% error rate while offering 91.5x faster simulation speed compared to existing accelerator simulators (Cho et al., 2024). LLMServingSim 2.0 extends this into a unified runtime-driven simulator for heterogeneous and disaggregated serving infrastructures, supporting batching, routing, offloading, memory, and power in a single loop, with an average error of 0.97% and simulation times of around 10 minutes even for complex configurations (Cho et al., 26 Feb 2026). In this usage, LLM-Sim no longer means “a simulator with LLM agents” but “a simulator for LLM systems,” which broadens the term’s technical scope substantially.
6. Evaluation and validation paradigms
A central divide in LLM-Sim evaluation is between endpoint correctness and process fidelity. SLALOM addresses this explicitly for social simulation by arguing that end-state matching alone creates a “stopped clock” problem. It treats social phenomena as multivariate time series, defines SLALOM gates as intermediate waypoint constraints, and uses Dynamic Time Warping to align simulated and empirical trajectories (Lee et al., 13 Apr 2026). The framework operationalizes structural realism as passing the right sequence of phase-specific regions rather than merely landing on the right final statistic.
Task-specific systems layer their own metrics on top of this distinction. EduSim-LLM uses a 108-instruction dataset split into simple, composite, and complex levels; reports success rates of 100%, 94.4%, and 88.9%; and shows that natural-language control reduces human operation time relative to manual control by more than 17 s on every complex task tested (Lu et al., 3 Jan 2026). SimCourt evaluates imprisonment, probation, and fine prediction with hit rate and relative error, and also uses human experts to compare simulated and real trial processes over 30 aspects of courtroom behavior (Zhang et al., 24 Aug 2025). Civil Court simulation adopts a weighted score
thereby separating judgment conclusion consistency, liability allocation, quantitative precision, legal basis, and multi-item adjudication (Chen et al., 8 Jun 2026). MASER’s MILE benchmark similarly separates interaction evaluation—Interactivity, Professionality, and Logicality—from goal evaluation of complaint sections and overall legal drafting quality (Yue et al., 8 Feb 2025).
Engineering and systems simulators emphasize fidelity against physical or operational ground truth. Battery-Sim-Agent compares against Bayesian optimization and reports large improvements in identifying accurate parameters across diverse chemistries and conditions, while also demonstrating real-world battery fitting (Chen et al., 28 May 2026). DrEureka compares LLM-designed reward and randomization against human-designed baselines and against iterative DR optimizers such as CEM and BayRn, with real-robot deployment as the decisive criterion (Ma et al., 2024). LLMServingSim 2.0 validates throughput, latency, memory, prefix hit rate, and power against real GPU and TPU deployments (Cho et al., 26 Feb 2026).
These evaluation practices indicate that LLM-Sim is converging on plural validation rather than single-score benchmarking: execution success, human effort, trajectory similarity, legal outcome structure, parameter recovery, and infrastructure realism all appear as legitimate but domain-specific notions of correctness.
7. Limitations and future directions
Several limitations recur across the literature. Robustness degrades with complexity: EduSim-LLM’s accuracy drops to 88.9% on complex tasks, and its planning remains open-loop without state-feedback replanning (Lu et al., 3 Jan 2026). Genie Sim PanoRecon inherits SHARP-like limits on unseen-region completion, assumes approximately manipulation space with limited parallax, and models static backgrounds rather than dynamic scenes (Li et al., 8 Apr 2026). SLALOM depends on high-quality longitudinal data and assumes a monotonic temporal path, making branching or looping social processes harder to assess (Lee et al., 13 Apr 2026).
Backbone dependence is another cross-cutting issue. AgentSchool reports backbone-dependent patterns in both student and teacher behavior and treats model choice as an explicit experimental condition (Ye et al., 28 May 2026). Legal simulators remain jurisdiction-specific: MASER focuses on Chinese civil complaint drafting, SimCourt on Chinese criminal procedure, and Civil Court on Chinese civil cases (Yue et al., 8 Feb 2025, Zhang et al., 24 Aug 2025, Chen et al., 8 Jun 2026). Law in Silico reproduces macro-level crime trends but also highlights the risk that LLM priors may reflect underreporting, stereotypes, or broader model bias (Wang et al., 28 Oct 2025).
Scientific and systems co-simulators face different ceilings. FoamPilot performs well on simple tasks but is fragile for multi-step HPC workflows and more complex case modifications (Xu et al., 2024). Battery-Sim-Agent does not eliminate parameter non-identifiability and remains constrained by simulator stability and backbone quality (Chen et al., 28 May 2026). LLMServingSim 2.0 is system-level rather than microarchitectural; it abstracts away fine-grained accelerator internals even as it models runtime interaction very effectively (Cho et al., 26 Feb 2026).
Future work in the corpus is correspondingly diverse: closed-loop replanning and real-robot transfer in robotics, richer interactive scene generation in embodied world building, improved memory representations and broader role sets in legal and educational simulators, hybrid symbolic–LLM legal reasoning, stronger calibration to longitudinal social data, and more comprehensive support for emerging hardware and disaggregated memory in serving simulators (Lu et al., 3 Jan 2026, Li et al., 8 Apr 2026, Ye et al., 28 May 2026, Yue et al., 8 Feb 2025, Cho et al., 26 Feb 2026). A plausible implication is that the next phase of LLM-Sim will be defined less by adding more agents and more by improving grounding, memory, verification, and controlled interfaces between language-model reasoning and simulated dynamics.