Optimizing Service Operations via LLM-Powered Multi-Agent Simulation

Abstract: Service system performance depends on how participants respond to design choices, but modeling these responses is hard due to the complexity of human behavior. We introduce an LLM-powered multi-agent simulation (LLM-MAS) framework for optimizing service operations. We pose the problem as stochastic optimization with decision-dependent uncertainty: design choices are embedded in prompts and shape the distribution of outcomes from interacting LLM-powered agents. By embedding key numerical information in prompts and extracting it from LLM-generated text, we model this uncertainty as a controlled Markov chain. We develop an on-trajectory learning algorithm that, on a single simulation run, simultaneously constructs zeroth-order gradient estimates and updates design parameters to optimize steady-state performance. We also incorporate variance reduction techniques. In a sustainable supply chain application, our method outperforms benchmarks, including blackbox optimization and using LLMs as numerical solvers or as role-playing system designers. A case study on optimal contest design with real behavioral data shows that LLM-MAS is both as a cost-effective evaluator of known designs and an exploratory tool that can uncover strong designs overlooked by traditional approaches.

• The paper introduces an LLM-powered multi-agent simulation framework that converts service system design into a decision-dependent stochastic optimization problem.
• It leverages hierarchical prompt engineering and a novel on-trajectory learning algorithm with guided perturbation and residual feedback to reduce variance.
• Empirical validations in sustainable supply chains and innovation contests demonstrate superior performance and uncover counterintuitive design insights compared to traditional methods.

Optimizing Service Operations via LLM-Powered Multi-Agent Simulation

Problem Motivation and Framework

The paper addresses the challenge of service system design where participant responses to operational levers—such as policy, incentive, or interface choices—are critical for performance, yet difficult to model analytically due to human behavioral complexity. Analytical models are hampered by limited rationality, rich qualitative contexts, and the costly acquisition of calibration data. To surmount these limitations, the authors propose an LLM-powered multi-agent simulation (LLM-MAS) framework that utilizes LLMs as behavioral proxies for human agents. Key design parameters are numerically embedded in agent prompts, thereby allowing the system dynamics and uncertainties induced by human-like decision making to be modulated directly via prompt engineering.

The resulting framework converts the system design problem into a stochastic optimization problem with decision-dependent uncertainty, conceptualized as a controlled Markov chain. This enables both the evaluation of candidate designs under conditions of human-like behavior and the optimization of such designs through end-to-end simulation-based methods.

Figure 1: Schematic for LLM-MAS in a three-echelon sustainable supply chain management context, where agents (manufacturer, retailer, consumer) interact under policy parameters and generate system-level outcomes.

The framework operates as a semantic digital twin: agent interactions, contextualized by relevant past and present numerical/textual information and guided by role-conditional prompts, yield trajectories for system-level state variables. These evolve according to the chosen design $\theta$ and comprise the basis for downstream optimization.

Figure 2: Overview of the LLM-MAS for system design optimization as a controlled Markov chain, where design $\theta$ induces system transitions and stationary distributions.

Hierarchical Propagation and Decision-Dependent Uncertainty

A distinctive feature of LLM-MAS is the hierarchical propagation of prompt-embedded parameters through the system: design choices such as taxes or subsidies are incorporated into prompts at each timestep, modulating token-level probabilities in autoregressive LLM outputs. Because LLM-generated text is both stochastic and high-dimensional, even small variations in prompt variables can induce significant distributional shifts in downstream system behavior and outcomes.

Figure 3: Illustration of how design parameter $\theta$ modulates hierarchical distributions from token-level output to agent actions and, ultimately, to global system state.

The authors highlight the Markovian nature of the induced system dynamics—each state depends on the prior only via current agent memories and the current embedded design. Over time, as the simulated system reaches stationarity, performance under a given design can be robustly estimated.

Algorithmic Contributions: Zeroth-Order On-Trajectory Learning

A central challenge in optimizing over such a simulation environment is the intractability of gradients. The mapping from design choices to system performance is complicated by decision-dependent, high-dimensional, and non-analytical uncertainty. The paper circumvents the need to explicitly compute or differentiate through the measure $\mu_\theta$ by developing an on-trajectory learning (OTL) algorithm rooted in simultaneous perturbation stochastic approximation (SPSA).

This approach uses a single Markov trajectory over which both design updates and gradient estimates are computed, using two perturbed versions of the underlying simulation at each step. To control the bias induced by using transient rather than stationary samples for gradient estimation, the algorithm employs two timescales (fast for gradient tracking, slow for parameter update), along with vanishing step sizes.

The theoretical analysis leverages a Poisson equation framework to control the gap between transient and stationary estimation and to guarantee almost sure convergence of iterates to stationary points, despite the adaptive and controlled Markovian structure.

The algorithm is further enhanced by two variance reduction methods:

• Guided perturbation (GP): Leverages explicit dependence of $F$ on $\theta$ to direct search along likely descent directions.
• Residual feedback (RF): Exploits information from previous iterates to construct control variates, reducing sample variance when $F$ depends on $\theta$ only implicitly.

Prompt Engineering and Extraction Interface

A critical engineering detail is the design of prompt templates that embed numerical design parameters and system context into semantically coherent agent prompts. The templates incorporate agent role, attributes (heterogeneity), the numerically parameterized design, simulation context, and explicit output requirements (including action variables and reasoning chains). Outputs are then parsed (e.g., via structured JSON) for extraction of relevant state and action data.

Figure 4: General and concrete prompt templates for embedding design and context information in LLM agent prompts.

Figure 5: Timeline of agent-environment interaction within a simulation round, detailing event order and exchange of contextual and action data.

Empirical Validation: Sustainable Supply Chain Application

Using a stylized supply chain with manufacturer, retailer, and consumer agents, the authors benchmark OTL and OTL-GP against several baselines:

• Bayesian optimization (BO): Black-box optimization with expensive long-run sample trajectories.
• LLM-as-a-Solver: The LLM itself is prompted to propose numerical designs directly.
• LLM-as-a-Designer: The LLM role-plays as a policy designer, leveraging system feedback.

OTL and OTL-GP demonstrate superior performance in speed and reliability, converging to effective policies with substantially fewer LLM queries due to their ability to learn efficiently from correlated data along a single trajectory. In contrast, BO and LLM-based heuristics suffer from high variance, sample inefficiency, or lack of search discipline.

Figure 6: Comparative performance of OTL, OTL-GP, BO, and LLM/heuristic baselines in supply chain design optimization; OTL-GP achieves the steadiest and fastest descent.

Case Study: Innovation Contest Design

As further validation, the authors apply LLM-MAS to replicate and optimize design in innovation contests, using behavioral data from human laboratory experiments. Here, agent personas are directly initialized from empirical participant attribute data.

Key findings:

• LLM-MAS accurately recapitulates entry and effort behaviors observed in the lab, including systematic deviations from Nash equilibrium predictions such as suboptimal entry rates and probabilistic effort choices.
• Rich agent reasoning can be directly extracted from LLM outputs, allowing for mechanistic post hoc explanations of behavioral variance—including risk aversion and competitiveness modulation.

Moreover, by releasing the game-theoretic coupling constraints and directly optimizing all design parameters via the RF-augmented MTL algorithm, LLM-MAS discovers alternative optimal designs that differ substantially from traditional game-theoretic recommendations. For example, it identifies that excessively high entry barriers (as prescribed by the theory under high liability) actually suppress participation and effort when behavioral/psychological factors (embedded in LLM outputs) are at play.

Figure 7: Analytical predictions of optimal contest design (liability vs. total effort) under Nash equilibrium.

Figure 8: Replication of contestant behavior (entry, effort versus ability) by LLM-MAS and in laboratory data across various contest configurations.

Figure 9: MTL-RF optimization trajectory over contest design parameter space; LLM-MAS optima diverge from theoretical predictions.

Figure 10: Comparison of total contestant effort under designs identified by LLM-MAS optimization, LLM-MAS under game-theoretic parameters, and the analytical benchmark—the LLM-MAS solution yields higher realized effort in the behavioral regime.

Figure 11: Behavioral regime identified by LLM-MAS (entry and effort as a function of ability) for the empirically optimized design.

Implications, Limitations, and Future Directions

This work establishes LLM-MAS as a principled computational twin for evaluating and optimizing service operations under realistic behavioral uncertainty. The empirical evidence shows that LLM-based agent models, when used in conjunction with rigorous stochastic optimization techniques, can serve as cost-effective testbeds for both the evaluation of theoretical designs and the discovery of high-performing alternatives that traditional methods miss.

Key implications include:

• LLMs are complements—not substitutes—for OR/OM optimization protocols; plain-language LLM agents or solvers alone underperform mathematically grounded search methods.
• Embedding human-like behavioral complexity in design optimization can surface counterintuitive or superior configurations, challenging conventional wisdom from theoretical models.
• The interpretability of LLM reasoning can supply actionable diagnostics for "why" a design works or fails.

The paper identifies scaling, complexity control, and high-dimensional search as principal avenues for future work, including the use of meta-models for agent or system surrogacy, richer inter-agent communication protocols, more advanced variance reduction, and multimodal LLM simulation environments.

Conclusion

This paper introduces a comprehensive and theoretically robust framework for system design optimization via LLM-powered multi-agent simulation. It integrates contemporary advances in foundation models, agent-based modeling, and stochastic optimization, validated in both synthetic and behavioral-data-driven contexts. Strong empirical results, theoretical underpinnings, and actionable insights demonstrate the value and feasibility of LLM-MAS as a strategic research and operational tool for service system design.