StockSim: Dual-Mode LLM Simulator

• StockSim is a dual-mode simulation platform that integrates order-level microstructure and candlestick-level execution to evaluate LLM-based agents.
• It implements asynchronous agent APIs with realistic market features like latency, slippage, and price–time priority for order matching.
• The simulator provides comprehensive metrics such as ROI, Sharpe ratios, and drawdown, supporting systematic agent comparison and calibration research.

Searching arXiv for the primary StockSim paper and closely related market-simulation frameworks. StockSim is an open-source, dual-mode order-level simulator for evaluating LLMs in financial markets. It was introduced as a platform that combines an order-level microstructure simulator and a candlestick-level execution mode behind a single asynchronous agent API, with explicit attention to latency, slippage, order-book microstructure, external information, reproducibility, and multi-agent coordination (Papadakis et al., 12 Jul 2025). In the broader literature on artificial markets, StockSim sits within a lineage that includes minimal continuous double auction simulators, Python-based agent platforms, and exchange-like microstructure environments, but its defining emphasis is systematic evaluation of LLM-based agents across execution regimes rather than solely market design, pedagogy, or stress testing (Cliff, 2018).

1. Definition and scope

StockSim is designed as a simulation platform for realistic financial decision-making scenarios involving LLM-based agents. Its stated purpose is to address two gaps: first, the limited fidelity of bar-level backtesting platforms that abstract away latency, slippage, partial fills, and microstructure; second, the limited practicality of high-fidelity order-book simulators that depend on costly order-level datasets and are not packaged for multi-agent LLM evaluation with external information such as news and fundamentals (Papadakis et al., 12 Jul 2025).

The platform is “dual-mode” in a specific sense. It provides an order-level microstructure mode with price–time priority, queueing, partial fills, and latency, and a candlestick-level mode intended for scalability and long-span historical evaluation. Agents are written once and evaluated across modes by a configuration change rather than by rewriting policy logic or data interfaces (Papadakis et al., 12 Jul 2025). This unification of execution regimes is the central architectural claim of the system.

Within the wider literature, StockSim is distinct from platforms whose primary goal is educational transparency, minimalism, or exchange realism for general algorithmic trading. BSE, for example, is a minimal Python 2.7 single-file continuous double auction simulator built for teaching and research, with a simplified limit-order-book design and quantity fixed at 1 (Cliff, 2018). By contrast, StockSim is positioned as a testbed for multi-agent LLMs under asynchronous execution and heterogeneous information flows (Papadakis et al., 12 Jul 2025).

2. System architecture and execution model

StockSim is organized around an exchange simulation engine, market-data and event scheduling, a limit order book module, latency and slippage handling, an indicator engine, an agent framework, and an evaluator. Agents run as separate processes and communicate with the engine via RabbitMQ. The engine schedules market events and agent messages in simulated time and resolves them deterministically (Papadakis et al., 12 Jul 2025).

The order-level mode is event-driven. It consumes order-event streams with millisecond timestamps, maintains bids and asks as price-indexed FIFO queues, and advances to the next market event or agent action arrival. Matching follows price–time priority. A buy limit order of size $Q$ at price $p$ matches against the best ask while $\text{best\_ask\_price} \le p$, consuming volume level-by-level in FIFO order; any residual quantity is then enqueued at the bid price after latency is applied (Papadakis et al., 12 Jul 2025).

Latency is modeled operationally through asynchronous message handling and event timing. If an agent submits at time $t_{\text{sub}}$, the effective arrival time is

$t_{\text{arrive}} = t_{\text{sub}} + \Delta,$

where $\Delta$ is induced by the simulator’s asynchronous architecture and discrete-event scheduling. The paper does not specify a fixed parametric distribution for $\Delta$; instead, latency emerges from event ordering in simulated time (Papadakis et al., 12 Jul 2025).

Slippage in order mode is likewise emergent rather than imposed by a closed-form impact law. It arises when the book moves between submission and arrival or when an order consumes multiple price levels. The realized execution price is the volume-weighted average of consumed levels,

$$p_{\text{exec}} = \frac{\sum_i q_i p_i}{\sum_i q_i},$$

with $q_i$ the fill at level $p_i$ (Papadakis et al., 12 Jul 2025). This is materially different from reduced-form simulators that hard-code an impact curve.

The candlestick-level mode operates bar-by-bar rather than event-by-event. It uses OHLCV bars, keeps outstanding conditional orders, and simulates a plausible intra-bar path consistent with the bar’s open, high, low, and close so that conditional orders such as STOP orders can trigger. In this mode there is no explicit queueing or price–time competition between agents; execution depends on whether the order price is reachable within the simulated bar path (Papadakis et al., 12 Jul 2025).

This architecture places StockSim between several established design choices in the literature. PAMS is also Python-based and designed for easy deep-learning integration, but it is a tick-time, event-driven CDA platform centered on agent-based experiments rather than a dual-mode evaluation system (Hirano et al., 2023). SHIFT, by contrast, emphasizes a distributed, exchange-like environment with FIX connectivity, NBBO routing, and asynchronous multi-agent interaction under U.S. equity market rules (Alves et al., 2020). StockSim adopts asynchronous exchange semantics, but its novelty lies in combining these with an LLM-first evaluation stack and a mode switch between order-level and candlestick-level execution (Papadakis et al., 12 Jul 2025).

3. Market microstructure and agent framework

In order-level mode, StockSim maintains a full limit order book with bids and asks represented as price-indexed FIFO queues. Supported order types are MARKET and LIMIT, and STOP orders become market orders upon trigger. Submission and cancellation are handled explicitly; modification is represented through cancellation followed by new submission on the simulator side (Papadakis et al., 12 Jul 2025).

The matching rule is price–time priority. If a buy limit order arrives at price $p$0 and size $p$1, it executes against the ask book while the best ask is no greater than $p$2. Partial fills are supported, residual quantities remain on the book, and queue position matters because later-arriving orders at the same price are placed behind earlier ones (Papadakis et al., 12 Jul 2025). These are precisely the microstructure features that bar-level backtesting frameworks typically abstract away.

The agent framework is role-based and modular. A single LLMTradingAgent can coordinate specialist roles such as market or technical analyst, news analyst, fundamental analyst, and trader or allocator. Agents can subscribe to order-book data in order mode or OHLCV data in candlestick mode; they may also consume indicators and external data such as news headlines, earnings calendars, splits, dividends, and fundamentals (Papadakis et al., 12 Jul 2025).

The action space includes MARKET, LIMIT, and STOP orders, with side, quantity, and optional rationale attached to each action. The same asynchronous message schema is used across both modes, which means observation interfaces and order APIs remain stable even though the execution semantics differ materially between microstructure and bar-based simulation (Papadakis et al., 12 Jul 2025).

A concise summary of the two execution regimes is useful.

Mode	State and timing	Execution semantics
Order-level	Full LOB, event-by-event, millisecond timestamps	Price–time priority, queueing, partial fills, latency
Candlestick-level	OHLCV bars, bar-by-bar	Reachability within intra-bar path, no explicit queueing

This design reflects a broader methodological divide in financial simulation. Minimal simulators such as BSE implement a continuous double auction with aggregated published book state and at-most-one resting order per trader, making them readable and experimentally transparent (Cliff, 2018). Exchange-like systems such as SHIFT incorporate richer order-driven rules and asynchronous multi-agent connectivity (Alves et al., 2020). StockSim adopts microstructure realism when data permit, but also permits broad-coverage bar-level studies without changing the agent code path (Papadakis et al., 12 Jul 2025).

4. Evaluation methodology and reported results

StockSim includes an evaluator that reconstructs positions and cash from fills and computes standard trading metrics. The paper explicitly lists ROI, Sharpe, Sortino, maximum drawdown, win rate, and profit factor, and exports JSON reports and interactive plots (Papadakis et al., 12 Jul 2025). The standard formulas used are:

$p$3

$p$4

$p$5

$p$6

The platform was validated for determinism and resource usage using deterministic agents such as moving-average crossover and buy-and-hold, with LLM variability excluded from this validation step. On a MacBook Pro with an Apple M3 Pro 11-core CPU and 18 GB RAM, StockSim scaled nearly linearly up to about 150 agents, and at 500 agents peak memory use was about 5.6 GB RAM. Under controlled agent sets, outputs such as orders, executions, and metrics remained identical across runs, which the paper treats as empirical validation of deterministic engine behavior (Papadakis et al., 12 Jul 2025).

The demonstration involving LLMs used candlestick mode on NVDA over the period 2025-04-28 to 2025-06-28, with orders placed before market open. Two models, GPT-o4-mini and GPT-o3, were evaluated under the same prompt and setup. The reported results were:

Metric	GPT-o4-mini	GPT-o3
ROI	0.0734	0.2956
Annualized Sharpe	2.6218	5.9682
Max Drawdown	0.0306	0.0323

Additional reported metrics include Sharpe 0.1652 versus 0.3760, Sortino 0.2868 versus 1.0587, win rate 0.6667 versus 1.0, number of trades 31 versus 9, total traded volume 931,416.78 versus 368,306.25, profit per trade 258.47 versus 4,520.13, and realized PnL 5,427.80 versus 27,120.75 for GPT-o4-mini and GPT-o3 respectively (Papadakis et al., 12 Jul 2025). The paper interprets this as GPT-o3 being more selective and producing larger, higher-conviction trades.

The evaluator’s metric set is consistent with broader practice in agent-based and market-simulation research, although other simulators emphasize different observables. BSE foregrounds trader PnL, price efficiency relative to a target equilibrium $p$7, volatility, and liquidity at the best levels (Cliff, 2018), while PAMS uses per-tick logs to support forecasting metrics and trading-performance statistics (Hirano et al., 2023). StockSim’s metric suite is more directly oriented toward model-comparison experiments for LLM trading agents (Papadakis et al., 12 Jul 2025).

5. Relation to adjacent simulators and research lineages

StockSim belongs to a larger ecosystem of artificial market simulators, but its design choices differ from adjacent systems in emphasis, abstraction, and intended workload.

BSE is a minimal centralized limit order book operating a continuous double auction. Its defining features are price–time priority, an anonymized aggregated book, simple order replacement logic, fixed quantity equal to 1, and a single-file Python implementation intended for novice programmers and classroom use (Cliff, 2018). This makes it highly transparent, but it omits many execution effects that StockSim foregrounds.

PAMS is a Python-based, event-driven, tick-time platform designed to make agent-based market experiments easy to modify and integrate with deep learning. It supports custom agents, sessions that separate order placement and execution, and an FCNAgent baseline, and it demonstrates integration of an LSTM-based predictive agent (Hirano et al., 2023). Relative to StockSim, PAMS is closer in language and modifiability, but it is not organized around dual-mode execution or LLM-first evaluation.

SHIFT is an exchange-like simulation platform implemented in C++ with FIX 5.0 SP2 connectivity, market and limit orders, FIFO price-time priority, NBBO routing, and brokerage-layer constraints such as buying power, transaction fees, and margin calls. It supports both replay mode and fully artificial mode and is explicitly aimed at market microstructure research, policy testing, and stress scenarios (Alves et al., 2020). Relative to StockSim, SHIFT is more exchange-replica-like, whereas StockSim emphasizes integrated evaluation workflows for heterogeneous LLM agents.

MAXE provides a message-driven C++ core with a Python API, configurable matching rules including pro-rata, and explicit latency modeling via scheduled message delivery. Its focus is microstructure, latency effects, and reinforcement-learning readiness (Belcak et al., 2020). This suggests a nearby design lineage for StockSim’s asynchronous semantics, though StockSim’s dual-mode structure and role-based LLM agent architecture are distinct (Papadakis et al., 12 Jul 2025).

A recurring misconception is that all “stock simulators” are merely historical replayers or simple backtesters. That characterization fits neither BSE, where transaction prices emerge endogenously from a live LOB (Cliff, 2018), nor SHIFT, where agents compete asynchronously for finite liquidity (Alves et al., 2020), nor StockSim, where order-level execution includes queueing, latency, and partial fills (Papadakis et al., 12 Jul 2025). Another misconception is that microstructure simulators are inherently incompatible with modern ML workflows; PAMS and StockSim were both explicitly developed to reduce that integration friction (Hirano et al., 2023).

6. Calibration, limitations, and extensions

The StockSim paper emphasizes execution realism, evaluation, and extensibility, but it deliberately does not prescribe a parametric model for latency, slippage, or impact. In order mode, these effects emerge from event timing and book dynamics; in candlestick mode, they emerge from path-crossing logic within a bar. The paper explicitly notes that it does not fix the exact stochastic model or calibration procedure for these effects (Papadakis et al., 12 Jul 2025).

This has two implications. First, realism depends materially on the richness of supplied data, especially in order-level mode. Second, calibration becomes a distinct research problem rather than a built-in component of the platform. A relevant methodological development is deep calibration of market simulators using embedding networks and neural density estimators, where a simulator $p$8 is calibrated to observed traces $p$9 by learning an embedding $\text{best\_ask\_price} \le p$0 and a conditional posterior approximation $\text{best\_ask\_price} \le p$1 (Stillman et al., 2023). The Bayesian target is

$\text{best\_ask\_price} \le p$2

and the training objective is

$\text{best\_ask\_price} \le p$3

That work was framed as a guide for adding deep calibration to a stock market simulator like “StockSim,” particularly for agent-based simulators with limit-order-book dynamics (Stillman et al., 2023). This suggests a natural extension of StockSim from evaluation platform to calibrated empirical simulator, but such integration is not claimed in the StockSim paper itself.

The stated limitations of StockSim are several. In candlestick mode, queueing and latency are abstracted and within-bar path generation is not specified in detail. In order mode, realism depends on the availability of high-fidelity order data. The platform acknowledges that it cannot capture all real-world liquidity constraints and market-impact complexities, especially in bar mode. Multi-agent LLM studies may also be limited by inference cost even when the engine itself scales well (Papadakis et al., 12 Jul 2025).

These limitations parallel trade-offs found elsewhere in the literature. BSE sacrifices realism for transparency (Cliff, 2018); PAMS prioritizes modifiability and Python-native deep-learning integration (Hirano et al., 2023); SHIFT emphasizes exchange realism but does not target LLM experimentation (Alves et al., 2020). StockSim’s main contribution is therefore not a claim of maximal realism along every dimension, but a particular compromise: realistic enough execution for microstructure-sensitive evaluation, scalable enough bar-mode testing for broad studies, and a unified asynchronous agent API suitable for systematic LLM comparison (Papadakis et al., 12 Jul 2025).