Testing replication for an agent-based model of market fragmentation and latency arbitrage

Abstract: This study strengthens the foundations of multi-venue market modeling by attempting an independent replication of Wah and Wellman's 2016 model of latency arbitrage in a fragmented market. We find that faithful replication is hindered by missing implementation details in the original paper and limited quantitative reporting. We demonstrate that increasing the number of simulation runs beyond the original design allows for the creation of bootstrap confidence intervals to support rigorous tests of quantitative alignment, compensating for lacking distributional information (e.g. variance). We also demonstrate that increased complexity across the modeled scenarios corresponds with increased difficulty aligning to the original results. We draw on a codebase released by the original authors in connection with a later paper to recover additional implementation details; however, we reject quantitative alignment between that codebase and the published results. Combining information from the paper and the released code, we achieve relational equivalence for most metrics but reject quantitative alignment for model settings where latency is non-zero. We show that many of the qualitative takeaways from the original paper on the effects of market fragmentation and latency arbitrage are sensitive to the specifics of a `greedy strategy' extension given to the zero-intelligence (ZI) trader agents. Under an alternative interpretation of this strategy, we find that market fragmentation decreases execution times in all experiments and increases trader welfare in most experiments. Finally, to facilitate future replication, critique, and extension, we provide an ODD (Overview, Design concepts, Details) protocol for our implementations of the model.

• The paper replicates a leading agent-based model and highlights its sensitivity to detailed implementations of zero-intelligence trading strategies.
• It scrutinizes model performance using extensive simulations, revealing quantitative divergences in metrics like trader surplus and latency arbitrage profits.
• The study underlines implications for regulatory modeling by emphasizing the importance of transparent, reproducible, and rigorously validated ABMs.

Replication Assessment of Agent-Based Modeling of Market Fragmentation and Latency Arbitrage

---

Introduction and Motivations

This paper conducts a critical replication of Wah and Wellman’s 2016 agent-based model (ABM), which analyzes the influence of market fragmentation and latency arbitrage on market efficiency and liquidity. The original model is notable for its inclusion of multiple continuous double auction (CDA) exchanges, communication latencies, and dedicated latency arbitrage (LA) agents—elements that are essential for modeling realistic U.S. stock market microstructure. The authors' replication effort is motivated both by the importance the SEC places on model validity for policy guidance and the lack of mature, reproducible ABM frameworks for market microstructure questions. The replication scrutinizes model implementation, quantitative reporting, and sensitivity to agent strategy specification.

---

Wah and Wellman’s Model: Structure and Features

Wah and Wellman’s model simulates a single-asset market with varying levels of fragmentation, communication latency, and the presence or absence of a theoretically idealized LA agent. Core agent types are zero-intelligence (ZI) background traders and, conditionally, a latency arbitrageur. The exchanges operate as CDAs (described in Figure 1), and market information aggregation is handled via a Security Information Processor (SIP), introducing cross-market communication delays.

Figure 1: Logic of a continuous double auction (CDA) exchange matching engine, showing order book updates and transaction flow.

The model underwent substantial evolution between its 2013 and 2016 versions. The 2016 revision added multi-round trading, more sophisticated surplus-seeking for ZI agents, and heterogeneity in ZI strategies calibrated via empirical game-theoretic analysis (EGTA). The model outputs include trader surpluses, LA profits, execution time, and spread metrics collected across three main “environments” with varying numbers of traders, arrival rates, and mean-reversion in fundamental value.

Figure 2: Visual ODD diagram of the WW model, detailing agent information flow, event scheduling, and key architectural elements.

---

Replication Methodology

Given incomplete reporting in the original publication (notably, absence of variance or distributional statistics), the authors perform replication at multiple levels of model equivalence, as defined by Axtell et al.: numerical identity, distributional equivalence, and relational (qualitative) equivalence. Recognizing that numerical identity is unattainable due to stochasticity, and that distributional equivalence cannot be tested without original run-level data, the replication focuses on “quantitative alignment” via bootstrap confidence intervals against reported means.

Three distinct implementations are tested:

1. BestGuess—faithful to the published text, filling gaps with plausible logic.
2. MarketSim—using the codebase released by the original authors, albeit with uncertain alignment to the published results.
3. BestGuess+MS—a hybrid that incorporates greedy strategy logic observed in the MarketSim codebase.

The replication increases simulation run volume an order of magnitude over the original experiments to construct reliable bootstrap confidence intervals for mean statistics.

---

Replication Outcomes

Quantitative Alignment

Despite extensive simulation (5,000 mixtures × 100 runs = 500,000 per experiment), the replication fails to establish quantitative alignment with the original reported means for most scenarios beyond the trivial (single-market) case, regardless of the implementation variant. Bootstrap CIs for mean surplus systematically exclude the original values in fragmented, latency-impaired environments, and especially in settings with latency arbitrage.

The MarketSim codebase, although nearly contemporaneous, also fails to quantitatively align with the reported WW results, despite successfully replicating subsequent work by the same authors. This indicates silent divergence in implementation details between published results and the released code, a problematic pattern for agent-based modeling transparency.

Sensitivity to Greedy ZI Strategy Specification

A critical finding is the acute sensitivity of outcomes to the minutiae of ZI agent “greedy” surplus-seeking logic. Under the BestGuess interpretation (traders exploit both BBO and NBBO to maximize surplus), market fragmentation frequently improves execution time, trader welfare, and even total surplus—in stark contrast to the original conclusions. In contrast, the MarketSim logic (traders act on primary exchange BBO only and are more aggressive) leads to systematically lower surplus and increased opportunities for LA, creating much higher LA profits in line with the original claims.

---

Relational Equivalence and Qualitative Patterns

Even when not achieving quantitative mean alignment, the replication facilitates evaluation of relational (qualitative) results, such as the ordering of metric values across experimental treatments. The authors report the following:

• Fragmentation and Latency Arbitrage: The presence of LA is universally detrimental to surplus and exacerbates spreads in all implementations, confirming a relational property of the original model.
• Fragmentation Effects: When ZI strategy logic is more sophisticated (BestGuess), fragmentation reduces execution time and (often) increases welfare. With less strategic (MarketSim) logic, the benefit is muted or reversed.
• Sensitivity (see Table below): The model’s top-level takeaways hinge critically on the precise implementation of the greedy ZI behavior, a finding that generalizes the replication’s importance beyond this specific model.

Feature	Effect reported by WW	BestGuess replication
Latency arbitrage	Degrades efficiency	Reduces surplus vs. 2M no LA
LA on spread and time	Increases spread, variable execution times	Increases spread, decreases time
Market fragmentation	Mixed; can improve or degrade efficiency	Generally improves surplus
Fragmentation on time	Variable, with consolidation sometimes improving	Fragmentation reduces time

---

Practical and Theoretical Implications

Significance for Financial ABM

This replication underscores deep challenges for reproducibility in financial ABM. Small changes in unreported implementation details produce not only quantitative but qualitative reversals in market effects of fragmentation and arbitrage. Even the presence of open-sourced code did not guarantee reproducibility for all metrics or scenarios, exemplifying the necessity for detailed reporting and independent validation in computational market science.

Implications for Regulatory and Policy Modeling

The findings directly reinforce the SEC’s skepticism about unverified ABM results as policy input. The potential for opposite conclusions based on subtle code-level choices means that regulatory interventions based on such models could misfire unless multiple independent implementations reach robust convergence.

Future Research Directions

This replication demonstrates the need for:

• Rigorous protocolization and open data/code for ABMs;
• Classification and stress-testing of ABM outcomes against implementation divergences;
• Extension of these replicated baselines to more realistic agent ecologies, stylized fact adherence, and structural market features relevant to modern regulatory questions.

---

Key Figures

Figure 1: High-level logic for a simple CDA market—visualizing the core order-matching rules underlying agent interaction in all experiments.

Figure 2: Visualization of event scheduling, agent information flows, and high-level ABM structure as implemented in the replication.

---

Conclusion

This study demonstrates that independent replication of complex ABMs for market microstructure is both labor-intensive and inherently fraught, especially when implementation details are incompletely specified and codebases silently diverge. The sensitivity of findings on market fragmentation and arbitrage to ZI agent logic in Wah and Wellman’s model exposes substantial limitations to top-line takeaways in the ABM literature. Far from being a settled empirical result, the sign and strength of fragmentation effects appear highly contingent on agent-level strategic sophistication and information use. This places stringent demands on future work: robust, protocol-driven replication, careful sensitivity analysis, and finer granularity in the specification of agent trading logic are required before ABMs can be reliably used in regulatory or policy contexts.

Future advances should aim to systematically explore parameter and model-code dependence, achieve alignment in stylized facts across replications, and foster a culture of transparent, cumulative science within agent-based market modeling.