Kalshi-based Prediction Market Episodes

• Kalshi-based episodes are a suite of trading environment reconstructions derived from real Kalshi market data, offering deterministic, event-driven simulations.
• They integrate orderbook, trade, and lifecycle data using precise timestamp ordering to emulate realistic market microstructure and fee structures.
• This platform enables researchers to rigorously backtest trading strategies and assess performance under transaction costs, settlement constraints, and volatile conditions.

Kalshi-based episodes are a standardized suite of trading environment reconstructions derived from the Kalshi CFTC-regulated U.S. prediction market, as implemented in the PredictionMarketBench framework. These episodes offer a deterministic, event-driven platform for evaluating algorithmic and LLM-based trading agents using replayed historical limit-order-book and trade data. Each episode encapsulates a distinct prediction market context—spanning cryptocurrency, weather, and sports—for the systematic backtesting of agent behaviors under realistic market microstructure, transaction cost, and settlement constraints (Arora et al., 28 Jan 2026).

1. Overview of Kalshi-based Episodes

The four Kalshi-based episodes are constructed directly from raw Kalshi market data streams, with sampling concentrated in January 2026. Each episode is characterized by its prediction domain, ticker structure, and event window:

Episode ID	Domain	Tickers	Duration	OB snaps	Trades
KXBTCD-26JAN2017	Crypto	23	37.4 h	311,998	6,283
KXHIGHNY-26JAN20	Weather	6	37.4 h	50,231	8,044
KXNCAAF-26	Sports (CFB)	2	37.4 h	8,320	171,786
KXNFLGAME-26JAN11BUFJAC	Sports (NFL)	2	67.4 h	8,047	111,160

• KXBTCD-26JAN2017: Bitcoin daily high threshold prediction with 23 YES/NO contracts (“Did BTC close above $$X?”) in a ≈37.4 hour window.</li> <li><strong>KXHIGHNY-26JAN20</strong>: NYC weather episode with 6 discrete high-temperature thresholds (“Will NYC high exceed T°?”), spanning the same interval.</li> <li><strong>KXNCAAF-26</strong>: College Football season-long futures with 2 championship outcome tickers; same time span, but extremely high trade count.</li> <li><strong>KXNFLGAME-26JAN11BUFJAC</strong>: NFL single-game spread bet (Buffalo vs. Jacksonville) over ≈67.4 hours.</li> </ul> <p>Each episode encompasses raw orderbook updates, trade prints, and settlement streams for its associated tickers. All content is pre-sharded by event identifier and organized under an episode directory structure (metadata.json, orderbook.parquet, trades.parquet, settlement.json).</p> <h2 class='paper-heading' id='data-extraction-and-state-construction'>2. Data Extraction and State Construction</h2> <p>Episodes are built from three market data streams—orderbook updates, trades, and lifecycle events—aligned to a global UTC timestamp and further disambiguated by sequence numbers. The episode’s state at each agent decision time$$t$$(with agent <a href="https://www.emergentmind.com/topics/cadence" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">cadence</a>$$\Delta t$, e.g., 5 minutes) comprises for each ticker$i$:</p> <ul> <li>$\mathrm{best\_bid}_i(t)$,$\mathrm{best\_ask}_i(t)$</li> <li>Mid-price:$m_i(t) = \frac{\mathrm{best\_bid}_i(t) + \mathrm{best\_ask}_i(t)}{2}$</li> <li>Top-$N$$orderbook level depths: volumes at adjacent ticks around <a href="https://www.emergentmind.com/topics/bombardier-beetle-optimizer-bbo" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">BBO</a></li> <li>History vector: last$$M$mid-price values$[m_i(t-\Delta t\cdot j)]_{j=1}^M$</li> <li>Agent’s open positions$\mathrm{pos}_i(t)$, cash balance$\Delta t$0
• List of active, unfilled orders (ID, side, price, size, time-in-force)

Feature engineering supports derived metrics such as:

• Simple moving average (SMA) of mid-price over window $\Delta t$1:

$\Delta t$2

• Rolling standard deviation $\Delta t$$3 over the same window.</li> </ul> <p>Raw data is presented as Parquet and JSON files. Timestamps and sequence IDs ensure unambiguous global event ordering.</p> <h2 class='paper-heading' id='agent-actions-execution-and-reward-structure'>3. Agent Actions, Execution, and Reward Structure</h2> <p>The action interface (via AgentContext API) exposes:</p> <ul> <li><code>submit_limit_order(ticker, side∈{BUY,SELL}, price, size, tif∈{IOC, GTC, POST_ONLY})</code></li> <li><code>submit_market_order(ticker, side, size)</code></li> <li><code>cancel_order(order_id)</code></li> </ul> <p>Execution uses maker/taker fee modeling:</p> <ul> <li>Taker fee$$\Delta t$4 (cents): $\Delta t$5
• Maker fee $\Delta t$6: $\Delta t$7
• Transaction cost for size $\Delta t$8 at price $\Delta t$9:

$i$0

with $i$1

Reward at each timestep is:

$i$2

At terminal settlement, all open positions are settled at outcome $i$3 with:

$i$4

Total episodic reward is $i$5 and incorporates both market-to-market P&L, transaction costs, and settlement corrections.

4. Deterministic Replay and Simulation Pipeline

The environment employs a deterministic, event-driven simulator to ensure reproducibility and fair comparison across trading agents. The canonical replay pipeline executes as follows:

$i$7

All episodic data is strictly partitioned by event identifier. Sequence numbers resolve any tie in event timestamps. This design enables precise event ordering, strict replay determinism, and supports both classical and tool-calling LLM agents with reproducible trajectories.

5. Usage, API Access, and Key Statistics

Researchers interact with Kalshi-based episodes programmatically via the PredictionMarketBench Python API. Core workflow:

$i$8

Observations are Python dicts mapping tickers to current quotes, depth arrays, positions, and cash. Actions are lists of dicts specifying ticker, side, order type, price, size, and TIF. The simulator outputs deterministic logs, timestamped fills, transactional fees, and detailed P&L records for reproduction and offline analysis.

Key statistics for each episode—duration, orderbook snapshots, trade volume, and ticker count—are summarized above. Decision steps per episode are proportional to duration and agent cadence (e.g., 37.4h at 5min → ≈448 steps). Aggregate volatility can be computed via $i$6 as an offline metric.

6. Research Implications and Observed Dynamics

The standardized Kalshi-based episodes offer a unique backtesting corpus with fee and settlement mechanisms characteristic of real prediction markets. Baseline analyses demonstrate that naive trading agents can underperform due to cumulative transaction costs and adverse settlement effects, while algorithmic, fee-aware agents display robustness in volatile regimes (Arora et al., 28 Jan 2026). This property highlights the critical influence of microstructure and execution modeling in algorithmic market design and validation. A plausible implication is that agents relying solely on directional signal without transaction cost modeling will systematically underperform relative to microstructure-sensitive strategies.

The tool supports studies into agent adaptivity, liquidity provision, settlement risk management, and the development of testable, reproducible results across artificial and learned agent classes. The strict replay determinism and event-partitioned design of Kalshi-based episodes represent a methodological advance aligning with best practices in empirical market microstructure and reinforcement learning benchmark design.