LLMsPark: LLM Gaming Benchmark & Urban Analytics

• LLMsPark is a dual-use term describing both a game theory-based evaluation benchmark for LLMs and a multimodal urban monitoring framework.
• The benchmarking system employs canonical games and Elo score aggregation to assess LLMs' strategic decision-making and social behaviors.
• The urban monitoring version leverages data alignment and domain-specific tools to provide traceable, accurate analysis of park development.

LLMsPark is a name used for two distinct large-language-model systems in the arXiv literature. In one usage, it denotes a game theory-based evaluation platform for measuring LLM decision-making strategies and social behaviors in classic game-theoretic settings (Chen et al., 20 Sep 2025). In another, it denotes a multi-modal agent framework built on GPT-4o for intelligent monitoring of urban park development through data alignment, domain-specific tooling, and LLM-based fusion and reasoning (Xiao et al., 28 Jan 2026). The shared name therefore refers not to a single research line but to two separate technical artifacts: one oriented toward strategic-agent evaluation, the other toward multi-modal urban analysis.

1. Terminological scope and dual usage

The strategic-gaming LLMsPark was introduced as “A Benchmark for Evaluating LLMs in Strategic Gaming Contexts” and was published on 2025-09-20 (Chen et al., 20 Sep 2025). Its stated purpose is to evaluate LLM intelligence beyond single metrics by examining interactive dynamics and strategic behaviors in classic game-theoretic settings. The system cross-evaluates 15 leading LLMs, including both commercial and open-source models, and exposes benchmark rankings through a public leaderboard (Chen et al., 20 Sep 2025).

The urban-monitoring LLMsPark appeared later in “Towards Intelligent Urban Park Development Monitoring: LLM Agents for Multi-Modal Information Fusion and Analysis,” published on 2026-01-28 (Xiao et al., 28 Jan 2026). In that usage, LLMsPark is a multi-modal agent framework for monitoring newly constructed parks, designed to address limitations of traditional change-detection methods based on remote sensing imagery. Its core architecture consists of three tightly coupled subsystems: Data Alignment, Domain-Specific Tooling, and LLM-Based Fusion & Reasoning (Xiao et al., 28 Jan 2026).

Because the two systems target different problem domains, the term is context-sensitive. In strategic-AI evaluation, LLMsPark refers to a benchmark and leaderboard infrastructure; in geospatial analytics, it refers to an end-to-end agent pipeline for park-development monitoring. This suggests that citations are essential for disambiguation in technical writing.

2. Strategic-gaming LLMsPark: benchmark design and game environments

The benchmark-oriented LLMsPark defines a multi-agent environment grounded in five canonical games: Prisoner’s Dilemma, Trust Game, Nim Game, Dictator Game, and Who Is Spy (Chen et al., 20 Sep 2025). These scenarios are used to probe decision-making strategies, social behaviors, and strategic depth under interactive conditions.

In Prisoner’s Dilemma, there are two players and two actions, Cooperate $(C)$ or Betray/Defect $(D)$. The paper describes the payoff structure qualitatively: both cooperate yields a moderate reward to each; unilateral defection gives the defector the highest payoff and the cooperator the lowest; mutual defection gives both a small penalty. Figure 1 illustrates the $2 \times 2$ game, but no numeric entries are specified. The paper notes only the standard inequality pattern $T>R>P>S$, without instantiating values (Chen et al., 20 Sep 2025).

In the Trust Game, two players repeatedly choose either “Cooperate,” which costs one coin, or “Cheat,” which costs nothing. If both cooperate, each doubles the investment and receives a net gain of $+1$ coin; if one cooperates and the other cheats, the cooperator loses $1$ coin and the cheater gains $2$ coins; if both cheat, both receive zero net gain. The paper gives this verbal specification but no printed numeric matrix (Chen et al., 20 Sep 2025).

The Nim Game is formulated as an impartial combinatorial game in which two players alternate removing at least one stone from a single pile or multiple piles, and the player who takes the last stone wins. The paper explicitly invokes the Nim-sum criterion: if the bitwise XOR of pile sizes is nonzero, the first mover has a forced win; otherwise the second mover does. No payoff matrix is needed, because the outcome is win $=1$ point and loss $=0$ (Chen et al., 20 Sep 2025).

The Dictator Game assigns one player the role of dictator and the other the role of receiver. The dictator receives a resource endowment and unilaterally decides how much to keep and how much to give. The dictator’s payoff equals the coins kept, and the receiver’s payoff equals the coins received. No further formalization is provided (Chen et al., 20 Sep 2025).

Who Is Spy is the only explicitly multi-player game in the benchmark, with the paper studying $N=6$. One player is randomly assigned the Spy role and the remaining players are civilians. Civilians share a secret vocabulary, while the Spy has a different vocabulary. The game proceeds through rounds of natural-language description and deduction, after which one player is executed by group vote in each round. Civilians win by correctly identifying the Spy, and the Spy wins by surviving or blending in. Again, only win/loss outcomes are defined; no numeric payoff matrix is provided (Chen et al., 20 Sep 2025).

3. Strategic-gaming LLMsPark: scoring, Elo aggregation, and evaluation protocol

The benchmark uses two layers of scoring. For games with numeric payoffs—Prisoner’s Dilemma, Trust, and Dictator—an agent’s score is the sum of coins or utility units earned over all rounds. The paper does not define a normalized per-game payoff formula or a cooperation-rate formula in LaTeX (Chen et al., 20 Sep 2025).

To aggregate results across heterogeneous games, LLMsPark maintains an Elo rating for each model. If two players have current ratings $(D)$0 and $(D)$1, their expected scores are

$(D)$2

After a match, if Model A’s actual score is $(D)$3, with win $(D)$4, draw $(D)$5, and loss $(D)$6, the ratings are updated as

$(D)$7

with $(D)$8 and initial rating $(D)$9; ties use $2 \times 2$0 (Chen et al., 20 Sep 2025).

The leaderboard methodology feeds every pair or group match outcome into this Elo system. Raw in-game payoffs are not directly ranked, and no further normalization across game types is described. Each head-to-head result carries equal weight because every match uses the same $2 \times 2$1-factor. Ties are handled directly through the Elo update rule (Chen et al., 20 Sep 2025).

The multi-agent setup is also explicitly specified. When a user registers an LLM as a “Player Agent,” the system pairs it with other LLMs in the pool to satisfy the required number of players per game: two for Prisoner’s Dilemma, Trust, Nim, and Dictator, and six for Spy. Pairing is described as effectively random, via round-robin or simultaneous queuing. Players see only public prompts and admissible information such as past decisions, remaining stones, or vote tallies. There is no side-channel communication or shared hidden state beyond what the rules permit. All agents are pre-loaded with the same description of the rules and initial conditions, and may reference their own memory of past rounds via the “Perception” and “Brain” modules, but do not share private observations (Chen et al., 20 Sep 2025).

4. Strategic-gaming LLMsPark: empirical findings, emergent behaviors, and limitations

The benchmark reports differentiated model performance across games. GPT-4 achieves the highest multi-round Elo ratings in Prisoner’s Dilemma and Trust Game, which the paper interprets as strong long-term strategic adaptation. Qwen-14B-Chat outperforms all others in Who Is Spy, suggesting stronger text-retrieval, inference, and deception or camouflage capabilities in that scenario. Phoenix-inst-chat-7b tops single-round Prisoner’s Dilemma, indicating an aggressive risk-seeking strategy in one-shot interactions. ChatGLM2-6B scores highest in the Dictator Game, which the paper associates with a notably selfish allocation bias (Chen et al., 20 Sep 2025).

The authors also report five emergent social behaviors in multi-player games that were not hard-coded into prompts: Trust, defined as conditional cooperation on evidence; Confrontation, defined as public accusations; Pretense, defined as false self-presentation, especially by the Spy; Leadership, defined as steering group decisions; and Deception, defined as fabricated events or claims used to mislead others (Chen et al., 20 Sep 2025). A plausible implication is that the benchmark is intended not only to capture static outcomes such as wins and losses, but also to surface interpretable behavioral regularities in dialogue-intensive strategic settings.

The paper further emphasizes discrepancies between single-round and multi-round performance. GPT-4 and GPT-3.5-turbo are described as over-cooperative in single-round Trust Games, choosing “cooperate” by default, but learning to sustain cooperation in iterated settings. Phoenix-inst-chat-7b, by contrast, shifts from high defection in one-shot Prisoner’s Dilemma to weaker performance in repeated Prisoner’s Dilemma, which the authors read as evidence of weak learning from history (Chen et al., 20 Sep 2025).

On statistical reporting, the paper states that score gaps greater than 100 Elo points separate top and bottom performers in each game, but it does not provide p-values or confidence intervals. The absence of formal statistical testing limits strict significance claims. Additional limitations include the lack of explicit numeric payoff matrices or normalized utility formulas for each game, as well as response latency and computational overhead in fast-paced games (Chen et al., 20 Sep 2025).

The authors identify several extensions: adding more intricate games such as Stag Hunt and Hawk–Dove; integrating human players to measure human–LLM strategic parity and enrich training data for theory-of-mind behaviors; and establishing standardized baselines and evaluation metrics such as cooperation rates and equilibrium-deviation measures (Chen et al., 20 Sep 2025). This suggests that the benchmark is positioned as an initial platform rather than a closed evaluation standard.

5. Urban-monitoring LLMsPark: architecture and alignment mechanisms

In the urban-analytics usage, LLMsPark is a multi-modal agent framework built on GPT-4o for intelligent monitoring of urban park development (Xiao et al., 28 Jan 2026). The framework is organized into three major modules. Data Ingestion & Alignment collects heterogeneous inputs, including remote-sensing imagery, LiDAR point clouds, ground-level photos, tabular or GIS attribute data, and textual metadata. The Domain-Specific Toolkit provides deterministic preprocessing and analysis tools exposed through a strict function-calling API with JSON schemas and in-context examples. LLM Fusion & Reasoning decomposes natural-language queries into subtasks, invokes appropriate tool calls, ingests structured outputs, and produces a unified human-readable analysis report (Xiao et al., 28 Jan 2026).

A key innovation claimed by the paper is the combination of horizontal and vertical alignment to maintain spatial, temporal, and semantic consistency. For horizontal alignment, the modality set is defined as $2 \times 2$2. For each modality $2 \times 2$3, raw data points take the form

$2 \times 2$4

where $2 \times 2$5 is the feature, $2 \times 2$6 the geo-coordinate, and $2 \times 2$7 the timestamp. A transform $2 \times 2$8 maps coordinates into a canonical coordinate reference system:

$2 \times 2$9

where $T>R>P>S$0 is a coordinate-conversion function such as an EPSG transformation. When resolutions differ, the framework resamples to a common grid, for example $T>R>P>S$1 cells, using nearest-neighbor or bilinear interpolation (Xiao et al., 28 Jan 2026).

The corresponding pseudocode is given as follows:

$+1$2

Vertical alignment is introduced to track lineage through processing steps and avoid “silent” data drift. Each raw data element receives a global unique identifier $T>R>P>S$2. At processing step $T>R>P>S$3, the framework appends to a log

$T>R>P>S$4

Given an initial element

$T>R>P>S$5

after step $T>R>P>S$6 the processed element becomes

$T>R>P>S$7

and the system stores $T>R>P>S$8 (Xiao et al., 28 Jan 2026).

The corresponding pseudocode is

$+1$3

The paper states that this global-ID-plus-logging schema ensures full data traceability and semantic alignment between user queries and actual data (Xiao et al., 28 Jan 2026).

6. Urban-monitoring LLMsPark: tooling, fusion pipeline, evaluation, and applications

The framework’s domain-specific toolkit is designed to mitigate hallucination by constraining the LLM to deterministic external tools. The paper lists several key components: a CSV Column Selector with signature select_columns(csv_name: str, columns: List[str]) → DataFrame; a Shapefile Processor with load_shapefile(shp_path: str) → GeoDataFrame; LiDAR Data Connector and Converter functions read_lidar(las_path: str) → PointCloud and convert_to_dem(pointcloud: PointCloud) → RasterImage; a Visual Results Generator with raster_overlay(raster1, raster2) → PNG and annotate_polygons(GeoDataFrame) → SVG/PNG; and a Correlation Analyzer with spatial_join(gdf1, gdf2, tolerance: float) → DataFrame and temporal_correlate(records, time_window: Δt) → DataFrame (Xiao et al., 28 Jan 2026). The LLM is instructed that numerical or spatial results can be obtained only through these tools, and the tool interfaces are paired with few-shot examples and JSON schemas.

The fusion pipeline is explicitly decision-level. Remote sensing is preprocessed into spectral indices and land-use or land-cover classification outputs, such as GeoJSON polygons with class labels. LiDAR yields Digital Elevation Model and Canopy Height Model raster layers. Ground-level photos produce geo-tagged object detections, including examples such as fountains, benches, and restrooms. Tabular and GIS metadata are represented as DataFrames of park attributes. Temporal tracking then associates each output polygon, raster cell, or detected object with $T>R>P>S$9, so that change queries can be answered through difference calculations such as $+1$0 (Xiao et al., 28 Jan 2026).

The geo-matching function that links features across modalities is defined as

$+1$1

The paper states that, combined with global-UUID lineage, this prevents double-counting and ensures consistent object tracking (Xiao et al., 28 Jan 2026).

Evaluation is based on a benchmark of 10 questions at three analysis levels: Basic Retrieval with 4 questions, Qualitative Reasoning with 4 questions, and Quantitative Change Analysis with 2 questions (Xiao et al., 28 Jan 2026). The compared systems are LangChain default agents for SQL, Pandas, and CSV; vanilla GPT-4o without tools; a single-modality CSV agent using only the CSV tools; and the full LLMsPark system (Xiao et al., 28 Jan 2026).

System	Correct answers	Accuracy
LangChain agents	0/10	0%
Vanilla GPT-4o	4/10	40%
Single-modality CSV agent	6/10	60%
LLMsPark agent	10/10	100%

The paper also reports an ablation: removing horizontal alignment caused a 20% drop because some parks were mismatched, while removing the toolkit caused GPT hallucination on 30% of answers (Xiao et al., 28 Jan 2026). Although the study does not compute IoU or F1 for semantic segmentation in the main evaluation, it reports internal tests on the land-use or land-cover classification step with mean IoU of 0.78 across four land-cover classes and an accuracy of 0.85 (Xiao et al., 28 Jan 2026).

Concrete application scenarios include urban planning queries such as identifying parks opened after 2016 that saw more than 10% increase in tree canopy by 2023 and locating new restrooms; resource-allocation queries such as estimating new pathway length added in Queens parks between 2018 and 2022; and environmental-impact queries such as producing a time series of grass cover versus impervious surface for Inwood Hill Park from 2010 to 2020 (Xiao et al., 28 Jan 2026). A plausible implication is that the framework is intended as a generalizable agent pattern for spatially grounded, multi-modal analytical tasks, with the paper explicitly suggesting extension to street-tree inventories and construction-site tracking.

7. Comparative significance and interpretive cautions

The two LLMsPark systems occupy different methodological niches. The strategic-gaming version treats LLMs as player agents inside controlled multi-agent environments and uses game-theoretic interactions, in-game payoff accumulation, and Elo updates to evaluate strategic behavior (Chen et al., 20 Sep 2025). The urban-monitoring version treats an LLM as a coordinating agent over aligned multi-modal data and deterministic tools, emphasizing traceability, hallucination mitigation, and decision-level fusion for applied geospatial analysis (Xiao et al., 28 Jan 2026).

Their commonality lies in the fact that both move beyond single-shot prompting. The benchmark paper argues that LLM evaluation should include interactive dynamics and strategic behaviors rather than relying only on single metrics (Chen et al., 20 Sep 2025). The urban-monitoring paper similarly argues that traditional methods are insufficient for current urban planning and management demands, motivating a workflow in which the LLM orchestrates multi-step reasoning over heterogeneous data sources (Xiao et al., 28 Jan 2026). This suggests a broader research pattern in which LLM capability is assessed or deployed through structured interaction, either among agents or among tools and modalities.

Several interpretive cautions follow directly from the papers. In the benchmark setting, the lack of explicit payoff matrices, normalized utility formulas, and formal statistical tests limits theoretical precision and inferential certainty (Chen et al., 20 Sep 2025). In the urban-monitoring setting, the reported benchmark comprises 10 questions and focuses on high-level change queries rather than comprehensive vision metrics, so the results should be read within that stated evaluation scope (Xiao et al., 28 Jan 2026). The shared name can itself create bibliographic ambiguity, and precise citation is therefore necessary whenever LLMsPark is discussed in academic contexts.