Evo-Sparrow: Evolutionary AI for Sparrow Mahjong

• Evo-Sparrow is a game-playing AI system that combines LSTM networks with CMA-ES to learn discard decisions in Sparrow Mahjong.
• It encodes game states into a compact 37-dimensional vector, enabling effective handling of partial observability and stochasticity.
• Empirical results show that Evo-Sparrow outperforms rule-based baselines and matches PPO performance under a fixed training budget.

Evo-Sparrow is a game-playing artificial intelligence system for Sparrow Mahjong in which a recurrent neural policy is trained by evolutionary optimization rather than by gradient-based reinforcement learning as the primary optimizer. The system combines a Long Short-Term Memory (LSTM) network with Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to learn discard decisions in a stochastic, partially observable, non-deterministic environment, and is reported to outperform random and rule-based baselines while remaining comparable to a Proximal Policy Optimization (PPO) baseline trained under the same total game budget (O'Connor et al., 11 Aug 2025).

1. Domain, objective, and problem formulation

Evo-Sparrow is designed for Sparrow Mahjong, which the paper presents as a three-player Mahjong variant retaining key properties of imperfect-information sequential decision-making. Each player starts with 5 tiles, draws a 6th tile on their turn, and then discards one tile. The winning condition is a hand composed of two melds—either sequences such as $3\text{-}4\text{-}5$ of the same suit or triplets such as three 7s—and one pair (O'Connor et al., 11 Aug 2025).

The system is intended to learn a strong discard policy under partial observability and stochasticity. A player can observe its own hand, public discards, and Dora information, but not opponents’ concealed hands. State transitions are affected by random tile draws and by opponent actions, so the environment combines hidden information, long-horizon dependencies, and delayed consequences. The paper identifies four terminal outcomes: win, draw, lose, and deal-in. A deal-in occurs when a player discards a tile that directly completes an opponent’s winning hand; wins yield positive points depending on hand strength, deal-ins lose points, and draws and ordinary losses do not change score (O'Connor et al., 11 Aug 2025).

Within this formulation, Evo-Sparrow addresses two coupled tasks. The first is state evaluation under incomplete information: the agent must extract actionable structure from private hand composition, public discard history, and Dora context. The second is policy optimization under noisy returns: the system must choose discards that increase long-run score and win rate while limiting deal-ins. The paper positions evolutionary search as a computationally effective alternative to policy-gradient training in this setting, particularly because the optimization target is sparse, stochastic, and not easily decomposed into shaped intermediate rewards (O'Connor et al., 11 Aug 2025).

2. State representation and policy network

The input to Evo-Sparrow is a 37-dimensional vector encoding the observable game state. Its components are 6 integers for the player’s hand, 30 integers for all players’ discarded tiles padded with 0s as needed, and 1 integer for the Dora indicator, giving

$6 + 30 + 1 = 37.$

Tiles are represented by face values following the PGX game-simulation library; ordinary tiles use their face values, while negative values indicate Dora tiles. The paper emphasizes that this compact encoding distinguishes tile identity and Dora status without the dimensional expansion associated with one-hot encodings (O'Connor et al., 11 Aug 2025).

The policy network consists of 3 LSTM layers followed by 8 fully connected layers with 32 neurons in the fully connected layers. The resulting model contains about $34{,}000$ weights. The authors describe this architecture as a compromise between sequential modeling capacity and the practical cost of full-covariance CMA-ES, whose memory and computation scale quadratically in parameter count (O'Connor et al., 11 Aug 2025).

The recurrent architecture is motivated by the temporal structure of Mahjong play. Discard decisions depend on prior discards, evolving hand development, and inferred opponent intentions. The paper argues that LSTMs are appropriate because they maintain a hidden state across turns and are better suited than vanilla recurrent networks to long-term dependencies. Batch normalization and dropout regularization are also mentioned as mechanisms to improve generalization and reduce overfitting, although their exact placement and rates are not specified (O'Connor et al., 11 Aug 2025).

The paper describes the network as producing discard decisions, but does not provide a formal action-selection equation for Evo-Sparrow, nor does it specify whether outputs correspond to tile identities or hand positions. Functionally, the network maps the encoded sequential game state to a discard action at each decision point (O'Connor et al., 11 Aug 2025).

3. Evolutionary optimization with CMA-ES

The distinctive feature of Evo-Sparrow is that the LSTM policy is optimized with CMA-ES rather than trained principally by backpropagated policy gradients. In the paper’s description, the network parameters are treated as a vector $\theta \in \mathbb{R}^d$, with $d \approx 34{,}000$, and candidate solutions are sampled from a multivariate normal search distribution,

$\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$

where $m$ is the current mean parameter vector, $\sigma$ is the global step size, and $C$ is the covariance matrix (O'Connor et al., 11 Aug 2025). In the broader evolution-strategy literature, CMA-ES belongs to the estimation-of-distribution family, in which candidate vectors are drawn from a Gaussian search distribution over continuous parameter space (Lange, 2022).

The initial global step size is

$\sigma = 1.$

Training is run for 50 generations with 35 candidate solutions per generation, so the CMA-ES population size is $6 + 30 + 1 = 37.$0. Each candidate network is evaluated by playing 200 games against two rule-based expert agents, with player order randomized each game. Fitness is defined as cumulative score across those 200 games, which can be written as

$6 + 30 + 1 = 37.$1

where $6 + 30 + 1 = 37.$2 is the score obtained by the candidate in game $6 + 30 + 1 = 37.$3 (O'Connor et al., 11 Aug 2025).

After evaluating all 35 candidates in a generation, their fitnesses are used to update the CMA-ES mean vector, covariance matrix, and step size. The paper specifically mentions cumulative step-size adaptation (CSA) for controlling $6 + 30 + 1 = 37.$4. It does not print the full CMA-ES rank-$6 + 30 + 1 = 37.$5, evolution-path, or covariance-update equations, but the optimization loop is described as repeated sampling, evaluation, ranking, and distribution update (O'Connor et al., 11 Aug 2025).

This setup yields a total training budget of

$6 + 30 + 1 = 37.$6

game episodes. Model selection is also specific: after 50 generations, the chosen policy is the best solution among the final set of candidate solutions, rather than the best checkpoint aggregated across all generations (O'Connor et al., 11 Aug 2025).

Component	Reported specification
Input representation	37-dimensional vector
Policy architecture	3 LSTM layers + 8 fully connected layers
Parameter count	About $6 + 30 + 1 = 37.$7 weights
CMA-ES step size	$6 + 30 + 1 = 37.$8
Generations	50
Candidates per generation	35
Games per candidate	200
Total training games	350,000

4. Training environment, baselines, and evaluation protocol

The simulations are implemented in PGX, which the paper describes as a hardware-accelerated parallel game simulator. During CMA-ES training, Evo-Sparrow is evaluated only against fixed rule-based expert opponents; the optimization process is therefore not self-play. This design makes the fitness signal a structured estimate of performance against a heuristic policy class rather than performance in an evolving population of self-generated opponents (O'Connor et al., 11 Aug 2025).

The rule-based expert agent is itself nontrivial. It prioritizes high-value tiles, avoids breaking existing sets or pairs, favors tiles with higher set-forming potential, disfavors terminal tiles such as 1 and 9, and breaks ties randomly among top discard candidates. The paper uses this agent both as an evaluation baseline and as the training opponent for Evo-Sparrow (O'Connor et al., 11 Aug 2025).

For optimizer-level comparison, the paper introduces PPO-Sparrow. This baseline uses the same LSTM backbone as Evo-Sparrow, controlling for network architecture while changing the optimization method. PPO-Sparrow employs a policy head with softmax over actions and a value head estimating expected cumulative reward. Its reported hyperparameters are 200 parallel games per training iteration, 4 optimization epochs per collected batch, PPO clipping on policy and value updates with

$6 + 30 + 1 = 37.$9

learning rate

$34{,}000$0

discount factor

$34{,}000$1

plus entropy regularization, generalized advantage estimation, normalized advantages, and gradient clipping. PPO is trained for 1,750 updates with 200 games per update, again totaling 350,000 games, which matches Evo-Sparrow’s total training budget exactly (O'Connor et al., 11 Aug 2025).

Final evaluation uses 1,000,000 games per configuration in order to reduce variance and improve statistical significance in a stochastic environment. Reported metrics are average score, win percentage, draw percentage, loss percentage, and deal-in percentage. For Evo-Sparrow versus PPO-Sparrow, the paper additionally reports a $34{,}000$2-test on score difference with

$34{,}000$3

and a chi-square test on win rates with

$34{,}000$4

The paper does not report confidence intervals or effect sizes (O'Connor et al., 11 Aug 2025).

5. Empirical performance and convergence behavior

The baseline experiments establish the strength ordering of the non-learning agents. When all three players are random, scores remain near zero, win rates are about $34{,}000$5–$34{,}000$6, draw rate is $34{,}000$7, and deal-in rate is about $34{,}000$8–$34{,}000$9. When one rule-based agent plays against two random agents, the rule-based agent records average score $\theta \in \mathbb{R}^d$0, win rate $\theta \in \mathbb{R}^d$1, draw rate $\theta \in \mathbb{R}^d$2, loss rate $\theta \in \mathbb{R}^d$3, and deal-in rate $\theta \in \mathbb{R}^d$4, while the random agents fall to average scores around $\theta \in \mathbb{R}^d$5, win rates $\theta \in \mathbb{R}^d$6, and deal-in rates around $\theta \in \mathbb{R}^d$7 (O'Connor et al., 11 Aug 2025).

Against mixed and heuristic opposition, Evo-Sparrow is substantially stronger. In a configuration with Evo-Sparrow, one rule-based agent, and one random agent, Evo-Sparrow achieves average score $\theta \in \mathbb{R}^d$8, win rate $\theta \in \mathbb{R}^d$9, draw rate $d \approx 34{,}000$0, loss rate $d \approx 34{,}000$1, and deal-in rate $d \approx 34{,}000$2. In the configuration with Evo-Sparrow against two rule-based agents, it records average score $d \approx 34{,}000$3, win rate $d \approx 34{,}000$4, draw rate $d \approx 34{,}000$5, loss rate $d \approx 34{,}000$6, and deal-in rate $d \approx 34{,}000$7, while both rule-based opponents have negative average scores and lower win rates (O'Connor et al., 11 Aug 2025).

Self-play results are near-symmetric. With three identical Evo-Sparrow agents, scores are all near zero, win rates lie between $d \approx 34{,}000$8 and $d \approx 34{,}000$9, and deal-in rates between $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$0 and $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$1. The paper interprets this as evidence that the learned policy is not merely exploiting peculiarities of the rule-based training opponent (O'Connor et al., 11 Aug 2025).

The most important direct comparison is the three-agent setting with Evo-Sparrow, PPO-Sparrow, and one rule-based agent. Over 1,000,000 games, Evo-Sparrow reports average score $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$2, win rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$3, draw rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$4, loss rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$5, and deal-in rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$6. PPO-Sparrow is close, with average score $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$7, win rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$8, draw rate $\theta^{(k)} \sim \mathcal{N}(m, \sigma^2 C),$9, loss rate $m$0, and deal-in rate $m$1. The rule-based agent is clearly weaker, with average score $m$2, win rate $m$3, and deal-in rate $m$4. The score difference between Evo-Sparrow and PPO-Sparrow is not statistically significant under the reported $m$5-test, but the win-rate difference is statistically significant under the reported chi-square test (O'Connor et al., 11 Aug 2025).

Setting	Agent	Key reported outcomes
Evo-Sparrow + Rule-Based + Random	Evo-Sparrow	Avg. score 0.8687; Win 28.55%; Deal-in 16.31%
Evo-Sparrow + Rule-Based + Rule-Based	Evo-Sparrow	Avg. score 0.2648; Win 26.39%; Deal-in 22.28%
Evo-Sparrow + PPO-Sparrow + Rule-Based	Evo-Sparrow	Avg. score 0.1934; Win 22.80%; Deal-in 23.52%

The paper also reports unusually extensive convergence evidence for an evolutionary game-learning study. Across 100 independent training runs, median fitness starts around $m$6 at generation 1, rises rapidly, becomes stably positive by about generation 20, continues improving more slowly thereafter, and stabilizes around generation 40. The interquartile range narrows during training, which the authors interpret as evidence of decreasing variability and reasonably consistent convergence (O'Connor et al., 11 Aug 2025).

Wall-clock cost is compared over 10 independent runs. Average training time is about 40 minutes 19 seconds for Evo-Sparrow and about 106 minutes 43 seconds for PPO-Sparrow. The paper attributes this advantage mainly to the ease of parallelizing independent candidate evaluations under CMA-ES (O'Connor et al., 11 Aug 2025).

6. Interpretation, limitations, and relation to other “sparrow” research

Evo-Sparrow is significant chiefly as a demonstration that a recurrent deep policy for a stochastic imperfect-information game can be trained competitively by black-box evolutionary search. The paper’s broader implication is that evolutionary optimization can be practical in domains characterized by hidden information, sparse and delayed outcomes, and noisy returns, where policy-gradient methods may require more algorithmic machinery for credit assignment and stabilization (O'Connor et al., 11 Aug 2025).

At the same time, the system has clear limits. The paper notes that CMA-ES remains computationally expensive because fitness must be estimated by repeated simulation, and that full-covariance CMA-ES becomes memory-intensive as network size grows. This is one reason the authors constrain the network to about $m$7 weights. The evaluation is also narrow by contemporary game-AI standards: because Sparrow Mahjong lacks established public benchmarks, the reported comparisons are restricted to random, rule-based, and author-implemented PPO baselines. There are no formal ablations separating the effects of recurrent memory, encoding choices, population size, opponent selection, or CMA-ES hyperparameters. Training is performed only against fixed rule-based opponents, which raises an obvious question about opponent-specific adaptation, even though the self-play results partly mitigate that concern (O'Connor et al., 11 Aug 2025).

A recurrent source of confusion is nomenclature. Despite its name, Evo-Sparrow is not a Sparrow Search Algorithm (SSA) method. It is a Mahjong agent whose neural weights are optimized with CMA-ES (O'Connor et al., 11 Aug 2025). This distinguishes it from SSA-based optimization papers such as the iterative-local-search-enhanced random-forest tuning study for VR user experience prediction (Tang et al., 2024), the GeoSSA continuous-optimization variant (Wei et al., 27 Jan 2026), and the EA4eigCS ensemble in which Sparrow Search is used only as a secondary operator for inferior individuals (Du et al., 15 Jan 2026). It is also separate from the clustered Sparrow SAT local-search algorithm (Dumitrescu, 2018) and from the video-grounding MLLM called SPARROW (Alansari et al., 12 Mar 2026). The shared term “sparrow” therefore denotes several unrelated research lineages; Evo-Sparrow belongs specifically to evolutionary game AI for Sparrow Mahjong (O'Connor et al., 11 Aug 2025).

In summary, Evo-Sparrow is best understood as a hybrid deep-learning and evolutionary-optimization system: a 3-layer LSTM plus 8-layer multilayer perceptron trained by full-covariance CMA-ES on 350,000 simulated games, evaluated at scale over 1,000,000-game tournaments, and shown to surpass heuristic baselines while matching or slightly exceeding a PPO baseline under a matched training budget (O'Connor et al., 11 Aug 2025). Its main technical contribution is not a new evolution-strategy formulation, but a concrete demonstration that direct evolutionary optimization of sequential neural policies can be viable for partially observable stochastic games.