Puzzle: Formal Models & Computational Benchmarks

• Puzzle is a class of structured problems defined by finite state spaces, admissible moves, and verifiable solution criteria.
• Research spans combinatorial graph puzzles, spatial arrangements, and NP-/ASP-hard models using algebraic and convex methods.
• Benchmarks employ puzzles to assess algorithmic reasoning in reinforcement learning and foundation models with clear performance metrics.

Searching arXiv for recent and foundational puzzle-related papers to ground the article. Puzzle denotes a broad class of structured problems in which a solver must discern patterns, apply deduction, and combine available information in order to arrive at a correct solution. Within the research literature represented here, the term encompasses combinatorial graph games, colored-cube and edge-matching constructions, sliding-block and path puzzles, logic-grid and natural-language inference datasets, and algorithmic reasoning benchmarks for reinforcement learning and foundation models. These treatments share an emphasis on explicit state spaces, constrained operations, and well-defined solution criteria, but they diverge in mathematical formalization, computational complexity, and research use. Some works study puzzle as a concrete mathematical object, such as Reeder’s puzzle on Dynkin diagrams (Evenor, 2015) or jigsaw percolation on paired graphs (Brummitt et al., 2012). Others treat puzzles as computational benchmarks for reasoning systems, including natural-language inference datasets (Szomiu et al., 2021), RL environments (Estermann et al., 2024), and foundation-model evaluations (Long et al., 7 Oct 2025). Still others analyze puzzle design itself through algebraic encodings, exhaustive enumeration, and complexity proofs (Kovalsky et al., 2014, Roa, 2016, Hamersma et al., 2020, Kiatchaipipat et al., 15 Aug 2025).

1. Formal notions of puzzle and solution

A recurring feature of the literature is that a puzzle instance is specified by a finite state space together with admissible transformations and a target condition. In Reeder’s puzzle, the state is a labeling $a=(a_1,\dots,a_n)\in (\mathbb{Z}/2\mathbb{Z})^n$ on a finite connected graph, and a move $T_i$ flips the label at vertex $i$ according to the parity of neighboring labels (Evenor, 2015). Two configurations are equivalent when they differ by a finite sequence of such moves, and the “solution” of the puzzle consists of determining the equivalence classes $\mathrm{Cl}(D)$, computing $\#\mathrm{Cl}(D)$, and choosing minimal representatives (Evenor, 2015). This formulation makes puzzle-solving a classification problem in discrete dynamical systems rather than a search for a single terminal state.

Other puzzle models take the form of exact cover, matching, or arrangement problems. In the mathematical model of Insanity-type cube puzzles, each color is represented by a distinct prime, opposite faces are encoded by products of two primes, and an $n$-cube puzzle becomes an $n\times 3$ matrix of such products (Roa, 2016). A valid solution is expressed through a “magic number” $M=\prod_{i=1}^n p_i^2$, with full solutions corresponding to two independent selections of opposite face-pairs whose products equal $M$ (Roa, 2016). In apictorial edge-matching puzzles, pieces are polygonal tiles whose edges have colors and orientations; a valid solution is a placement in which matched edges coincide in position, have equal colors, and opposite normals (Kovalsky et al., 2014).

Several recent benchmark papers explicitly define puzzles as reasoning environments. The survey "Puzzle Solving using Reasoning of LLMs: A Survey" defines puzzles as problems that test cognitive abilities including logical reasoning, spatial cognition, and creative thinking by requiring the solver to discern patterns, apply deduction, and combine insights from available information in order to arrive at the correct solution (Giadikiaroglou et al., 2024). "PUZZLES: A Benchmark for Neural Algorithmic Reasoning" frames each puzzle as an episodic MDP with deterministic transitions, sparse end-of-episode rewards, and configurable state representations (Estermann et al., 2024). "PuzzlePlex: Benchmarking Foundation Models on Reasoning and Planning with Puzzles" formalizes each puzzle as either a single-player MDP or a two-player game with explicit state $S$, actions $T_i$0, and transitions $T_i$1 (Long et al., 7 Oct 2025).

This suggests that puzzle is best understood not as a single genre but as a family of finite formal systems in which admissible operations are sharply constrained and correctness is externally checkable.

2. Graph-theoretic and combinatorial puzzle models

Graph-based puzzles occupy a central place in the mathematical literature. Reeder’s puzzle is a “chip–flipping” game played on a graph, originally motivated by Weyl-group actions and later generalized to non-simply-laced Dynkin diagrams and affine Dynkin diagrams (Evenor, 2015). For simply-laced graphs, the move at vertex $T_i$2 is

$T_i$3

with the sum computed modulo $T_i$4 (Evenor, 2015). The paper provides complete solutions for all Dynkin and affine Dynkin diagrams and proves the $T_i$5-Tree Theorem: for a simply-laced tree containing $T_i$6 as a subgraph, besides fixed labelings there are exactly two classes, distinguished by parity of the number of connected components of 1’s (Evenor, 2015).

Jigsaw percolation offers a different graph-theoretic paradigm. The model uses two graphs on the same vertex set $T_i$7: a people graph $T_i$8 and a puzzle graph $T_i$9 (Brummitt et al., 2012). Clusters merge only when they are both people-adjacent and puzzle-adjacent, and the puzzle is solved if the dynamics eventually produce the single cluster $i$0 (Brummitt et al., 2012). For Erdős–Rényi people graphs and the $i$1-cycle puzzle graph, the critical value satisfies

$i$2

more precisely

$i$3

(Brummitt et al., 2012). For arbitrary connected bounded-degree puzzle graphs, $i$4 and $i$5 for every $i$6 (Brummitt et al., 2012). By contrast, for power-law degree distributions in the people graph, the model implies that with probability tending to 1 such social networks cannot solve any bounded-degree puzzle (Brummitt et al., 2012).

The Sol LeWitt puzzle studied in "Exploration of another Sol Lewitt puzzle from Barry Cipra" is likewise combinatorial and graph-like, though embedded on a torus (Cipra et al., 2019). A $i$7 array of 16 non-rotated tiles, one for each 4-bit pattern $i$8, induces a 4-regular graph on the torus with 64 arc segments (Cipra et al., 2019). The paper proves a Parity Theorem: the number of loops and the number of crossings have the same parity, implying that with one of each tile the number of loops must be even (Cipra et al., 2019). Since a single loop would have odd parity, a 64-arc Hamiltonian-style loop is impossible; the paper exhibits a configuration with loop lengths 60 and 4, establishing 60 as the maximal loop length in that setting (Cipra et al., 2019). It also shows that every loop length is divisible by 4 (Cipra et al., 2019).

These graph-theoretic cases illustrate three distinct research roles for puzzles: as orbit problems under local moves, as percolation processes on paired networks, and as topological-combinatorial systems with loop invariants.

3. Spatial, geometric, and mechanical puzzles

A second major cluster concerns spatial puzzles whose states are arrangements of pieces in Euclidean or grid-like environments. The paper "A Global Approach for Solving Edge-Matching Puzzles" studies apictorial edge-matching puzzles in which polygonal pieces with colored edges must be placed so that touching edges have the same color and opposite orientation (Kovalsky et al., 2014). The puzzle is encoded as a polynomial system

$i$9

derived from the identity

$\mathrm{Cl}(D)$0

for all edge types $\mathrm{Cl}(D)$1 and suitable test functions $\mathrm{Cl}(D)$2 (Kovalsky et al., 2014). Under explicit conditions, this polynomial representation is complete: its solutions correspond exactly to valid puzzle assemblies (Kovalsky et al., 2014). The paper then develops convex relaxations via Vandermonde and rank-one Hankel formulations, leading to LP and SDP heuristics for approximate global solution (Kovalsky et al., 2014).

"Mutando of Insanity" studies colored-cube puzzles derived from Instant Insanity (Roa, 2016). A candidate puzzle is an $\mathrm{Cl}(D)$3 proper matrix over $\mathrm{Cl}(D)$4, where each row encodes the three opposite face-pairs of a cube and each prime in $\mathrm{Cl}(D)$5 represents a color (Roa, 2016). For $\mathrm{Cl}(D)$6, the paper enumerates admissible cube rows, constructs all puzzles modulo equivalence, and computes the number of solutions by determining the set $\mathrm{Cl}(D)$7 of partial $\mathrm{Cl}(D)$8-solutions and the set $\mathrm{Cl}(D)$9 of full solutions (Roa, 2016). It finds that the maximal number of solutions is 72 for $\#\mathrm{Cl}(D)$0 and 18 for $\#\mathrm{Cl}(D)$1, and that not every intermediate count occurs (Roa, 2016). The same framework produces the "Mutando of Insanity", a four-cube Insanity puzzle that is simultaneously solvable as a $\#\mathrm{Cl}(D)$2 tower with each long face showing all four colors exactly once and as a $\#\mathrm{Cl}(D)$3 slab whose six outer faces are monochromatic (Roa, 2016).

"Gourds: a sliding-block puzzle with turning" introduces a single-empty-cell puzzle on a hexagonal grid of $\#\mathrm{Cl}(D)$4 cells populated by $\#\mathrm{Cl}(D)$5 rigid $\#\mathrm{Cl}(D)$6 pieces (Hamersma et al., 2020). Legal moves are slide, turn, and pivot moves involving exactly one gourd and the empty cell (Hamersma et al., 2020). For proper boards—hole-free, 2-connected, odd-sized boards that are not the Star of David—the paper proves that both Numbered Gourd Reconfiguration and Colored Gourd Reconfiguration can always be solved (Hamersma et al., 2020). It gives a tight worst-case bound of $\#\mathrm{Cl}(D)$7 moves for reconfiguration (Hamersma et al., 2020). The proof uses a Hamiltonian cycle of the board graph, alignment of gourds to that cycle, and a divide-and-conquer argument on the dual tree of a triangulation (Hamersma et al., 2020). The paper also studies Colored Gourd Placement, proving NP-completeness for arbitrarily many colors and a randomized polynomial-time algorithm when the number of colors is a fixed constant (Hamersma et al., 2020).

These works treat puzzles as exact spatial arrangement or motion problems, but their methods differ sharply: algebraic geometry and convex relaxation for edge-matching, number-theoretic encoding and exhaustive enumeration for cube puzzles, and Hamiltonian-cycle structure and graph decomposition for sliding-block motion.

4. Computational complexity and hardness of puzzles

The complexity-theoretic perspective treats puzzle-solving as a family of decision, counting, search, and reconfiguration problems. The strongest result in this set is the ASP-completeness framework for pencil puzzles. "ASP-Completeness Proofs of Puzzles Using the T-Metacell Framework" shows that Grand Tour, Entry Exit, Yagit, and Zahlenschlange are ASP-complete (Kiatchaipipat et al., 15 Aug 2025). In the paper’s formulation, an NP search problem is ASP-complete if each NP search problem reduces to it together with a polynomial-time bijection between solution sets (Kiatchaipipat et al., 15 Aug 2025). The consequences are that the decision version is NP-complete, the counting version is $\#\mathrm{Cl}(D)$8-complete, and the $\#\mathrm{Cl}(D)$9-ASP problem of finding another solution given $n$0 known ones is NP-complete for any $n$1 (Kiatchaipipat et al., 15 Aug 2025).

The reductions use the T-metacell framework developed by Tang and the MIT Hardness Group (Kiatchaipipat et al., 15 Aug 2025). Finding Hamiltonian cycles on rectangular grids of undirected T-metacells, and on rectangular grids of asymmetric required-edge undirected T-metacells, is ASP-complete (Kiatchaipipat et al., 15 Aug 2025). The paper constructs puzzle-specific gadgets that simulate T-metacells: forced-edge Hamiltonian-cycle gadgets in Grand Tour, region-entry/exit gadgets in Entry Exit, sheep–wolf partition gadgets in Yagit, and numbered 3×3 gadgets in Zahlenschlange (Kiatchaipipat et al., 15 Aug 2025). This modular hardness framework shows that the same combinatorial core can be embedded in multiple apparently disparate pencil puzzles.

Earlier complexity results within the supplied literature are more specialized. Colored Gourd Placement is NP-complete, even on a hole-free board of height 4 hexagons (Hamersma et al., 2020). The paper’s proof is by reduction from Monotone 1-in-3SAT with exactly 3 occurrences per variable and exactly 3 literals per clause (Hamersma et al., 2020). For fixed $n$2 colors, however, the same problem is solvable in randomized polynomial time via colored-matching techniques (Hamersma et al., 2020). The edge-matching paper emphasizes that its exact polynomial encoding yields a nonconvex continuous system whose convex relaxations are heuristic rather than guaranteed globally convergent (Kovalsky et al., 2014). "Mutando of Insanity" notes that the generalization of Brown’s algorithm to arbitrary $n$3 is NP-complete, while its exhaustive computations for $n$4 were feasible and completed in less than 24 hours (Roa, 2016).

The complexity viewpoint also appears in benchmark design. The LLM survey stresses that many rule-based puzzles are closed worlds with explicit state transitions and constraints, making them natural tests of algorithmic reasoning (Giadikiaroglou et al., 2024). PuzzlePlex explicitly uses rule-based puzzles with clear state transitions and objectives to measure reasoning and planning without external factual knowledge (Long et al., 7 Oct 2025). PUZZLES similarly argues that many included tasks correspond to algorithmic problems such as constraint propagation, graph search, combinatorial optimization, and sliding-tile search (Estermann et al., 2024).

A plausible implication is that complexity-theoretic hardness and benchmark difficulty are related but not identical. The former concerns worst-case asymptotics and exact reductions; the latter concerns empirical solvability by learned systems under finite compute and representation constraints.

5. Puzzles as datasets and benchmarks for machine reasoning

In recent work, puzzles are increasingly treated as evaluation environments for AI systems rather than merely mathematical objects. "A Puzzle-Based Dataset for Natural Language Inference" introduces PuzzTE, a dataset built from three families of logical puzzles: comparison puzzles, knights and knaves, and zebra puzzles (Szomiu et al., 2021). Each puzzle is translated into first-order logic using feature grammars and background axioms; all ground atomic questions are then generated, and each is labeled as entailment, contradiction, or unknown by Prover9 and Mace4 (Szomiu et al., 2021). For a puzzle theory $n$5 and ground atom $n$6, the effective labeling rule is

$n$7

(Szomiu et al., 2021). The complete-information subset contains 4,136 questions, while the ambiguous subset contains 16,745 questions (Szomiu et al., 2021). The paper emphasizes that “good puzzles” have two properties: each piece of information is necessary and no unnecessary information is provided (Szomiu et al., 2021).

The survey "Puzzle Solving using Reasoning of LLMs: A Survey" organizes textual puzzles into rule-based and rule-less classes (Giadikiaroglou et al., 2024). Rule-based puzzles include deterministic games such as Sudoku, the 8-puzzle, Rubik’s Cube, and maze navigation, and stochastic games such as Minesweeper and Poker (Giadikiaroglou et al., 2024). Rule-less puzzles include riddles, multiple-choice question puzzles, programming puzzles, and commonsense reasoning puzzles (Giadikiaroglou et al., 2024). The survey reviews prompting methods such as chain-of-thought, Tree-of-Thought, Graph-of-Thought, and XoT, neuro-symbolic methods that translate puzzle descriptions into formal languages, and fine-tuning on puzzle datasets (Giadikiaroglou et al., 2024). It finds that structured prompting and search help especially on deterministic, rule-based puzzles, whereas commonsense and lateral-thinking puzzles remain difficult (Giadikiaroglou et al., 2024).

"PUZZLES: A Benchmark for Neural Algorithmic Reasoning" builds a Gymnasium-compatible RL benchmark from 40 puzzles in Simon Tatham’s Portable Puzzle Collection (Estermann et al., 2024). The environment supplies two observation modalities—internal symbolic state or RGB pixels—a small discrete action space, deterministic transitions, and sparse end-of-episode rewards (Estermann et al., 2024). The paper evaluates PPO, Recurrent PPO, A2C, A3C, TRPO, DQN, QRDQN, MuZero, and DreamerV3 (Estermann et al., 2024). DreamerV3 achieves the best overall baseline performance, with average successful episode length $n$8 and success rate 62.7%, but remains far from the manually constructed optimal upper bound average of 217 steps (Estermann et al., 2024). Many puzzles, including Loopy, Pearl, Pegs, Solo, and Unruly, remain effectively unsolved by the baselines (Estermann et al., 2024).

PuzzlePlex extends the benchmark idea to foundation models. It contains 15 rule-based puzzles spanning single- and two-player, deterministic and stochastic, text-only and text–image settings (Long et al., 7 Oct 2025). It evaluates models in two modes: instruction-based interaction and code-based execution (Long et al., 7 Oct 2025). Scores are normalized to $n$9, and results are also summarized via Elo ratings

$n\times 3$0

with $n\times 3$1 (Long et al., 7 Oct 2025). Reasoning-specialized models such as DeepSeek-R1, o4-mini, Gemini-2.5-pro, and QwQ-32B outperform non-reasoning models in instruction-based settings, while code-based settings remain substantially harder (Long et al., 7 Oct 2025). The benchmark also decomposes failures into legal terminations, rule violations, instruction-following failures, syntax errors, runtime errors, and timeouts (Long et al., 7 Oct 2025).

These benchmark papers recast puzzles as controlled laboratories for algorithmic reasoning, planning, inference, and generalization.

6. Conceptual unification and research significance

Across these domains, the literature supports a layered understanding of puzzle. At the most formal level, a puzzle is a finite structured system endowed with admissible operations and a correctness criterion. Reeder’s puzzle uses graph labelings and parity-sensitive moves (Evenor, 2015). Insanity-type cube puzzles use prime-product matrices and multiplicative constraints (Roa, 2016). Edge-matching puzzles use colored edge alignments encoded by polynomial equations (Kovalsky et al., 2014). Logic-puzzle datasets use first-order theories and theorem provers (Szomiu et al., 2021). RL and foundation-model benchmarks use explicit state, action, and transition interfaces (Estermann et al., 2024, Long et al., 7 Oct 2025).

At the methodological level, puzzle research serves several distinct purposes. It enables exact combinatorial classification, as in the determination of equivalence classes for Dynkin and affine Dynkin diagrams (Evenor, 2015). It supports exhaustive enumeration and design of new artifacts, as in the search over all Insanity puzzles with $n\times 3$2 and the construction of the Mutando of Insanity (Roa, 2016). It motivates algebraic formulations that replace discrete search with polynomial systems and convex relaxations (Kovalsky et al., 2014). It provides natural sources of computational hardness and reduction gadgets (Kiatchaipipat et al., 15 Aug 2025). It furnishes clean datasets and environments for evaluating natural-language and reinforcement-learning systems (Szomiu et al., 2021, Estermann et al., 2024, Long et al., 7 Oct 2025).

At the conceptual level, several tensions recur. One is the tension between local rules and global structure: local flips in Reeder’s puzzle preserve or alter global component invariants (Evenor, 2015); local crossing changes in the Sol LeWitt puzzle control the global parity of loops (Cipra et al., 2019); local edge constraints in edge-matching induce global polynomial identities (Kovalsky et al., 2014). A second is the contrast between exact symbolic reasoning and approximate computational methods. PuzzTE relies on theorem proving (Szomiu et al., 2021), whereas PUZZLES and PuzzlePlex expose the limitations of learned policies and LLMs in long-horizon or highly constrained settings (Estermann et al., 2024, Long et al., 7 Oct 2025). A third is the contrast between human-style puzzle design and formal complexity: “good puzzle” discipline in PuzzTE emphasizes necessity and nonredundancy of clues (Szomiu et al., 2021), while ASP-completeness results show that even deciding whether another solution exists can be computationally hard (Kiatchaipipat et al., 15 Aug 2025).

This suggests that puzzle has become a unifying term for a heterogeneous but mathematically coherent class of problems at the intersection of combinatorics, algorithms, topology, logic, optimization, and AI evaluation. In that sense, puzzle is not merely a recreational category. It is a research object that can encode group actions, percolation thresholds, geometric reconfiguration, NP- and ASP-hardness, formal semantics, and algorithmic generalization, depending on which structure is foregrounded (Evenor, 2015, Brummitt et al., 2012, Hamersma et al., 2020, Kiatchaipipat et al., 15 Aug 2025, Szomiu et al., 2021, Estermann et al., 2024, Long et al., 7 Oct 2025).