PuzzleClone: Dual Research Systems

• PuzzleClone represents two distinct research systems, one for generating Arabic educational crosswords with LLMs and another for synthesizing logic puzzles via SMT.
• The Arabic crossword system uses a five-stage LLM pipeline with prompt engineering, heuristic filtering, and grid optimization to produce educational puzzles.
• The SMT-powered framework encodes puzzles as finite-domain constraint problems and employs solver-based checks to ensure both solvability and unique solutions.

PuzzleClone is a name used for two distinct 2025 research systems. In "From Arabic Text to Puzzles: LLM-Driven Development of Arabic Educational Crosswords" (Zeinalipour et al., 19 Jan 2025), it denotes an LLM-driven pipeline that takes arbitrary Arabic text and produces educational crossword puzzles through clue generation, clue-answer extraction, heuristic filtering, grid construction, and rendering. In "PuzzleClone: An SMT-Powered Framework for Synthesizing Verifiable Data" (Xiong et al., 21 Aug 2025), it denotes a formal framework for synthesizing logic and arithmetic puzzles by encoding seed puzzles as finite-domain constraint systems, generating randomized variants, and validating satisfiability and uniqueness with SMT. The shared name therefore covers two markedly different technical programs: one centered on Arabic educational crossword generation, the other on programmatically validated puzzle synthesis for reasoning data.

1. Disambiguation and scope

The name PuzzleClone appears in two separate technical contexts, with different problem formulations, different target outputs, and different validation regimes.

Usage	Primary artifact	Validation basis
PuzzleClone in (Zeinalipour et al., 19 Jan 2025)	Arabic educational crossword puzzles from text	ROUGE-L, human A–E quality ratings, grid density
PuzzleClone in (Xiong et al., 21 Aug 2025)	Verifiable logic and arithmetic puzzles	SMT satisfiability, SMT uniqueness checks, benchmark performance

In the Arabic crossword setting, the central object is a natural-language clue-answer pair grounded in a supplied Arabic passage or a list of keywords plus a category. In the SMT setting, the central object is a finite-domain puzzle instance whose rules are encoded as constraints over integer-typed variables and whose validity is established by solver-based checks. The shared name can therefore obscure the fact that the two systems are not variants of a single framework, but separate lines of work with distinct methodological commitments (Zeinalipour et al., 19 Jan 2025, Xiong et al., 21 Aug 2025).

2. Arabic educational crossword PuzzleClone

The Arabic PuzzleClone workflow is specified as an end-to-end architecture with five layers. In the input layer, a teacher or end-user supplies an arbitrary Arabic passage or a list of keywords plus a target category such as Chemistry or History. In the clue-generation module, a prompt template is filled with the passage, keyword, and category; GPT-4-Turbo in zero/few-shot mode first generates high-quality examples for a new dataset, and GPT-3.5-Turbo and Llama3-8B-Instruct are then fine-tuned on that dataset. In candidate clue-answer extraction, each model emits $(\text{clue}, \text{answer}, \text{category})$ tuples, after which a small heuristic or learned ranker filters out low-quality clues. In the crossword grid builder, a classical slot-filling/backtracking algorithm places answers in a grid while optimizing for grid density,

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$

The system may also add scoring functions for minimum word intersections or theme density. The output renderer then produces across/vertical clues, a grid image such as SVG or HTML, and an answer key (Zeinalipour et al., 19 Jan 2025).

This pipeline is explicitly educational in orientation. The underlying paper presents it as a response to the scarcity of advanced educational tools tailored for the Arabic language, and situates the system in language learning, cognitive development, gamification, and interactivity. The model and dataset are publicly available, which makes the framework not only a deployed generation pipeline but also a reusable research substrate for Arabic clue generation and crossword construction (Zeinalipour et al., 19 Jan 2025).

3. Arabic-Clue-Instruct dataset, prompting, and evaluation

The Arabic crossword system is built around the Arabic-Clue-Instruct dataset. The dataset begins with 14,497 distinct $(\text{text}, \text{keyword})$ pairs drawn from Arabic Wikipedia across 20 educational topics such as Geography, Biology, and Philosophy. After GPT-4 clue generation, it expands to 54,196 unique $$(\text{text}, \text{answer}, \text{clue}, \text{category})$$ tuples. Each entry contains a text field of 75–1,000 words, with most texts between 40–200 words; an answer or keyword of 2–20 Arabic characters, restricted to single or two-word terms; a clue averaging 20–30 characters; and one of 20 category labels. The curation pipeline is defined as: scrape the first section of Arabic Wikipedia articles; filter out texts with fewer than 50 words or more than 1,000 words; discard keywords longer than two words or containing numerals or special characters; prompt GPT-4-Turbo to generate at least three clues per $(\text{text}, \text{keyword}, \text{category})$; and perform human-expert review on a random subset with A–E quality ratings (Zeinalipour et al., 19 Jan 2025).

The prompt template used with GPT-4-Turbo for dataset creation is given explicitly: “Given the following Arabic passage and the target keyword ‘X’ in category ‘Y,’ write an instructive crossword clue in Arabic that does not use the keyword itself but clearly points to it.” Fine-tuning recipes are also specified: GPT-3.5-Turbo uses batch_size=16, lr=1e-2, epochs=3, while Llama3-8B-Instruct via LoRA uses $(r=32, \alpha=64)$ with batch_size=128, lr=3×10⁻⁴, epochs=3. Inference settings differ by model role: GPT-4-Turbo for dataset generation uses temperature=0.7, top_p=0.9; GPT-3.5-Turbo fine-tuned inference uses temperature=0.3, top_p=0.9; and Llama3-8B fine-tuned inference uses temperature=0.1, top_p=0.95, top_k=50. Grid filling uses standard backtracking with heuristics to maximize word intersections, and the system periodically re-prompts LLMs if a clue-answer pair cannot be placed (Zeinalipour et al., 19 Jan 2025).

The evaluation protocol combines automated and human criteria. Grid density is defined as

$$\text{density} = \frac{\text{filled\_cells}}{\text{total\_cells}}.$$

Clue-context fidelity uses ROUGE-L, described as the F-measure over longest common subsequences; for generated clue $C$ and source text $T$,

$$\text{ROUGE-L}(C,T) = \frac{(1 + \beta^2)\cdot R \cdot P}{R + \beta^2 \cdot P},$$

where

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$0

An optional inter-clue coherence metric is based on cosine similarity among clue embeddings,

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$1

and an example clue difficulty score is

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$2

On 200 test contexts against GPT-4 clues, the reported ROUGE-L values are 0.0278 for GPT-4-Turbo against itself, 0.0148 for GPT-3.5-Turbo base, 0.0063 for Llama3-8B base, 0.0405 for GPT-3.5-Turbo fine-tuned, and 0.0354 for Llama3-8B fine-tuned. In human evaluation on 200 contexts times 3 clues, GPT-3.5 base receives 28.5% A, 23.2% B, 0.0% C, 35.1% D, and 13.2% E; GPT-3.5 fine-tuned receives 54.3% A, 22.3% B, 15.0% C, 8.3% D, and 0.0% E; Llama3-8B base receives 36.0% A, 12.8% B, 0.3% C, 37.5% D, and 13.3% E; and Llama3-8B fine-tuned receives 78.9% A, 2.7% B, 0.0% C, 18.1% D, and 0.3% E. The reported takeaways are that fine-tuning yields large gains in both ROUGE and human ratings, that Llama3-8B-Instruct fine-tuned achieves the highest A score at 78.9%, and that GPT-3.5-Turbo fine-tuned closes much of the gap with GPT-4 on simple clue generation (Zeinalipour et al., 19 Jan 2025).

4. Portability of the Arabic crossword framework

The Arabic crossword PuzzleClone is presented as extensible to other languages and domains. For data collection, the procedure is to swap in the corresponding Wikipedia or domain corpus, such as medical articles, and apply the same length and keyword filters while adjusting thresholds for language-specific tokenization. For prompt adaptation, the recommendation is to translate and culturally adapt the prompt template and provide two to three few-shot examples in the target language for categories and clue style. For model selection, the framework allows any multilingual or language-specific LLM, with BLOOM and Mistral for French given as examples, followed by fine-tuning on a newly created Clue-Instruct-style dataset. For evaluation, the quality scales are to be redefined with native speakers while retaining the core metrics of grid density, ROUGE, and semantic coherence (Zeinalipour et al., 19 Jan 2025).

This suggests that the Arabic implementation is intended less as a one-off application than as a recipe: scrape structured text, build a self-instruct dataset with GPT-4, fine-tune open LLMs, construct an efficient grid-filling backend, and combine automated with human evaluation. A plausible implication is that the domain dependence lies primarily in corpus choice, prompt localization, and native-speaker evaluation rather than in the high-level architecture itself (Zeinalipour et al., 19 Jan 2025).

5. SMT-powered PuzzleClone as formal puzzle synthesis

The second PuzzleClone is a formal framework for generating “small-to-medium-sized pencil-and-paper logic and arithmetic puzzles” such as Latin squares, KenKen, Kakuro, and “sum” puzzles. Each puzzle is treated as a finite-domain Constraint Satisfaction Problem. The framework targets puzzles whose rules can be written as a finite set of integer-typed variables $$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$3, each with a small domain $$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$4, together with a finite conjunction of Boolean constraints $$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$5 built from linear arithmetic, equalities, and optionally user-defined relations such as “distinct” or “sum.” The seed puzzle is encoded as

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$6

A valid puzzle instance must satisfy two conditions: it must be satisfiable, $$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$7, and it must be uniquely solvable, meaning exactly one model in the finite grid $$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$8 (Xiong et al., 21 Aug 2025).

The specification language uses the SMT-LIBv2 core theory of Int, plus two common helpers:

$$\text{grid\_density} = \frac{\text{number\_of\_filled\_cells}}{\text{total\_cells}}.$$9

The paper illustrates the encoding with a “Sum-3” seed puzzle: choose three distinct numbers in $(\text{text}, \text{keyword})$0 that add to 9. In SMT-LIB style, this is written with integer declarations for a, b, and c, assertions that each lies between 1 and 5, an assertion (= (+ a b c) 9), an assertion (distinct a b c), and then (check-sat). The framework then applies systematic randomization to domain bounds and numerical constants in constraints while preserving overall structure. In the pseudocode for GenerateVariant, each variable receives randomized domain bounds, each constraint is either left unchanged or mutated according to p_mutate, and the result is a new formula $\phi&#39;=(V,D&#39;,C&#39;)<code>. If the seed contains</code>(= (+ a b c) 9)<code>, a variant may become</code>(= (+ a b c) 10)<code>or</code>(\le (+ a b c) 12)` (<a href="/papers/2508.15180" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Xiong et al., 21 Aug 2025</a>).</p> <p>Validity is enforced by what the paper calls a reproduction mechanism. First, an existence check requires <code>check-sat(φ&#39;)</code> to return <code>sat</code>, yielding a model $(\text{text}, \text{keyword})$1. Second, uniqueness is tested by asserting a blocking clause

$(\text{text}, \text{keyword})$2</p> <p>and requiring <code>check-sat(φ&#39; \wedge B)</code> to return <code>unsat</code>. If this holds, the candidate has exactly one solution in the finite search space. The worked example begins with $(\text{text}, \text{keyword})$3 defined by $(\text{text}, \text{keyword})$4, $(\text{text}, \text{keyword})$5, and $(\text{text}, \text{keyword})$6, for which Z3 returns sat with model $(\text{text}, \text{keyword})$7 and a successful uniqueness check. A randomized variant mutates the sum bound $(\text{text}, \text{keyword})$8 and domain upper bound $(\text{text}, \text{keyword})$9, producing

$(\text{text}, \text{answer}, \text{clue}, \text{category})$0

for which Z3 yields $(\text{text}, \text{answer}, \text{clue}, \text{category})$1 and confirms uniqueness via the blocking-clause check (Xiong et al., 21 Aug 2025).

6. SMT pipeline, benchmark construction, and learning results

The SMT PuzzleClone implementation is in Python and uses the Z3 API for all SMT calls. Its pipeline has five main stages: a Parser/Encoder that reads a human-author seed description and emits $(\text{text}, \text{answer}, \text{clue}, \text{category})$2; a Variant-Generator that runs GenerateVariant in parallel across worker threads; a Solver Manager that issues batched SMT queries for existence and uniqueness; a Deduplicator that hashes each accepted formula up to symmetry to avoid repeats; and a Serializer that exports puzzles into JSON, SMT-LIB text, and plain-text formats. The reported optimizations are caching the solver context for each seed to avoid re-parsing, adaptive mutation rates that shrink $(\text{text}, \text{answer}, \text{clue}, \text{category})$3 if too many mutants fail uniqueness, and lock-free queues for distributing work to thousands of cores (Xiong et al., 21 Aug 2025).

The final benchmark contains 83,215 puzzles drawn from four families: 15,230 “Sum-3” instances, 18,542 Latin squares up to $(\text{text}, \text{answer}, \text{clue}, \text{category})$4, 28,002 KenKen instances for $(\text{text}, \text{answer}, \text{clue}, \text{category})$5 to $(\text{text}, \text{answer}, \text{clue}, \text{category})$6, and 21,441 Kakuro instances on small $(\text{text}, \text{answer}, \text{clue}, \text{category})$7 to $(\text{text}, \text{answer}, \text{clue}, \text{category})$8 grids. Difficulty is measured by total Z3 solve time on a single core. The category-wise distributions are: Sum-3, 80% Easy $(\text{text}, \text{answer}, \text{clue}, \text{category})$9, 19% Medium $(\text{text}, \text{keyword}, \text{category})$0, 1% Hard $(\text{text}, \text{keyword}, \text{category})$1; Latin, 60% Easy, 30% Medium, 10% Hard; KenKen, 45% Easy, 40% Medium, 15% Hard; Kakuro, 50% Easy, 35% Medium, 15% Hard. All puzzles are released in three formats: SMT-LIB v2, a JSON schema, and human-readable ASCII (Xiong et al., 21 Aug 2025).

The learning experiments fine-tune a transformer-based CodeLM first through supervised fine-tuning on 60,000 PuzzleClone puzzles and then through reward-weighted policy gradient using the uniqueness check as a reward. The detailed evaluation section reports exact-match results on a held-out PuzzleClone test set of 8,000 puzzles and two external benchmarks. On PuzzleClone, performance moves from 20.1% for the base model to 45.3% after SFT and 53.2% after SFT+RL. On GSM8K, the corresponding values are 14.8%, 16.9%, and 18.4%. On Math23K, they are 19.5%, 22.0%, and 23.1%. The paper states that statistically significant gains with $(\text{text}, \text{keyword}, \text{category})$2 under the Wilcoxon signed-rank test are observed across all benchmarks after RL. The abstract reports a broader summary: post training raises PuzzleClone average from 14.4 to 56.2 and delivers consistent improvements across 7 logic and mathematical benchmarks up to 12.5 absolute percentage points, with AMC2023 improving from 52.5 to 65.0. Code and data are available at the project repository (Xiong et al., 21 Aug 2025).