GPT-OSS-Puzzle-88B: Optimized MoE Model
- gpt-oss-puzzle-88B is an 88-billion-parameter open-source GPT model that uses an extended Puzzle NAS framework to optimize mixture-of-experts configurations for efficient reasoning.
- It reduces inference cost by dynamically tuning expert counts, integrating selective window attention, and applying FP8 KV-cache quantization to boost throughput and lower latency.
- The model also explores text-native puzzle solving and cooperative evaluation via benchmarks like MultiZebraLogic and AsymPuzl to assess structured reasoning performance.
to=arxiv_search.search 天天中彩票的 彩神争霸是 code 天天中彩票公众号json {"8query8 OR \8"Extending Puzzle for Mixture-of-Experts Reasoning Models with Application to GPT-OSS Acceleration8\8 OR MultiZebraLogic OR AsymPuzl8", "8max_results8 8gpt-oss-puzzle-88B OR \8query8, "8sort_by8 "8relevance8 to=arxiv_search.search 手机天天彩票 code 天天中彩票上json {"8query8 Puzzle for Mixture-of-Experts Reasoning Models with Application to GPT-OSS Acceleration8\8 "8max_results8 8relevance8, "8sort_by8 "8relevance8 to=arxiv_search.search ,最新高清无码专区 code  ̄奇米json {"8query8 &&&8gpt-oss-puzzle-88B OR \8&&&, &&&8 OR MultiZebraLogic OR AsymPuzl8&&&, &&&8max_results8&&&)", "8max_results8 8gpt-oss-puzzle-88B OR \8query8, "8sort_by8 "8relevance8 gpt-oss-puzzle-88B denotes an 88-billion-parameter open-source GPT-style model situated at the intersection of post-training architecture optimization and puzzle-oriented reasoning research. In one usage, it is the deployment-optimized derivative of gpt-oss-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B produced by extending the Puzzle post-training neural architecture search (NAS) framework to mixture-of-experts (MoE) reasoning models (&&&8max_results8&&&). In another, closely related usage, it names an initiative that scales the “unnatural language” puzzle-solving paradigm—treating mazes, Rubik’s Cube states, and Sudoku grids as plain-text token sequences—to an approximately 88 B-parameter open-source transformer (&&&8gpt-oss-puzzle-88B OR \8&&&). Taken together, these usages identify a model family and research program concerned with reducing inference cost while preserving, and in some cases improving, performance on reasoning-heavy workloads and structured puzzle domains.
8gpt-oss-puzzle-88B OR \8. Scope and conceptual lineage
The designation combines two distinct senses of “Puzzle.” In the optimization literature, Puzzle is a post-training NAS method that shrinks an existing LLM under hard deployment constraints such as memory, latency, and throughput, without further pre-training (&&&8max_results8&&&). In puzzle-solving work, by contrast, “puzzle” refers to domains such as mazes, Rubik’s Cube, Sudoku, zebra puzzles, and asymmetric cooperative symbolic tasks, all expressed or evaluated through textual or formal interfaces (&&&8gpt-oss-puzzle-88B OR \8&&&).
This dual lineage matters because the model’s identity is not exhausted by either strand alone. As an optimized derivative of gpt-oss-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B, gpt-oss-puzzle-88B is a systems artifact: it inherits the parent’s 8gpt-oss-puzzle-88B OR \88-layer base, preserves hidden size and head count, and modifies the MoE and attention subblocks to improve serving efficiency (&&&8max_results8&&&). As an 88 B-scale puzzle-solving initiative, it is also a methodological vehicle for testing whether large decoder-only transformers can solve sparse-reward combinatorial problems directly from text archives, without search trees, explicit constraint checks, or human-crafted heuristics (&&&8gpt-oss-puzzle-88B OR \8&&&).
A common misconception is to read the model name as implying only a benchmark specialization in puzzles. The available sources instead describe a broader configuration: the word “Puzzle” refers both to the optimization framework that produced the 88 B derivative and to a family of evaluation environments in which such a model can be studied. This suggests that the model is best understood as a convergence point between inference-efficient reasoning-model design and structured puzzle evaluation.
8 OR MultiZebraLogic OR AsymPuzl8. Extension of Puzzle to mixture-of-experts reasoning models
In its vanilla form, Puzzle treats each transformer layer as a block with alternative implementations for attention and FFN subblocks, assigns each candidate block a replace-8gpt-oss-puzzle-88B OR \8-block score, solves a mixed-integer program (MIP) to select layerwise replacements under resource constraints, and then heals the heterogeneous student model through a short end-to-end knowledge-distillation pass (&&&8max_results8&&&). For gpt-oss-puzzle-88B, this framework was extended to MoE LLMs.
The parent gpt-oss-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B uses a 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88-expert MoE in each transformer layer. The extension therefore expands each MoE FFN slot into a library of subblocks that keep only PRESERVED_PLACEHOLDER_8query8^ experts out of the original 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88, where PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8. Each such MoE variant is scored by activation-based replace-8gpt-oss-puzzle-88B OR \8-block mean-squared error, and the resulting expert-count choices are incorporated into Puzzle’s MIP alongside attention alternatives (&&&8max_results8&&&).
The expert-scoring procedure is defined as
PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8^
where PRESERVED_PLACEHOLDER_8max_results8^ is a parent-model MoE layer and PRESERVED_PLACEHOLDER_8sort_by8^ is the same MoE with expert PRESERVED_PLACEHOLDER_8relevance8^ zeroed out. Experts are sorted by PRESERVED_PLACEHOLDER_8query8^ from smallest to largest, and subblocks are then built by keeping the top-PRESERVED_PLACEHOLDER_8\8^ experts. This is the mechanism by which the search process determines which layers tolerate aggressive pruning and which must remain close to the parent (&&&8max_results8&&&).
The resulting student has 88 B parameters, or 8\8max_results8% of the parent’s 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8^ B. The source describes this as a reduction in per-token FFN FLOPs, weight bandwidth, and memory footprint, which in turn lifts batch sizes and throughput, particularly in the MoE-dominated short-context regime (&&&8max_results8&&&). The architecture is therefore not a simple uniform downsizing; it is a heterogeneous student selected by constrained post-training search.
8max_results8. Optimization stack and architectural configuration
The final model is assembled from four complementary optimizations rather than a single compression step (&&&8max_results8&&&).
| Component | Design choice | Stated role |
|---|---|---|
| MoE expert pruning | Keep experts per layer | Reduce FFN FLOPs, weight bandwidth, and memory footprint |
| Selective window attention | Convert 8 of 8gpt-oss-puzzle-88B OR \88^ global layers to window attention with | Bound long-context compute and KV-cache cost |
| FP8 KV-cache quantization | Store KV in FP8 with calibrated per-layer scales | Halve KV-cache footprint and unlock faster attention microkernels |
| Post-training RL tuning | Average “High-only” and “Balanced” RL checkpoints | Match high-effort accuracy while retaining medium verbosity |
The layerwise expert counts are explicitly heterogeneous. Early layers remain at 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88^ experts, while later layers drop as low as 8, with the 8gpt-oss-puzzle-88B OR \88-layer profile given as
PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8query8^
This pattern reflects the MIP’s decision about which layers can be pruned with minimal impact (&&&8max_results8&&&).
Long-context optimization is handled through selective window attention rather than blanket replacement. Standard full attention is described as incurring
PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8gpt-oss-puzzle-88B OR \8^
where PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8^ is sequence length. Sliding-window attention with window size PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8max_results8^ reduces these to
PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8sort_by8^
However, converting all global layers to window attention breaks long-range dependencies, so the search procedure enumerates, for each original global-attention layer, a full-attention alternative and a window-attention alternative with PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8relevance8, scores window variants by their drop on AA-LCR, and jointly optimizes short- and long-context scenarios. The final selection converts exactly 8 of 8gpt-oss-puzzle-88B OR \88^ global layers to window attention. The YaRN RoPE scaling factor is also tuned from 8max_results8 OR MultiZebraLogic OR AsymPuzl8^ to 8relevance8query8^ to stabilize phase wrapping and boost accuracy at 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88^ K (&&&8max_results8&&&).
The KV-cache is quantized to FP8 using per-layer scales obtained by max calibration over the LNPT-gpt-oss dataset of high-effort prompts. Each layer’s scale factor is rounded up to the nearest power of two. The stated rationale is to trade negligible rounding noise for the simplicity of bit-shifts while mitigating underflow (&&&8max_results8&&&).
Finally, an RL fine-tuning phase is applied with router and experts frozen, over three reasoning environments: math, code, and general. The reward combines task correctness and length regularization. Two variants are trained: “High-only,” which yields peak accuracy but inflates verbosity, and “Balanced,” which uses a uniform mix of high-, medium-, and low-effort episodes. The final policy is a weight average of these checkpoints and is reported to match the high-effort accuracy while retaining medium verbosity and restoring the effort-length ratio to within 8gpt-oss-puzzle-88B OR \8query8% of the teacher (&&&8max_results8&&&).
8sort_by8. Efficiency metrics, throughput, and accuracy retention
The model’s performance analysis explicitly distinguishes per-token speed from request-level efficiency. Per-token throughput is defined as
PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8query8^
and latency per token as
PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \8\8^
On an 8×H8gpt-oss-puzzle-88B OR \8query8query8^ node, the reported throughput rises from PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \88^ to PRESERVED_PLACEHOLDER_8gpt-oss-puzzle-88B OR \89 in the PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8query8^ setting, for a 8gpt-oss-puzzle-88B OR \8.8 OR MultiZebraLogic OR AsymPuzl8 OR MultiZebraLogic OR AsymPuzl8× speedup, and from PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8gpt-oss-puzzle-88B OR \8^ to PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8 OR MultiZebraLogic OR AsymPuzl8^ in the PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8max_results8^ setting, for a 8gpt-oss-puzzle-88B OR \8.8query8max_results8× speedup (&&&8max_results8&&&). On a single NVIDIA H8gpt-oss-puzzle-88B OR \8query8query8^ GPU, the reported gains are 8 OR MultiZebraLogic OR AsymPuzl8.8sort_by8sort_by8× in the PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8sort_by8^ scenario and 8 OR MultiZebraLogic OR AsymPuzl8.88 OR MultiZebraLogic OR AsymPuzl8× in the PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8relevance8^ scenario (&&&8max_results8&&&).
The paper stresses that token counts vary with reasoning effort and model variant, so tok/s and ms/token do not necessarily imply end-to-end speedups. A 8 OR MultiZebraLogic OR AsymPuzl8× throughput gain can be erased if traces grow 8 OR MultiZebraLogic OR AsymPuzl8×, and throughput gains can instead be spent on additional reasoning tokens to improve accuracy (&&&8max_results8&&&). This is an important corrective to simplistic speed comparisons.
To address that issue, the request rate is defined as
PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8query8^
where PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl8\8^ is the average tokens generated per request. Plotting average reasoning accuracy against relative request rate across low-, medium-, and high-effort settings yields an accuracy–speed frontier. Along that frontier, gpt-oss-puzzle-88B is reported to dominate gpt-oss-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B: at high effort on 8×H8gpt-oss-puzzle-88B OR \8query8query8^ it delivers 8 OR MultiZebraLogic OR AsymPuzl8.8query88× higher normalized request rate while retaining 8gpt-oss-puzzle-88B OR \8query8query8.8% of accuracy, and at low effort it achieves 8gpt-oss-puzzle-88B OR \8.8 OR MultiZebraLogic OR AsymPuzl89× higher request rate with 8gpt-oss-puzzle-88B OR \8query88.8 OR MultiZebraLogic OR AsymPuzl8% accuracy retention (&&&8max_results8&&&).
Accuracy retention is summarized across eight reasoning benchmarks—MMLU-Pro, GPQA, HLE, AALCR, AIME-8 OR MultiZebraLogic OR AsymPuzl8relevance8, SciCode, IFBench, and RULER-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88K—and three effort levels. The fully quantized student achieves suite-average retention of 8gpt-oss-puzzle-88B OR \8query8query8.8% at high effort, 8gpt-oss-puzzle-88B OR \8query8max_results8.9% at medium effort, and 8gpt-oss-puzzle-88B OR \8query88.8 OR MultiZebraLogic OR AsymPuzl8% at low effort (&&&8max_results8&&&). The source interprets this as showing no quality sacrifice and small gains at medium and low effort.
8relevance8. Relation to text-native puzzle solving
The earlier “unnatural language” paradigm provides a complementary view of what an 88 B open-source GPT puzzle model might do when applied directly to symbolic problem text rather than architecture search (&&&8gpt-oss-puzzle-88B OR \8&&&). In that framework, mazes, Rubik’s Cube states, and Sudoku grids are represented as raw ASCII or token strings. A 8relevance8×8relevance8^ maze is flattened into text, a scrambled cube is encoded as a 8relevance8sort_by8-character face string in URFDBL order followed by move symbols, and a Sudoku instance appears as a token sequence of the form [WP]8^^^^8gpt-oss-puzzle-88B OR \8^^^^-digit start [RESPONSE]8^^^^8gpt-oss-puzzle-88B OR \8^^^^-digit solution (&&&8gpt-oss-puzzle-88B OR \8&&&).
The model is then fine-tuned to translate unsolved token streams into solved ones without search trees, explicit constraint checks, or human-crafted heuristics. The training objective is standard maximum likelihood:
PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl88^
with no reinforcement-learning or policy-gradient terms and no explicit reward shaping. Self-attention remains standard,
PRESERVED_PLACEHOLDER_8 OR MultiZebraLogic OR AsymPuzl89
The only modification described is the introduction of special tokens such as <|startoftext|>, [WP], [RESPONSE], and <|endoftext|> into the byte-pair encoding vocabulary (&&&8gpt-oss-puzzle-88B OR \8&&&).
The source presents concrete baseline results for smaller GPT-8 OR MultiZebraLogic OR AsymPuzl8-scale models. On Rubik’s Cube, using a 8query8query8gpt-oss-puzzle-88B OR \8-example test set, 8relevance8\8query8^ outputs are valid but incorrect formulas, 8gpt-oss-puzzle-88B OR \8gpt-oss-puzzle-88B OR \8^ are invalid outputs, and 8gpt-oss-puzzle-88B OR \8sort_by8^ are fully solved. On a random sample of 8gpt-oss-puzzle-88B OR \8query8,8query8query8query8^ Sudoku instances, fully solved grids remain at 8query8%, with many near-complete solutions but frequent row, column, or sub-block errors. On 8sort_by8×8sort_by8^ and 8relevance8×8relevance8^ mazes, valid maze–solution pairings exceed 98%, and mean solution length is within 8gpt-oss-puzzle-88B OR \8^ step of the optimal breadth-first result (&&&8gpt-oss-puzzle-88B OR \8&&&).
Within that framework, the 88 B “gpt-oss-puzzle-88B” is presented as a scaling hypothesis rather than as the already optimized MoE derivative of the later work. The proposed gains include increasing context length beyond 8 OR MultiZebraLogic OR AsymPuzl8,8query8sort_by88^ tokens, exploiting greater capacity to memorize rare long-distance patterns, introducing lightweight auxiliary heads or constrained decoding to enforce legality, and leveraging in-context learning for new puzzle variants (&&&8gpt-oss-puzzle-88B OR \8&&&). This suggests a second research trajectory for the same model name: not only a cheaper reasoning model, but also a larger platform for search-free combinatorial problem solving in plain text.
8query8. Benchmarking environments: zebra logic and asymmetric cooperation
Two later benchmark frameworks give formally specified evaluation environments for an 88 B open-source GPT-style model, but they should not be conflated with published results for gpt-oss-puzzle-88B itself. MultiZebraLogic is a multilingual logical-reasoning benchmark built from zebra puzzles in nine Germanic languages, with datasets of 8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl88^ and 8gpt-oss-puzzle-88B OR \8query8 OR MultiZebraLogic OR AsymPuzl8sort_by8^ puzzles for sizes 8 OR MultiZebraLogic OR AsymPuzl8×8max_results8^ and 8sort_by8×8relevance8^ (&&&8query8&&&). The benchmark varies language, theme, puzzle size, 8gpt-oss-puzzle-88B OR \8sort_by8^ real clue types, and up to 8 red-herring types. Its generation loop samples candidate real clues from the ground-truth solution, keeps only those that preserve a unique solution under python-constraint, then adds uninformative red herrings and shuffles all clues into final puzzle text (&&&8query8&&&).
The formal encoding uses Boolean variables PRESERVED_PLACEHOLDER_8max_results8query8, uniqueness constraints
PRESERVED_PLACEHOLDER_8max_results8gpt-oss-puzzle-88B OR \8^ and PRESERVED_PLACEHOLDER_8max_results8 OR MultiZebraLogic OR AsymPuzl8, together with positional variables PRESERVED_PLACEHOLDER_8max_results8max_results8^ and symbolic encodings for clue types such as found_at, not_at, same_object, next_to, and between (&&&8query8&&&). Evaluation is defined by puzzle-level accuracy, where the entire solution matrix must match the reference, and cell-wise accuracy, defined as the number of correct cells divided by PRESERVED_PLACEHOLDER_8max_results8sort_by8^ (&&&8query8&&&). The benchmark reports that adding 8relevance8^ red herrings to 8sort_by8×8relevance8^ puzzles drops PRESERVED_PLACEHOLDER_8max_results8relevance8^ by PRESERVED_PLACEHOLDER_8max_results8query8, and it states that scores of o8max_results8-mini on 8sort_by8×8relevance8^ puzzles are not significantly affected by English versus Danish or by the common houses theme versus the country-specific smørrebrød theme (&&&8query8&&&). Translation guidance prioritizes correctness, unambiguity, naturalness, ease of generation, consistency, and diversity, and extension paths include non-unique attributes, super-attributes, ordinal comparisons, and probabilistic “half-herring” clues (&&&8query8&&&).
AsymPuzl supplies a different kind of controlled evaluation: a two-agent asymmetric puzzle environment for studying communication under information asymmetry (&&&8 OR MultiZebraLogic OR AsymPuzl8&&&). A puzzle state is a sequence PRESERVED_PLACEHOLDER_8max_results8\8^ over distinct shapes and colors, with each shape and each color appearing exactly once. Alice observes the ordered list of shapes with colors hidden, while Bob observes the unordered set of shape-color pairs with positions hidden. Each maintains a private working hypothesis updated through replace(position=i, by=(shape, color)) actions, and communication proceeds in strings from the message space PRESERVED_PLACEHOLDER_8max_results88^ until either convergence or a maximum of PRESERVED_PLACEHOLDER_8max_results89 turns (&&&8 OR MultiZebraLogic OR AsymPuzl8&&&).
The evaluation metrics are success rate, average turns to solution, and average actions per position, all reported over PRESERVED_PLACEHOLDER_8sort_by8query8^ seeds. Feedback modes include no feedback, own, own detailed, joint, both, and both detailed. Under the “Both” feedback mode on 8relevance8-piece puzzles, OSS-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B reaches PRESERVED_PLACEHOLDER_8sort_by8gpt-oss-puzzle-88B OR \8^ success with Wilson confidence interval PRESERVED_PLACEHOLDER_8sort_by8 OR MultiZebraLogic OR AsymPuzl8, while Llama 8max_results8.8 OR MultiZebraLogic OR AsymPuzl8-8gpt-oss-puzzle-88B OR \8gpt-oss-puzzle-88B OR \8B records PRESERVED_PLACEHOLDER_8sort_by8max_results8. OSS-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B without feedback solves only 8relevance8max_results8.8max_results8 but “Own detailed” feedback raises it to 8gpt-oss-puzzle-88B OR \8query8query8%, and “Both detailed” yields 98query8.8\8 Communication analysis reports approximately 8 OR MultiZebraLogic OR AsymPuzl8.8max_results8^ edits per position for OSS-8gpt-oss-puzzle-88B OR \8 OR MultiZebraLogic OR AsymPuzl8query8B on PRESERVED_PLACEHOLDER_8sort_by8sort_by8, against more than 8sort_by8^ edits per position for Llama 8gpt-oss-puzzle-88B OR \8gpt-oss-puzzle-88B OR \8B, with failure modes including ignoring partner messages, over-correction, and miscommunication under “Both detailed” feedback (&&&8 OR MultiZebraLogic OR AsymPuzl8&&&). The source recommends that an 88 B-parameter GPT-style model be evaluated under the same suite of feedback modes and puzzle sizes, with attention to success rate, edits per position, and turn-wise convergence curves (&&&8 OR MultiZebraLogic OR AsymPuzl8&&&).
Taken together, these frameworks show how gpt-oss-puzzle-88B can be situated within a broader empirical ecosystem. MultiZebraLogic probes multilingual logical deduction with formally encoded constraints and controllable distractors, while AsymPuzl probes cooperative communication under asymmetric observation. In that sense, the model’s significance lies not only in its post-training efficiency gains, but also in the range of structured reasoning regimes to which an 88 B open-source GPT architecture can be systematically subjected.