DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding

Abstract: Speculative decoding accelerates LLM inference by drafting several tokens and verifying them in parallel. Block-diffusion drafters such as DFlash produce a draft block in one pass but model only per-position marginals; best-first tree methods such as DDTree expand candidate trees from those marginals. The released Domino drafter adds a GRU-based causal correction that makes each draft token's distribution path-dependent, a structure DDTree's factorized formulation cannot represent. We introduce DominoTree, a training-free best-first draft tree scored by Domino's conditional, non-factorized correction along each root-to-node path, made practical by restricting the per-node correction to a candidate top-M. On Qwen3-4B across eight benchmarks, DominoTree reaches up to 6.6x speedup over autoregressive decoding and the highest mean accept length of any evaluated method, up to 10.7 tokens per round, at every temperature we test. DominoTree constructs its tree with a GPU-native, CUDA-graph builder that is bit-identical to a reference Python implementation, so acceptance is unchanged, while keeping per-round tree construction cheap. With this builder as default, DominoTree wins throughput over the released Domino decoder at every temperature, 9-10% overall on Qwen3-4B and up to +22% on Alpaca, and over DDTree/CaDDTree at every temperature we test. On Qwen3- 8B, DominoTree keeps the highest accepted length at every temperature and adds a decisive throughput win at T=0, +24% over DDTree; at higher temperature that edge over DDTree/CaDDTree narrows to a tie and a small loss, while its Overall aggregate wins over DFlash and Domino persist.

• The paper demonstrates that path-conditional, training-free tree drafting using Domino's GRU correction significantly improves LLM inference throughput.
• The methodology overlays a best-first draft tree with per-node conditional scoring and top-M candidate selection to reduce cost and enhance acceptance quality.
• Empirical results show up to 6.6× speedup and increased accepted tokens across models, validating the approach on various datasets.

DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding

Problem Definition and Context

Accelerating LLM inference through speculative decoding hinges on maximizing accepted tokens per round while minimizing round cost. State-of-the-art drafters fall on a spectrum between parallel block-wise proposals with limited per-position marginal modeling (e.g., DFlash (Chen et al., 5 Feb 2026)) and serial autoregressive drafters with high-quality path-conditional proposals but high sequential cost. A recent advance, Domino (Huang et al., 28 May 2026), augments block-diffusion with a lightweight causal GRU-based correction, allowing partial conditioning on the realized token path at modest overhead. However, the Domino release explores only single-path chains per drafting round, foregoing branching that could boost acceptance.

Tree-based speculative drafting methods like DDTree (Ringel et al., 14 Apr 2026) and CaDDTree (Zhang et al., 1 Jun 2026) have attempted to exploit block drafters' parallelism via best-first prefix trees, but their factorized scoring cannot represent Domino's path-conditioned GRU correction. This paper proposes DominoTree, a training-free draft-tree construction exploiting Domino's non-factorized conditional correction to simultaneously advance acceptance quality and throughput.

Methodology

DominoTree overlays a best-first draft tree onto the released Domino drafter checkpoint. Unlike previous factorized-marginal tree methods, each tree node’s score is the cumulative log-probability along the realized path, with Domino’s sequential GRU correction recomputed for every path. This enables conditional scoring over trees, where the score for extending a path incorporates the actual sequence of sampled tokens, not just the depth. To render per-node correction practical, the child set at each node is reduced to the top-$M$ marginal candidates, determined once per depth.

The implementation leverages a CUDA-graph-based builder for per-node correction, preserving the heap schedule of DDTree but substantially reducing Python-induced kernel-launch overhead. This approach ensures that the tree constructed is bit-identical to a reference Python implementation, thus guaranteeing the accept-cell statistics remain stable and any empirical gains are purely algorithmic.

Conditional accepted-length estimation, a criterion for CaDDTree’s adaptive budget, over-credits target acceptance in the corrected tree regime, saturating the budget and providing no gain. DominoTree therefore defaults to a fixed node budget, empirically ablated.

Figure 2: Comparison of DominoTree against DFlash, DDTree, CaDDTree, and Domino on Qwen3-4B ($T{=}0$), in terms of mean accept length (left) and speedup over AR decoding (right).

Empirical Results

On the Domino evaluation suite (Qwen3-4B and Qwen3-8B models, eight datasets spanning math, code, and chat), DominoTree consistently achieves the highest mean accepted token length $\tau$ at each tested temperature and outperforms all baselines—including the released Domino and both tree-based DFlash methods—on overall throughput. For Qwen3-4B ($T{=}0$), it attains up to $6.6\times$ speedup over AR decoding and up to 10.7 accepted tokens per round, producing aggregate throughput increases of 9–10% over Domino and 7–8% over DDTree/CaDDTree (95% paired-bootstrap CI).

These advantages persist across temperatures and model scales. The only consistent underperformance is in the code domain, where DDTree/CaDDTree’s acceptance length marginally exceeds DominoTree's; nevertheless, aggregate throughput remains competitive.

Substantial ablations isolate the throughput gain to DominoTree’s conditional, path-dependent scoring, as opposed to mere tree construction. Swapping DDTree’s marginal scorer into the same builder and verifier erases nearly all throughput advantage. The GPU-native builder is critical: it translates acceptance length gains into real wall-clock throughput, especially pronounced at the Qwen3-8B scale.

Analytical and Practical Implications

This research demonstrates that training-free, path-conditional draft trees can be constructed efficiently atop Domino’s partial-conditional corrector, leveraging modest per-node candidate restriction and graph-captured execution to minimize cost. While prior art such as JetSpec (Hu et al., 16 Jun 2026) shows similar gains with newly trained causal heads, DominoTree’s contribution is retaining these benefits without any additional training or parameters, relying solely on released weights.

From a practical standpoint, this method improves speculative decoding throughput, especially salient in low-acceptance domains (e.g., chat) where breadth is valuable. The introduced implementation is suitable as a research reference; production integration (e.g., batching, continual serving) awaits further engineering.

Figure 1: Calibration of CondAdaptive—predicted acceptance probability vs. empirical acceptance shows estimator over-creditting, especially severe on math, causing adaptive budgets to saturate.

A limitation is the miscalibration of Domino's corrected path-probabilities, undermining cost-aware adaptive budgeting. Future developments could revisit this with alternatives to GRU-based correction that admit better-calibrated acceptance models or more sophisticated tree-building heuristics.

Comparison to Related Work

DominoTree is situated at the intersection of block-diffusion speculative drafting (DFlash, Domino), tree-structured acceleration (DDTree, CaDDTree), and path-conditional tree drafting (JetSpec, EAGLE family). Unlike EAGLE's autoregressive trees, DominoTree's parallel backbone keeps expansion cost low; unlike JetSpec’s causal head, no new training is required.

Verification employs standard ancestor-masked attention, retaining losslessness up to floating-point ties—the same guarantees as in SpecInfer (Miao et al., 2023) and prior tree-based verifiers.

Conclusion

DominoTree establishes that path-conditional, training-free tree drafting atop Domino is both computationally viable and empirically advantageous, outperforming marginal-based and chain-only approaches in terms of both acceptance and throughput. The critical insight is that shared parallel backbones plus lightweight per-path correction suffice for effective tree-based speculative decoding without the cost of full autoregressive rescoring or training new heads.

The work leaves open adapting path-calibrated budgets and scaling toward high-parallel serving environments. As the field matures, integrating such conditional, flexible, and efficient tree drafting will underpin the deployment of ever-faster LLM inference stacks.