Joint design and trade-off space for composing SSD with EAGLE and token-tree speculation

Investigate the joint design and performance trade-offs of integrating Speculative Speculative Decoding (SSD) with EAGLE-style draft models and token-tree speculative decoding methods. Specifically, determine how to coordinate verification-outcome prediction (fan-out allocation), cache-aware sampling for residual control, and fallback strategies when SSD is composed with EAGLE and tree-based speculation so as to maximize end-to-end speedups while preserving the lossless correctness guarantees across batch sizes and sampling temperatures.

Background

The paper introduces Speculative Speculative Decoding (SSD), which parallelizes drafting and verification by precomputing speculations for likely verification outcomes while the target model verifies the prior round. Saguaro is an optimized SSD algorithm addressing prediction of verification outcomes, balancing acceptance vs. cache-hit rates via a novel sampling scheme, and handling cache misses.

Prior work such as EAGLE and tree-based speculative decoding methods (e.g., token trees) improve acceptance or offer multiple candidate paths but operate sequentially (speculate then verify). The authors note SSD can be combined with these approaches, but the optimal combined design—spanning cache construction, sampling, and fallback—has not been characterized. This motivates a systematic study of the joint design and trade-offs when composing SSD with EAGLE and token-tree speculation.

References

Much remains open. SSD composes naturally with EAGLE and token-tree speculation (Appendix\ref{app:combine}); the joint design and tradeoff space is largely unexplored.

— Speculative Speculative Decoding  (2603.03251 - Kumar et al., 3 Mar 2026) in Conclusion and Limitations, Section 6

A promising direction is therefore to train drafter models that support longer draft blocks and investigate whether DARTree can translate the expanded drafting horizon into higher acceptance and end-to-end speedups.

— DARTree: Speculative Diffusion Decoding with Autoregressive Draft Trees  (2608.13524 - Li et al., 13 Aug 2026) in Appendix F, Future Work

Extending to temperature-based sampling would require the rejection-sampling correction of~\citet{leviathan2023fast} and~\citet{chen2023accelerating}, which adjusts the target distribution conditioned on draft probabilities; the mask-based rewind composes with this correction in principle (the cache state is the same either way), but we have not validated it empirically.

— Pre-Compiled Pipeline Shards for Distributed LLM Inference on Intel AI PC Fleets  (2608.19147 - Berenbaum et al., 19 Aug 2026) in Section 4.2, “Mask-Based Rewind”