Verify dynamic early-exit benefits

Determine whether dynamic early exit, in which individual tokens exit at different layers according to a heuristic, router, or auxiliary model, yields better accuracy-throughput trade-offs for models pretrained with layer dropout than static early exit.

Background

The paper demonstrates benefits for static early exit, where all tokens terminate at the same layer, and hypothesizes that token-dependent dynamic early exit could improve the accuracy-throughput trade-off. The proposed dynamic mechanisms are not evaluated.

The unresolved problem is therefore to test whether layer-dropout pretraining provides a usable advantage for dynamic token-level exit policies and to quantify that advantage relative to static exit.

References

We hypothesize that ``dynamic early exit'' where each token exits at a different layer based on a heurestic, router, or an auxiliary model (e.g., ~\citet{CALM}), will lead to better accuracy-throughput tradeoffs on a model pretrained with layer dropout. However, we leave verifying this hypothesis for future work.

— Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference  (2609.05275 - Elhoushi et al., 4 Sep 2026) in Section 6, Subsection “Early Exit”