Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation

Abstract: LLM-based generative list-wise recommendation has advanced rapidly, but decoding remains sequential and thus latency-prone. To accelerate inference without changing the target distribution, speculative decoding (SD) uses a small draft model to propose several next tokens at once and a target LLM to verify and accept the longest prefix, skipping multiple steps per round. In generative recommendation, however, each item is represented by multiple semantic-ID tokens, often with separators, and current drafts typically treat these tokens uniformly. This overlooks two practical facts: (i) a token's semantics depend on its within-item slot, and (ii) uncertainty tends to increase with speculation depth. Without modeling these effects, SD's speedups can be limited. We introduce PAD-Rec, Position-Aware Drafting for generative Recommendation, a lightweight module that augments the draft model with two complementary signals. Item position embeddings explicitly encode the within-item slot of each token, strengthening structural awareness. Step position embeddings encode the draft step, allowing the model to adapt to depth-dependent uncertainty and improve proposal quality. To harmonize these signals with base features, we add simple gates: a learnable coefficient for item slots and a context-driven gate for draft steps. The module is trainable, easy to integrate with standard draft models, and adds negligible inference overhead. Extensive experiments on four real-world datasets show up to 3.1x wall-clock speedup and about 5% average wall-clock speedup gain over strong SD baselines, while largely preserving recommendation quality.

• The paper introduces PAD-Rec, a module that enhances inference speed in LLM-based listwise recommendation by integrating item position embeddings and step position embeddings.
• It employs lightweight gating mechanisms to fuse slot-aware semantics with decoding tokens, addressing latency issues in sequential auto-regressive decoding.
• Experiments on real-world datasets show up to 3.1× wall-clock speedup with negligible impact on recommendation quality metrics like Recall@10 and NDCG@10.

Position-Aware Drafting for Efficient Inference in LLM-Based List-Wise Recommendation

Introduction and Motivation

The paper "Position-Aware Drafting for Inference Acceleration in LLM-Based Generative List-Wise Recommendation" (2604.27747) addresses the critical latency bottleneck in transformer-based generative recommendation systems. As LLMs are increasingly deployed to generate entire recommendation lists via sequence modeling over tokenized item IDs, sequential auto-regressive decoding results in high latency, making real-time deployment challenging. While speculative decoding (SD) frameworks have shown promise for lossless acceleration by employing fast draft models to propose multiple token continuations at once, their naivety in handling the structured nature of item representations common in recommendation (e.g., tuples of semantic-ID tokens per item) and growing uncertainty with draft depth limits their efficacy.

The authors systematically identify two central challenges for SD in generative recommendation: (i) underutilization of within-item token slot information, and (ii) lack of explicit modeling for uncertainty that increases with draft depth. These gaps motivate the proposal of PAD-Rec, a position-aware drafting module that augments any draft model with item slot and speculative depth context, governed by lightweight gating mechanisms for effective fusion with basic token and contextual features.

Figure 1: Schematic of multi-step HASS-style drafting in list-wise generation and the core challenges—item-structured tokenization and depth-dependent uncertainty.

Methodology: Position-Aware Drafting with Lightweight Gating

PAD-Rec enhances speculative decoding with two complementary positional augmentations:

• Item Position Embeddings (IPE): Each token in the flat decoding stream is tagged by its role within the item (e.g., its position in a semantic ID tuple), retrieved via a learned embedding. This injects explicit slot-aware semantics, strengthening intra-item modeling that is crucial for high alignment in structured output regimes.
• Step Position Embeddings (SPE): Each prediction step within the speculative draft is also embedded according to its draft depth, enabling the draft model to develop depth-aware behaviors and adapt its proposal confidence adaptively as uncertainty increases with position.

Both embeddings are integrated via customized gates:

• A global, learnable scalar modulates IPE strength, preventing slot information from overwhelming token semantics.
• A context-sensitive, step-aware gate based on learned parameters allows flexible conditioning on SPE, especially critical as proposal uncertainty grows with depth.

This position-aware drafting paradigm is compatible with existing SD frameworks (notably HASS and EAGLE), introduces negligible computational or parameter overhead, and is straightforward to integrate.

Figure 2: The PAD-Rec training pipeline showing how draft input is augmented with IPE and SPE, driving richer and more precise proposal contexts.

Figure 3: During unrolled multi-step training, draft representations are progressively shifted from target to draft features, with positional embeddings ensuring explicit modeling of slot and depth context.

Experimental Evaluation

Extensive benchmarking is conducted on four real-world datasets (Amazon Beauty, Instruments, Games, Yelp). The key efficiency and recommendation-quality metrics include wall-clock speedup, average acceptance length ($\tau$), Recall@10, and NDCG@10.

Key Results:

• PAD-Rec achieves up to 3.1× wall-clock speedup, outperforming strong baselines including HASS, EAGLE-2, FSPAD, and GRIFFIN.
• It delivers an average 5% speedup gain over the best SD baselines—a strong improvement given all models preserve the exact decoding distribution.
• PAD-Rec maintains near-identical recommendation quality compared to the target LLM, with negligible changes in Recall@10 and NDCG@10 across both deterministic (temp=0) and stochastic (temp=0.5) sampling regimes.
• The method attains a higher or nearly highest acceptance length (2–7 tokens on average across settings), directly correlating with practical acceleration.

  Figure 4: Ablation analysis on the contribution of IPE and SPE to speedup and acceptance length across representative datasets.

  

  Figure 5: Gate ablation reveals the major effect of the item gate, with the step gate providing additional gains in depth-stochasticity adaptation.

  

  Figure 6: Speedup and acceptance length as a function of speculative draft depth, highlighting that maximal acceleration is reached at moderate depths (typically $B_{\text{test}}\approx4{-}6$).

  

  Figure 7: Model size scaling results show that speculative decoding efficiency increases with LLM scale, with PAD-Rec widening the gap over HASS as backbone size increases.

Theoretical and Practical Implications

PAD-Rec provides a tailored architectural solution to the structured output and depth-uncertainty issues in SD for recommendation. The explicit position embedding design injects strong inductive bias and maintains compatibility with high-throughput, tree-based speculative decoding, without departing from the original target LLM or verification semantics.

The findings demonstrate that even subtle structural priors, when modulated by lightweight gating mechanisms, substantially increase the effectiveness of speculative inference in LLM-based generation, particularly in domains with well-defined output schemas (e.g., tuple-encoded items).

Practical deployment implications include:

• Significant reduction in real-world inference latency for LLM-based recommender systems, making large-scale real-time generation more tractable.
• Maintaining output fidelity, allowing acceleration without distributional shift, which is essential for user-facing applications.
• Generalizability: The approach is agnostic to model scale, backbone architecture, SD variant, and applies to any structured sequence task reliant on semantic tuples or similar output constraints.

Limitations and Future Directions

Areas for further exploration include:

• Extending position-aware drafting to deeper and wider speculative graphs, borrowing techniques from tree-based or retrieval-augmented SD.
• Exploring dynamic or learned gating functions that further adapt per-context or per-user, potentially leveraging reinforcement or meta-learning to optimize for longer accepted prefixes under stricter latency constraints.
• Investigating the interplay between position-aware speculative decoding and advanced item tokenizers or hybrid generative-retrieval recommenders, particularly as the space of semantic IDs grows.
• Quantitative assessment of robustness under extreme stochastic generation and in the presence of distributional drift in user preferences or output schemas.

Conclusion

PAD-Rec advances the state of the art in inference acceleration for LLM-based generative recommendation by injecting explicit, learnable structure- and depth-aware signals into the speculative decoding pipeline. Its demonstrated improvements in both speed and output fidelity, combined with ease of integration and low computational overhead, underscore the importance of incorporating task-driven inductive biases for efficient LLM deployment in structured generation applications (2604.27747).