LBR: Towards Mitigating Length Bias in Large Language Models for Recommendation

Abstract: LLMs have recently emerged as powerful backbones for recommender systems by reformulating recommendation as a token-level generation task. Despite their promise, we identify a pervasive yet underexplored issue: $\textit{Length Bias}$. Because items are represented by textual descriptions of varying lengths, LLM-based recommenders can be systematically biased in two ways. On the input side, longer item descriptions occupy more tokens in the context and thus receive disproportionately large aggregate attention mass during user preference modeling. On the output side, decoding based on summed autoregressive log-likelihood score inherently disfavors long items. Worse still, conventional length normalization can introduce an additional bias and even degrade recommendation performance. To address this problem, we propose $\textbf{LBR}$ ($\textbf{L}$ength $\textbf{B}$ias $\textbf{R}$eduction), a lightweight and model-agnostic framework for mitigating length bias in LLM-based recommendation. LBR mitigates input-side bias via Length-Aware Attention Calibration, which incorporates a length-dependent offset into attention logits to neutralize attention skew. For the output side, LBR introduces Effective Information Length Normalization, replacing naive token count with an information-theoretic length surrogate derived from the branching structure of the prefix tree. Extensive experiments on three real-world Amazon datasets and two representative LLM-based recommenders demonstrate that LBR substantially alleviates length bias while consistently improving recommendation accuracy and fairness, with negligible additional training and inference overhead (with an average NDCG@5 gain of 16.82%). The code is available at https://github.com/Void-JackLee/LBR.

• The paper introduces a novel LBR framework that uses a length-dependent logit offset in self-attention to counteract input-side length bias in LLM-based recommendations.
• It replaces naive length normalization with an information-theoretic approach that aligns decoding scores with effective token information, ensuring fair treatment of longer items.
• Empirical evaluations on Amazon datasets show a 16.82% improvement in NDCG@5, highlighting enhanced accuracy and fairness with minimal computational cost.

Mitigating Length Bias in LLM-Based Recommendation: An Expert Analysis of LBR

Problem Formulation and Characterization of Length Bias

LLM-driven recommendation reformulates item retrieval as token-level sequence generation, leveraging Transformer-based attention mechanisms and constrained beam search within a Trie. This formulation introduces two distinct forms of length bias: (1) input-side bias, wherein longer item descriptions accumulate excessive attention mass in prompt context modeling; and (2) output-side bias, where autoregressive log-likelihood scoring disproportionately penalizes longer items, and naive length normalization exacerbates the problem due to heterogeneous token informativeness.

Figure 1: LLM-based recommendations and length bias—longer items increase attention accumulation, while decoding favors shorter items.

Figure 2: Trie-Constrained decoding restricts generated token options; valid-token set size varies by step.

Figure 3: Positive correlation between item token length and cumulative attention during prediction.

The paper rigorously quantifies these phenomena, showing that item length directly predicts cumulative attention mass Figure 3, and longer items receive lower likelihood scores during constrained decoding. Key empirical findings highlight that length normalization by raw token count fails: recommendation proportions across varying token lengths are skewed both with and without length penalty (Figure 4 and Figure 5), and tokens at nodes with low branching factors are disproportionately easy to predict Figure 6.

LBR Framework: Model-Agnostic Debiasing Approach

Length-Aware Attention Calibration (LAAC)

LBR introduces a length-dependent logit offset in the self-attention computation, counteracting the observed $g(l)$ correlation between item length and cumulative attention mass. The offset, parameterized as $\delta(l_v) = -\log(g(l_v))$, is jointly optimized with main model parameters, yielding invariance in attention weights across item lengths under mild assumptions (see theoretical proof in the appendix). This modification neutralizes the passive amplification effect of long items.

Figure 5: Attention weights with LBR exhibit near invariance to item length, effectively mitigating input-side length bias.

Effective Information Length Normalization (EILN)

On the decoding side, LBR replaces naive length normalization with an information-theoretic length surrogate, derived as Hartley entropy per token: $\mu_k = \log_2 |\mathcal{V}_k|$, summed across sequence steps to yield total uncertainty $U_y$. Decoding scores are normalized as $$s_{EIF}(y) = \frac{1}{U_y} \sum_{k=1}^{|y|} \mu_k \cdot \log P_\theta(y_k | y_{<k}, X)$$, reflecting average log-probability per bit of information—thus equating the scoring landscape across items with heterogeneous length and varying Trie branching structure.

Figure 7: Binary-decision tree view: each Trie branching factor $H$ represents $\log_2 H$ bits of decision depth, aligning item token selection with effective information length.

Empirical Evaluation and Ablation

LBR is extensively validated on three Amazon datasets, compared against conventional RS, LLM-enhanced RS, Transformer backbones (BIGRec, LLaRA), and prior debiasing strategies (D3, Reweight, IGD, CFT, etc.). Results show LBR achieves average NDCG@5 improvements of 16.82% over the best baseline, consistently elevating accuracy and fairness. The ablation study (Table: 'ablation study of LBR on NDCG@5') confirms that both LAAC and EILN are essential; removing either component markedly degrades performance.

Figure 8: LBR minimizes recommendation proportion variance across item-length groups, closely matching true distribution.

Figure 9: LBR achieves state-of-the-art accuracy with negligible computational overhead compared to counterfactual or reference-model-based methods.

Figure 10: Sensitivity analysis reveals conventional length normalization parameter $\alpha$ impairs accuracy as it increases, reinforcing LBR’s theoretical stance.

Practical and Theoretical Implications

LBR’s interventions are lightweight (additive offset, entropy-based normalization), model-agnostic, and computationally efficient ($O(N)$ complexity). These attributes facilitate deployment in large-scale sequential recommendation systems. Theoretically, LBR underscores the necessity of accounting for structural and information-theoretic properties of tokenized item representations, especially under constrained generation regimes (e.g., Trie-guided item decoding).

The empirical evidence regarding length bias—in both modeling and decoding stages—raises broader questions about fairness, compositional representation in generative RS, and the design of prompt structures in LLM-based recommendation. Addressing such systemic biases is critical for aligning model behavior with user intent and platform fairness.

Future Directions

The findings indicate promising avenues for extending information-theoretic normalization to other generative tasks beyond recommendation, where Trie or constrained decoding is prevalent (translation, code completion, etc.). Further research may explore dynamic prompt architectures, adaptive calibration strategies based on observed length distributions, and multi-objective optimization that incorporates fairness and interpretability constraints.

Conclusion

LBR systematically addresses length bias in LLM-based recommendation by introducing calibrated attention and information-aware decoding normalization. The framework delivers substantial quantitative improvements in accuracy and fairness, with minimal cost and robust generalization across backbones and datasets. This work advances understanding of the interaction between sequence modeling, constrained generation, and fairness, providing a foundation for future research into bias mitigation in large-scale generative recommender systems.