LLM-DMsRec: Hybrid Session Recommender

• LLM-DMsRec is a session recommendation framework that fuses a pre-trained GNN’s structural modeling with LLM-inferred multi-semantic intent from session texts.
• The method separates explicit and latent intents from candidate items and aligns them with the GNN’s session representation using a KL-divergence loss.
• Empirical results on datasets like Amazon Beauty and MovieLens-1M show significant gains in MRR and Precision, validating the hybrid semantic augmentation approach.

LLM-DMsRec is a session-based recommendation framework that augments a pre-trained graph neural network (GNN) backbone with LLM-derived semantic intent. It is defined for anonymous next-item prediction, where a session is a short interaction sequence and the objective is to predict the next clicked item from the current session. The method uses a pre-trained GNN to produce a candidate set, prompts an LLM to infer multiple semantic intents from the session text and candidate items, separates these intents into explicit and latent components, and aligns them with the GNN’s structural session representation through a KL-divergence-based auxiliary loss. In this formulation, LLM-DMsRec is neither a pure LLM recommender nor a pure GNN recommender; it is a hybrid architecture designed to bridge graph-structured transition modeling and language-based semantic reasoning (Zhang et al., 27 Jul 2025).

1. Problem formulation and conceptual motivation

LLM-DMsRec is formulated in the standard session-based recommendation setting. Let $V=\{v_1,v_2,\dots,v_n\}$ be the item set and $S=\{s_1,s_2,\dots,s_m\}$ the session set. A session at time $t$ is $s_t=(v_1,v_2,\dots,v_l)$, and the task is to predict the next item $v_{l+1}$. The model objective is written as

$$\hat{y}=\arg\underset{y \in \mathcal{Y}}{\max} P(y \mid s_t; \Theta) =\arg\underset{y \in \mathcal{Y}}{\max} P(s_t; y; H; \Theta),$$

where $\mathcal{Y}$ is the candidate item set, $H$ is the user intention embedding matrix, and $\Theta$ are model parameters (Zhang et al., 27 Jul 2025).

The framework is motivated by three limitations attributed to prevailing GNN-based session recommenders. First, such models rely heavily on item IDs and transition structure, while under-utilizing textual semantics. Second, they typically compress a session into a single structural representation and therefore do not explicitly disentangle multi-faceted user intent. Third, they do not directly exploit an LLM’s ability to infer plausible goals from titles and candidate items. The paper uses “multi-semantic intent” to denote multiple semantic facets of a session, including both explicit intent, directly reflected by clicked items, and latent intent, inferred from context and candidate items. A representative example is a session such as “iPhone 16 → AirPods Pro → AirPods Max → Watch SE,” which can imply both “buy headphones” and “buy phone-connected devices” (Zhang et al., 27 Jul 2025).

A common misconception is that LLM-DMsRec replaces GNNs with an LLM. The architecture does not do so. The GNN remains the structural backbone, while the LLM is used to infer semantic intent over a constrained candidate set. Another common misconception is that it is an end-to-end fine-tuned LLM recommender. In fact, the LLM is used offline and remains frozen; the online recommendation stack remains close to the computational profile of the underlying GNN model (Zhang et al., 27 Jul 2025).

2. Three-phase architecture

The framework is organized into three phases: candidate item selection, LLM-based multi-semantic intent extraction, and intent alignment with joint training. The first phase uses any pre-trained GNN-based session recommender, such as SR-GNN, GCE-GNN, DHCN, SHARE, or MSGIFSR, as a retrieval model. Given a session $s_t$, the pre-trained GNN returns the top-$S=\{s_1,s_2,\dots,s_m\}$0 items

$S=\{s_1,s_2,\dots,s_m\}$1

which are then mapped to titles

$S=\{s_1,s_2,\dots,s_m\}$2

This candidate set functions both as a recommendation shortlist and as a knowledge base constraining the LLM’s reasoning (Zhang et al., 27 Jul 2025).

In the second phase, the model constructs two prompts. An instruction prompt tells the LLM to infer user intents from the session and to list multiple intents if appropriate, separated by semicolons, with the additional constraint that intents must be selected from the candidate item set. The input prompt supplies the chronological session titles and the candidate titles. The LLM used in the reported experiments is Qwen2.5-7B-Instruct. It produces a set of intent outputs

$S=\{s_1,s_2,\dots,s_m\}$3

where $S=\{s_1,s_2,\dots,s_m\}$4 is the textual session sequence and $S=\{s_1,s_2,\dots,s_m\}$5 is the $S=\{s_1,s_2,\dots,s_m\}$6-th inferred intent (Zhang et al., 27 Jul 2025).

Each intent is then categorized as explicit or latent through membership in the session text: $S=\{s_1,s_2,\dots,s_m\}$7 The explicit intents $S=\{s_1,s_2,\dots,s_m\}$8 and latent intents $S=\{s_1,s_2,\dots,s_m\}$9 are encoded by a frozen BERT encoder: $t$0

$t$1

Average pooling yields global semantic summaries

$t$2

These two vectors are the explicit and latent semantic intent features injected into the GNN side of the model (Zhang et al., 27 Jul 2025).

3. Structural–semantic alignment mechanism

The central technical contribution of LLM-DMsRec is an alignment mechanism between GNN-derived structural intent and LLM-derived semantic intent. The GNN backbone computes a structural session representation

$t$3

which the paper assigns the role of structural intent. The semantic vectors are linearly projected into the same space: $t$4 The final session representation is then formed through concatenation and projection: $t$5 This fused representation combines graph-structured transition evidence with both observed and inferred semantic intent (Zhang et al., 27 Jul 2025).

Alignment is imposed at the distribution level. First, softmax is applied to each representation to form distributions: $t$6 The explicit and latent alignment losses are

$t$7

$t$8

They are combined as

$t$9

This design makes explicit and latent semantic intent separately align to the GNN’s structural representation rather than being merged into a single undifferentiated text feature (Zhang et al., 27 Jul 2025).

Prediction uses the fused session vector $s_t=(v_1,v_2,\dots,v_l)$0 and item embeddings $s_t=(v_1,v_2,\dots,v_l)$1: $s_t=(v_1,v_2,\dots,v_l)$2 The recommendation loss is cross-entropy,

$s_t=(v_1,v_2,\dots,v_l)$3

and the final objective is

$s_t=(v_1,v_2,\dots,v_l)$4

A plausible implication is that the framework treats semantic intent as a regularizer on structural session modeling rather than as a replacement for it (Zhang et al., 27 Jul 2025).

4. Training procedure, inference path, and efficiency profile

Training begins with a standard pre-training stage for the GNN backbone. That backbone is then used offline to generate the top-$s_t=(v_1,v_2,\dots,v_l)$5 candidate item set for each session; the paper fixes $s_t=(v_1,v_2,\dots,v_l)$6. The LLM is next run offline on prompt pairs consisting of session titles and candidate titles to infer multiple intents. These inferred intents are split into explicit and latent subsets and encoded by a frozen BERT model. During joint training, batches of sessions are paired with their precomputed semantic intent embeddings, the GNN computes structural intent, the model evaluates $s_t=(v_1,v_2,\dots,v_l)$7 and $s_t=(v_1,v_2,\dots,v_l)$8, and parameters are updated with Adam (Zhang et al., 27 Jul 2025).

This training strategy has a direct systems implication: LLM-DMsRec avoids runtime dependence on heavy LLM inference. Qwen2.5-7B-Instruct and BERT are not fine-tuned; they are used as frozen feature extractors. The online path therefore resembles the underlying GNN backbone plus additional linear projections and an auxiliary representation branch. The paper explicitly notes that this design keeps online scoring lightweight and makes the framework practically integrable into existing GNN pipelines (Zhang et al., 27 Jul 2025).

The architecture also clarifies what the framework is not. It is not a generative recommendation model that directly decodes items at serving time. It is not a full end-to-end LLM training pipeline. It is a hybrid, offline-augmented session recommender in which the candidate set both restricts the LLM’s semantic search space and mitigates hallucination by forcing intent inference over a controlled shortlist (Zhang et al., 27 Jul 2025).

5. Empirical evaluation and ablation results

The reported evaluation uses two datasets: Amazon Beauty and MovieLens-1M. After preprocessing, Beauty contains 158,717 training sessions, 17,422 test sessions, 198,502 clicks, 12,101 items, and average session length 8.66. ML-1M contains 69,906 training sessions, 7,768 test sessions, 999,611 clicks, 3,416 items, and average session length 12.79. The metrics are Precision@K and Mean Reciprocal Rank@K (Zhang et al., 27 Jul 2025).

Across a range of GNN backbones, LLM-DMsRec consistently improves recommendation quality, with particularly large gains on MRR@K. On top of GCE-GNN, the framework yields the following representative changes:

Setting	Baseline	LLM-DMsRec
Beauty, MRR@5	4.3143	6.1504
ML-1M, MRR@5	1.9797	3.2731
ML-1M, P@5	4.6663	5.7702

For GCE-GNN on Beauty, this corresponds to $s_t=(v_1,v_2,\dots,v_l)$9 on MRR@5, a relative gain of $v_{l+1}$0. For GCE-GNN on ML-1M, MRR@5 changes from $v_{l+1}$1, a relative gain of $v_{l+1}$2, while P@5 changes from $v_{l+1}$3, a relative gain of $v_{l+1}$4. The same trend appears on SR-GNN and MSGIFSR. Even when the backbone is already strong, as with MSGIFSR, the framework still improves both Precision@K and MRR@K on Beauty and ML-1M. The main negative result is that P@20 can slightly decrease for some backbones; the authors attribute this to LLM-induced noisy items becoming more visible in longer recommendation lists (Zhang et al., 27 Jul 2025).

The ablation studies isolate three major effects. Removing the semantic module reduces performance to the original GNN level. Keeping semantic features but removing KL alignment remains better than the baseline, but underperforms the full model. Removing either explicit intent or latent intent also degrades results relative to the full architecture, indicating that the two semantic branches are complementary. In one Beauty ablation with GCE-GNN, P@5 moves from 8.8681 without semantics, to 9.069 without KL, to 9.6659 without latent intent, to 9.4937 without explicit intent, and to 9.93 in the full model. This pattern supports the interpretation that semantic augmentation alone is useful, but aligned dual-intent augmentation is strongest (Zhang et al., 27 Jul 2025).

Sensitivity analyses further show that the alignment weight $v_{l+1}$5 improves performance when increased from 0 to a moderate value, peaking around $v_{l+1}$6, and then harms performance when too large. The explicit and latent weights $v_{l+1}$7 and $v_{l+1}$8 also matter: the paper reports that on ML-1M, MRR@20 tends to improve with larger explicit weight and smaller latent weight, while both branches remain beneficial overall (Zhang et al., 27 Jul 2025).

6. Position within LLM-based recommendation research

Within the broader LLM-for-recommendation literature, LLM-DMsRec occupies a distinct systems position. It does not optimize ranking by directly fine-tuning an LLM on pairwise outputs, as in Direct Multi-Preference Optimization, which frames personalized ranking as pairwise sequence comparison and optimizes the probability gap between preferred and dispreferred completions (Bai et al., 2024). Instead, LLM-DMsRec keeps ranking inside a GNN-based session recommender and uses the LLM primarily as an intent inference module (Zhang et al., 27 Jul 2025).

It also differs from DELRec, which uses soft-prompt-based pattern distillation from conventional sequential recommenders into an LLM and then fine-tunes the LLM with AdaLoRA for next-item prediction (Zhang et al., 2024). DELRec turns the LLM into the main sequential recommender; LLM-DMsRec instead preserves the GNN as the main predictor and aligns semantic side information to it. Relative to DRDT, which treats sequential recommendation as a prompt-only retriever–reranker problem with dynamic reflection and divergent thinking and does not fine-tune the LLM (Wang et al., 2023), LLM-DMsRec is more explicitly hybrid: the GNN supplies candidate generation and structural intent, while the LLM supplies multi-semantic intent. Finally, it shares a design intuition with DLRREC, which argues that LLM-derived multi-modal features should be denoised and fused with collaborative signals inside the recommendation architecture rather than used as static features (Tian et al., 29 Nov 2025). This suggests a broader research pattern: LLM signals are increasingly being used not as standalone recommenders, but as structured semantic priors that must be aligned with collaborative or graph-based inductive biases.

7. Limitations and prospective directions

The framework’s limitations are explicit. It relies on the quality of the LLM’s understanding of item texts and domains; if textual coverage is weak or domain shift is severe, inferred intents may be noisy. Although the candidate set constrains output space, hallucinations are not eliminated, which the authors connect to the slight P@20 degradation observed in some settings. Prompt engineering is manual and task-specific, and the explicit–latent split is implemented with a simple string-matching rule based on whether an inferred intent appears in the session text. The LLM is integrated in a non-differentiable, offline manner, so gradients do not flow through semantic intent generation (Zhang et al., 27 Jul 2025).

These limitations imply a specific frontier for future work. The paper points to more effective methods for accurately inferring user intent with LLMs. A plausible implication is that stronger intent taxonomies, more robust alignment objectives, or differentiable coupling between semantic inference and recommendation loss could further improve stability. Another plausible implication is that the explicit–latent distinction could be generalized beyond literal title matching to richer semantic equivalence criteria. As presented, however, LLM-DMsRec establishes a clear hybrid recipe: use a pre-trained GNN for structural session modeling and candidate selection, use an LLM to infer multiple semantic intents over a constrained candidate set, separate explicit and latent intent, and align both to the structural representation before prediction (Zhang et al., 27 Jul 2025).