FlowRec: Flow Matching for Sequential Rec

• FlowRec is a sequential recommendation framework that models a user’s preference as a continuous trajectory from current state to future interest.
• It replaces diffusion-based methods with a behavior-based prior and a single-step alignment loss for robust and efficient embedding prediction.
• Empirical results demonstrate that FlowRec outperforms traditional baselines on various datasets while significantly reducing inference steps.

FlowRec is a sequential recommendation framework that uses flow matching to model how a user’s preference moves from a current latent state to a future next-item interest. In the formulation introduced in "Preference Trajectory Modeling via Flow Matching for Sequential Recommendation" (Li et al., 25 Aug 2025), the method addresses two limitations attributed to diffusion-based sequential recommenders: high sensitivity to a Gaussian prior and historical-condition quality, and computationally expensive multi-step inference. FlowRec replaces pure Gaussian noise with a personalized behavior-based prior, learns a continuous-time vector field over user-preference trajectories, and introduces a single-step alignment loss that incorporates both positive and negative samples. The resulting model is positioned as a generative recommender that is both more robust and more efficient than diffusion-based baselines (Li et al., 25 Aug 2025).

1. Problem setting and motivation

Sequential recommendation assumes that each user $u$ is associated with a chronological interaction sequence

$$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$

and the task is to predict the next item $i_n^u$ by solving

$$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$

FlowRec treats this task not as one-shot classification, but as modeling the distribution of the next-item representation conditioned on the historical sequence (Li et al., 25 Aug 2025).

The framework is motivated by recent diffusion-based sequential recommendation models such as DiffuRec, PreferDiff, DreamRec, and SeedRec. In the description accompanying FlowRec, these models typically start from pure Gaussian noise, corrupt target item embeddings with Gaussian noise in a forward process, and then iteratively denoise the representation in many inference steps. Two difficulties are emphasized. First, the Gaussian prior $\mathcal{N}(0,I)$ can be far from the true distribution of next-item embeddings, especially under sparse or noisy user histories. Second, inference requires many denoising iterations, often 30–50 steps, which increases latency and computational cost (Li et al., 25 Aug 2025).

FlowRec reframes the generative problem by modeling user preference as a continuous trajectory from a current preference state to the next-item embedding. This suggests a different inductive bias: rather than recovering future interest from uninformative noise, the model starts from a sequence-conditioned latent state that is already aligned with the user’s recent behavior.

2. Flow-matching formulation

The central object in FlowRec is a continuous-time mapping $\phi_t$ that transforms an initial state $x_0$ into a target state $x_1$, governed by the ordinary differential equation

$$\frac{d x_t}{dt} = v_\theta(x_t, t), \qquad x_0 \sim p_0, \qquad x_1 \sim p_1.$$

Here $v_\theta$ is a learned vector field. More generally, flow matching defines

$$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$0

with density evolution $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$1. In practice, FlowRec uses conditional flow matching (CFM) and learns a neural approximation $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$2 to the conditional vector field (Li et al., 25 Aug 2025).

A defining design choice is the replacement of the Gaussian prior by a behavior-based prior. Let each item $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$3 have embedding $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$4, and let the user’s sequence embedding be

$$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$5

where $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$6. FlowRec uses the last token representation as the initial state,

$$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$7

while the target state is the next-item embedding,

$$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$8

The description identifies $$\mathcal{S}^u = [i_1^u, i_2^u, \ldots, i_{n-1}^u], \qquad i_j^u \in \mathcal{I},$$9 as a personalized latent preference aligned with recent context (Li et al., 25 Aug 2025).

FlowRec then adopts a simple linear interpolation path

$i_n^u$0

for which the true conditional vector field is constant: $i_n^u$1 This choice induces straight trajectories in embedding space. To regularize learning, the intermediate point $i_n^u$2 is perturbed with an element-wise Gaussian mask $i_n^u$3, whose entries satisfy $i_n^u$4 with $i_n^u$5. The flow model input is

$i_n^u$6

and the vector field is parameterized by a 2-layer MLP

$i_n^u$7

Under this path construction, the CFM loss reduces to

$i_n^u$8

A common misconception is to view FlowRec as merely a faster denoising model. In the formulation of (Li et al., 25 Aug 2025), it instead replaces the reverse diffusion mechanism with an ODE-based trajectory model whose prior is explicitly sequence-conditioned.

3. Objective design and alignment with ranking

FlowRec does not rely on vector-field regression alone. Its objective is explicitly decomposed into a prior loss, a CFM loss, and an alignment loss, each serving a distinct role (Li et al., 25 Aug 2025).

The prior loss trains the behavior encoder so that the initial state $i_n^u$9 is already predictive of the next item. It is a softmax cross-entropy objective: $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$0 This uses the true next item as the positive and all items as negatives. The associated interpretation in the source description is that the prior is not merely user-specific and sequence-conditioned, but already near the target next-item embedding (Li et al., 25 Aug 2025).

The second key addition is the single-step alignment mechanism. Given an intermediate point $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$1 and the predicted vector field $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$2, FlowRec estimates the endpoint by a one-step integration: $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$3 This assumes that the vector field remains approximately constant after time $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$4, consistent with the linear-path construction. The model then treats $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$5 as a predicted item embedding and applies another softmax cross-entropy loss: $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$6 Unlike standard flow-matching or diffusion objectives based primarily on pointwise mean-squared error, this term is ranking-aware and incorporates both positive and negative samples (Li et al., 25 Aug 2025).

The joint objective is

$$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$7

with $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$8 tuned from $$\arg\max_{i_j \in \mathcal{I}} P(i_n^u = i_j \mid \mathcal{S}^u).$$9 and $\mathcal{N}(0,I)$0 tuned from $\mathcal{N}(0,I)$1. The reported interpretation is that balancing trajectory supervision and ranking alignment is essential: overemphasizing either term degrades performance (Li et al., 25 Aug 2025).

4. Architecture, training, and inference

FlowRec uses a 4-layer bidirectional Transformer with 4 attention heads as the behavior encoder. The embedding dimension is $\mathcal{N}(0,I)$2, and the hidden size is 128. The vector-field network $\mathcal{N}(0,I)$3 is a 2-layer MLP. Time is encoded explicitly, while sequence information is injected through $\mathcal{N}(0,I)$4, so no additional sequence encoder is used inside the flow model (Li et al., 25 Aug 2025).

Training uses four benchmark datasets: Amazon Beauty, Amazon Toys, MovieLens-1M, and Yelp. Users and items with fewer than 5 interactions are removed. The maximum sequence length is 50, using truncation or padding, and the split is standard leave-one-out: the last item for test, the second-last for validation, and all previous interactions for training. Optimization uses Adam with learning rate 0.005, batch size 512, early stopping with patience 10 based on validation NDCG/HR, embedding dropout 0.1, and hidden dropout 0.3 (Li et al., 25 Aug 2025).

At inference time, FlowRec first encodes the prior $\mathcal{N}(0,I)$5 from the test sequence, then integrates the learned ODE with Euler integration: $\mathcal{N}(0,I)$6 After $\mathcal{N}(0,I)$7 steps, the final state $\mathcal{N}(0,I)$8 is scored against all item embeddings by inner product: $\mathcal{N}(0,I)$9 and top-$\phi_t$0 items are recommended by the highest scores (Li et al., 25 Aug 2025).

A practical property emphasized in the source is that FlowRec reaches near-optimal performance with $\phi_t$1–10 Euler steps, whereas DiffuRec needs around 30–35 steps. Even single-step integration using $\phi_t$2 remains competitive because the alignment loss explicitly trains the model for that regime. This suggests that FlowRec’s efficiency is not only architectural, but also objective-driven.

5. Empirical performance

The four evaluation datasets cover both sparse, short-sequence settings and a denser, long-sequence setting.

Dataset	Sequences / Items / Interactions	Avg. length / Sparsity
Amazon Beauty	22,363 / 12,101 / 198,502	8.53 / 99.93%
Amazon Toys	19,412 / 11,924 / 167,597	8.63 / 99.93%
MovieLens-1M	6,040 / 3,416 / 999,611	165.5 / 95.16%
Yelp	30,431 / 20,033 / 316,354	10.4 / 99.95%

Evaluation uses HR@5/10 and NDCG@5/10 under full ranking over all items. Baselines include classical sequential recommenders—GRU4Rec, Caser, SASRec, and BERT4Rec—contrastive methods CL4SRec and CaDiRec, and generative methods ACVAE, DiffuRec, and PreferDiff (Li et al., 25 Aug 2025).

Reported highlights indicate that FlowRec outperforms all baselines on all four datasets. On Amazon Beauty, HR@5 increases from DiffuRec’s 5.38% to FlowRec’s 5.91%, HR@10 from 7.59% to 8.28%, and NDCG@5 from 3.92% to 4.22%. On Amazon Toys, HR@5 increases from 5.70% to 6.25%, HR@10 from 7.35% to 8.23%, and NDCG@5 from 4.29% to 4.62%. On Yelp, HR@5 rises from 3.83% to 3.97% and HR@10 from 5.70% to 5.94%. On ML-1M, HR@5 rises from 15.29% to 15.58%, HR@10 from 23.79% to 24.15%, and NDCG@10 from 12.82% to 13.12% (Li et al., 25 Aug 2025). The gains are largest on the sparser Amazon datasets.

Efficiency results reinforce the same pattern. With $\phi_t$3 for both FlowRec and DiffuRec, per-epoch timings on Toys are: training 5.33s for SASRec, 10.11s for DiffuRec, and 9.98s for FlowRec; inference 0.88s for SASRec, 8.51s for DiffuRec, and 1.27s for FlowRec. On Yelp, training is 11.67s for SASRec, 19.10s for DiffuRec, and 17.62s for FlowRec; inference is 2.93s, 14.77s, and 3.67s, respectively (Li et al., 25 Aug 2025). The factual implication is that FlowRec’s inference is close to a non-generative recommender such as SASRec while remaining several times faster than DiffuRec.

6. Ablations, robustness, and limitations

Ablation studies in the description attribute substantial performance gains to each major component. On Beauty, removing the prior loss reduces HR@5 from 5.91 to 2.95; removing the CFM loss reduces it to 5.31; removing the alignment loss reduces it to 3.79. Replacing the Transformer prior encoder with a GRU lowers performance relative to the full model, though the resulting variant remains stronger than vanilla GRU4Rec. These results indicate that the behavior-based prior and the ranking-aware alignment loss are not secondary design choices, but central to the framework’s effectiveness (Li et al., 25 Aug 2025).

Further analyses are directed at trajectory geometry and robustness. A t-SNE visualization of 10-step trajectories on Amazon Toys shows that DiffuRec produces winding and unstable paths from noise to target embedding, whereas FlowRec produces smooth, almost straight trajectories from $\phi_t$4 to the target item. Another comparison evaluates recommendations produced directly from the initial state $\phi_t$5 versus the FlowRec-refined final state; on Toys and Yelp, the refined state consistently improves HR/NDCG over $\phi_t$6, indicating that the learned vector field performs meaningful refinement rather than merely reproducing the prior (Li et al., 25 Aug 2025).

The model is also reported to be especially robust in weak-context settings. When users are grouped by whether the last pre-target item is a head item or a long-tail item, FlowRec’s improvements over DiffuRec are larger on long-tail items: on Toys, +8.85% for head versus +21.28% for long-tail; on Beauty, +1.17% versus +14.05%. When sequences are divided into short, middle, and long groups in a 1:2:1 ratio, FlowRec outperforms SASRec, ACVAE, and DiffuRec in all groups, with particular robustness on short and long sequences (Li et al., 25 Aug 2025). This supports the paper’s central argument that a sequence-conditioned prior is especially valuable when the conditional signal is sparse or unstable.

The main observed limitation is that performance can decrease when the number of sampling steps exceeds approximately 25. The accompanying explanation is that the alignment loss assumes a near-constant vector field after the sampled time $\phi_t$7, and this assumption becomes less well matched when many fine steps are taken. Hyperparameter sensitivity studies show a similar trade-off: excessively large $\phi_t$8 overemphasizes vector-field regression, while excessively large $\phi_t$9 overemphasizes ranking alignment (Li et al., 25 Aug 2025).

Conceptually, FlowRec occupies a distinct position within generative recommendation. It preserves the generative objective of modeling a distribution over future interests, but replaces stochastic reverse diffusion from Gaussian noise with deterministic ODE trajectories starting from a structured, user-specific prior. The formulation therefore suggests a view of next-item prediction as learning a residual trajectory from current preference to future interest, rather than discovering the future item “from scratch” out of noise.