HyenaRec: Scalable Sequential Recommendation

• HyenaRec is a sequential recommendation model that replaces quadratic self-attention with Hyena-style convolutions, efficiently handling long and sparse user histories.
• It integrates a long convolution path with a gated short-path modulation and explicitly parameterized Legendre polynomial kernels to balance global trends with local interest bursts.
• Experimental results show improved NDCG and Recall metrics over baseline models on datasets with ultra-long sequences, while offering significant training speedups.

Searching arXiv for HyenaRec and closely related sequential recommendation work to ground the article and confirm the queried model identity. HyenaRec is a sequential recommendation model for next-item prediction on long user interaction histories. It replaces quadratic self-attention with a Hyena-style long convolution whose kernels are explicitly parameterized by Legendre orthogonal polynomials and complemented by gated short-path modulation, with the stated goal of balancing global temporal evolution and localized user interests under sparse feedback while scaling linearly with sequence length (Liu et al., 26 Mar 2026). The model is positioned for regimes in which user behavior is long, sparse, noisy, and often non-periodic, and where attention-based architectures incur prohibitive training memory and time costs.

1. Problem setting and motivation

HyenaRec is formulated on an ordered interaction sequence $\mathcal S=[i_1,\dots,i_T]$, where the task is to predict the next item $i_{T+1}$ from a corpus $\mathcal Y$ of size $V$ (Liu et al., 26 Mar 2026). The model outputs logits $\mathbf s \in \mathbb R^V$ and is trained with next-item cross-entropy / negative log-likelihood,

$$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$

At inference time, items are ranked by the softmax scores and evaluated with Recall@$k$ and NDCG@$k$.

The motivating diagnosis is that standard sequential recommendation methods encounter distinct failure modes on long histories. Classical Markov models are characterized as too local. RNNs and CNNs are described as struggling with long-range dependence or receptive-field limitations. Attention-based Transformers such as SASRec and BERT4Rec have strong modeling power, but their cost grows quadratically with sequence length because they explicitly compute pairwise token interactions. In long-history recommendation, this yields high training memory and time cost and also makes incremental inference expensive.

Hyena-style sub-quadratic sequence operators are introduced as efficient alternatives to attention in language modeling, but HyenaRec argues that recommendation imposes additional constraints. User-item interaction data are sparse, noisy, and often non-periodic, and user behavior may combine slowly evolving preferences with short bursts of activity. The model therefore treats direct transplantation of Hyena into recommendation as insufficient. This suggests that HyenaRec’s contribution is not merely operator substitution, but operator adaptation to the specific statistical structure of recommender data.

2. Architectural composition

HyenaRec has three main parts: an input embedding layer, a Hyena-based sequential backbone, and a prediction layer with tied item embeddings (Liu et al., 26 Mar 2026). Each item ID is mapped to a $D$-dimensional embedding, producing $\mathbf E \in \mathbb R^{B\times L\times D}$ for batch size $i_{T+1}$0 and sequence length $i_{T+1}$1. Rather than adding standard positional embeddings directly as in Transformers, position information is injected into the filter generation process.

The positional construction begins from a normalized temporal grid,

$i_{T+1}$2

followed by complex exponential features

$i_{T+1}$3

and the positional representation

$i_{T+1}$4

These features are supplied to the filter generator rather than being directly superposed onto token embeddings.

The backbone is a stack of $i_{T+1}$5 Hyena blocks. Each block contains a mixer, an MLP, residual connections, and normalization, and maps hidden states according to

$i_{T+1}$6

The key architectural modification lies in the mixer, which is a Hyena operator adapted for recommendation.

The prediction layer projects the final hidden states $i_{T+1}$7 to item logits,

$i_{T+1}$8

with weight tying between $i_{T+1}$9 and the input embedding table. In practical terms, this makes HyenaRec a sequence encoder followed by a standard full-vocabulary next-item predictor.

3. Hyena adaptation: long convolution, short-path modulation, and Legendre kernels

The core operator replaces explicit attention matrices with a hierarchical composition of a short path for local, data-dependent modulation and a long path for depthwise long convolution (Liu et al., 26 Mar 2026). Let $\mathcal Y$0 denote the input embeddings. Hyena first expands channels:

$\mathcal Y$1

where $\mathcal Y$2 and $\mathcal Y$3 depends on the operator order $\mathcal Y$4.

The short path splits $\mathcal Y$5 into several modulation streams and an initial state:

$\mathcal Y$6

The long path then performs stage-wise gated depthwise convolutions. For each stage $\mathcal Y$7,

$\mathcal Y$8

where $\mathcal Y$9 is element-wise modulation, $V$0 is causal depthwise 1D convolution, and $V$1 is the stage-specific convolution kernel. The output is

$V$2

with $V$3.

A major contribution is the replacement of the original implicit Hyena filter generator with an explicit polynomial-based kernel parameterization using Legendre orthogonal polynomials. For each stage $V$4, the kernel is represented as

$V$5

where $V$6 is the $V$7-th Legendre polynomial, $V$8 are learnable coefficients, and $V$9 is the number of basis functions. The Legendre polynomials are defined by Rodrigues’ formula,

$\mathbf s \in \mathbb R^V$0

and can also be computed by the recurrence

$\mathbf s \in \mathbb R^V$1

The full basis matrix $\mathbf s \in \mathbb R^V$2 is formed by evaluating the polynomials on the temporal grid, and the channel-wise kernels are computed as

$\mathbf s \in \mathbb R^V$3

with $\mathbf s \in \mathbb R^V$4. This enforces per-channel $\mathbf s \in \mathbb R^V$5 normalization,

$\mathbf s \in \mathbb R^V$6

which is stated to help prevent signal blow-up or vanishing across stacked layers.

The paper emphasizes three reasons for selecting Legendre polynomials: orthogonality under the uniform measure, controlled boundary values with $\mathbf s \in \mathbb R^V$7, and a non-periodic inductive bias more suitable for recommendation than Fourier bases because user behavior is sparse and not naturally periodic. This suggests that the kernel basis is intended not only as a parameter-efficient representation, but also as an inductive bias aligned with recommender sequence statistics.

4. Short-term burst modeling, convolutional implementation, and complexity

HyenaRec does not rely only on long convolutions. The multiplicative term $\mathbf s \in \mathbb R^V$8 functions as a data-dependent gate that decides which signals are passed into the long convolution at each stage (Liu et al., 26 Mar 2026). This component is motivated by the presence of short behavioral bursts in recommendation sequences, including sudden changes in intent, repeated clicks on a narrow topic, and immediate transitions reflecting short-term preferences. Long convolution is characterized as effective for slowly varying and global dependencies but liable to smooth away local bursts; the gating branch is therefore presented as a complementary mechanism preserving fine-grained local structure while modulating long-range propagation.

Once kernels are built, the depthwise convolution is applied as

$\mathbf s \in \mathbb R^V$9

and the model computes these convolutions efficiently in the frequency domain using FFT. The intended effect is near-linear scaling together with long effective receptive fields.

The complexity discussion contrasts standard self-attention and HyenaRec. Standard self-attention requires building an $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$0 similarity matrix, with time complexity $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$1 and memory complexity $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$2. HyenaRec uses embedding and projection layers, Legendre kernel generation, and FFT-based long convolution, with per-layer cost reported as roughly

$$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$3

and memory about

$$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$4

The Legendre basis generation itself costs $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$5, stated to be small because $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$6. The paper also notes that inference supports $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$7 incremental computation per step in a streaming setting. In the model’s framing, these asymptotics are the basis for handling ultra-long sequences such as XLong, whose average length is approximately $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$8, without reverting to quadratic pairwise interaction modeling.

5. Experimental protocol and quantitative results

Experiments are conducted on four real-world datasets: ML-1M, Steam, Video, and XLong (Liu et al., 26 Mar 2026). ML-1M contains 6,040 users and 3,416 items, with average length 165.5 and sparsity 95.2%. Steam contains 334,730 users and 13,047 items, with average length 11.0 and sparsity 99.96%. Video contains 31,013 users and 23,715 items, with average length 9.3 and sparsity 99.9%. XLong contains 69,691 users and 2,122,932 items, with average length 958.8 and sparsity 99.95%. The evaluation protocol is leave-one-out: the last interaction is used for test, the second-to-last for validation, and the rest for training; items with fewer than 5 interactions and users with fewer than 2 interactions are filtered.

The baseline set comprises GRU4Rec, NARM, SASRec, BERT4Rec, EulerFormer, HSTU, and LRURec. Additional comparisons include Mamba4Rec and an AtrousHyena variant in appendix experiments. The reported metrics are Recall@10, Recall@20, NDCG@10, and NDCG@20. The training details are also specified: AdamW optimizer, learning rate $$\mathcal L(\theta) = -\log \frac{\exp(s_{i_{T+1}})}{\sum_{j\in\mathcal Y}\exp(s_j)}.$$9, batch size 128 or 32 for XLong, maximum 500 epochs, early stopping if Recall@10 does not improve for 10 validations, hidden size 64, 2 layers, default dropout 0.2, and weight decay tuned among $k$0.

HyenaRec is reported as the best or second-best model across all datasets and metrics. On ML-1M, it achieves NDCG@10 $k$1, NDCG@20 $k$2, Recall@10 $k$3, and Recall@20 $k$4. Compared to the strongest baseline LRURec, this corresponds to about +4.94% on NDCG@10, +5.10% on NDCG@20, +3.30% on Recall@10, and +4.04% on Recall@20. On Steam, the reported values are NDCG@10 $k$5, NDCG@20 $k$6, Recall@10 $k$7, and Recall@20 $k$8. On Video, the reported values are NDCG@10 $k$9, NDCG@20 $k$0, Recall@10 $k$1, and Recall@20 $k$2.

The clearest gains appear on XLong. There HyenaRec reports NDCG@10 $k$3, NDCG@20 $k$4, Recall@10 $k$5, and Recall@20 $k$6. The model is stated to be slightly below LRURec’s $k$7 on Recall@20, but to win clearly on NDCG, especially top-$k$8 ranking quality. The strongest reported improvements are +17.65% on NDCG@10 and +13.83% on NDCG@20 over LRURec. This supports the paper’s claim that the model is especially effective on long-sequence recommendation rather than uniformly dominant on every metric.

Training-speed comparisons are also reported. On ML-1M, SASRec takes 1.77$k$9 longer, BERT4Rec takes 4.25$D$0 longer, and HSTU takes 1.62$D$1 longer. The abstract states up to 6$D$2 speedup. The interpretation offered in the paper is that the linear/sub-quadratic operator design and efficient FFT convolution jointly account for this training-time advantage.

6. Ablations, operating regime, limitations, and name disambiguation

The ablation study isolates the importance of the main design choices (Liu et al., 26 Mar 2026). Removing the polynomial kernel, denoted w/o PK, lowers performance across datasets: for example, Steam Recall@20 drops from $D$3 to $D$4, and on XLong Recall@20 drops from $D$5 to $D$6. Removing GLU/gating, denoted w/o GLU, also hurts performance, especially on ML-1M and Video. Replacing the kernel with Fourier bases worsens results, especially on XLong, where Recall@20 becomes $D$7 versus $D$8. Replacing the kernel with Chebyshev bases also underperforms Legendre kernels, though sometimes not as badly as Fourier. The reported takeaway is that both components matter: the Legendre polynomial kernel provides smoother, more stable long-range modeling, and the gated short path captures localized bursts.

Sensitivity analysis indicates that the model is fairly robust to the polynomial order $D$9, with best performance around $\mathbf E \in \mathbb R^{B\times L\times D}$0. Performance improves from $\mathbf E \in \mathbb R^{B\times L\times D}$1 to $\mathbf E \in \mathbb R^{B\times L\times D}$2, then slightly drops at $\mathbf E \in \mathbb R^{B\times L\times D}$3, suggesting possible overfitting or unnecessary complexity at larger basis sizes. Additional observations are that moderate dropout works best, too much dropout hurts long-range preference modeling, and weight decay around $\mathbf E \in \mathbb R^{B\times L\times D}$4 is a good default.

The operating regime in which HyenaRec is most valuable is stated explicitly: user histories are long, feedback is sparse, and behavior mixes long-term drift with short-term bursts. Its advantage is largest on XLong and other longer sequences, where attention becomes expensive and long-range modeling matters most. On shorter or less challenging datasets, the margin over strong baselines is described as smaller, though still positive. A related limitation is that some variants such as atrous convolutions provide only modest gains, suggesting that increased multi-scale convolutional complexity alone is not sufficient; the structured kernel design and gating are more important.

The name “HyenaRec” should also be distinguished from similarly named but unrelated works. It is not HyTRec, the hybrid temporal-aware attention architecture for long behavior sequential recommendation (Xin et al., 20 Feb 2026), and it is not HYREC-2, a hydrogen recombination code for cosmology (Lee et al., 2020). Within recommendation research, HyenaRec denotes the model that adapts the Hyena operator to sequential recommendation through Legendre orthogonal polynomial kernels and gated convolutions.