Co-NAML-LSTUR: Unified Neural News Rec Model

• The paper proposes Co-NAML-LSTUR, which unifies multi-view attentive news encoding (using DistilBERT) with dual-scale user representation to tackle information overload and dynamic user interests.
• The model integrates heterogeneous news features (titles, abstracts, categories) via attentive fusion alongside an LSTM-based user encoder enhanced with candidate-aware attention.
• Empirical results on MIND benchmarks show significant gains in AUC, MRR, and nDCG metrics, validating the neural predictor’s effectiveness over traditional dot-product scoring.

Co-NAML-LSTUR is a neural news recommendation model that explicitly combines attentive multi-view news encoding and long- and short-term user representations within a single framework. It was proposed to address three intertwined challenges in news recommendation: information overload, multi-view news representation, and dynamic user interests that span both stable preferences and recent behavior. The model integrates NAML for multi-view news modeling, LSTUR for long- and short-term user modeling, DistilBERT-based word embeddings for semantic feature extraction, and a DKN-style neural interaction module for user–news matching (Nguyen et al., 27 Jul 2025).

1. Problem setting and model lineage

The model is situated in the standard news recommendation setting in which a system must rank a small set of candidate news per impression from a large and continuously changing news pool. The paper identifies three core difficulties. First, users face thousands of new articles, so the recommender must mitigate information overload. Second, each news article contains heterogeneous views, including free-text fields such as title and abstract and structured fields such as category and subcategory. Third, user interests are dynamic and span long-term preferences and short-term, temporary interests (Nguyen et al., 27 Jul 2025).

Within this framing, NAML is described as focusing on attentive multi-view news encoding but essentially using a single user representation aggregated from clicked news, whereas LSTUR is described as focusing on explicit long- and short-term user encoding without treating multi-view news content as richly as NAML. Co-NAML-LSTUR fuses these strengths into a single architecture. On the news side it uses NAML-style multi-view encoding with BERT-based word embeddings; on the user side it uses LSTUR-style long/short-term modeling with an LSTM instead of a GRU and an additional attention mechanism over user history conditioned on the candidate; on the matching side it uses a DKN-inspired neural predictor rather than only a dot product.

A common misconception is to treat the model as a simple serial combination of NAML and LSTUR. The paper explicitly positions it as a more carefully integrated architecture: multi-view attentive news encoding, long-term ID-based user representation, short-term sequential modeling, candidate-aware attention over history, contextual language representations from DistilBERT, and a neural click predictor are all jointly incorporated into the final design.

2. Architectural composition

The architecture has three main modules: a Multi-view News Encoder, a User Encoder, and a Click Predictor (Nguyen et al., 27 Jul 2025).

The Multi-view News Encoder takes as input a title sequence $\{w_1^t, \dots, w_M^t\}$, an abstract sequence $\{w_1^{abs}, \dots, w_P^{abs}\}$, a category ID $v_c$, and a subcategory ID $v_{sc}$. Its output is a unified news vector $r \in \mathbb{R}^d$. This module is the Co-NAML component.

The User Encoder receives a sequence of clicked news embeddings $\{r_1,\dots,r_k\}$ produced by the news encoder, a user ID $u$, and a candidate news embedding $r_c$ for attention. It contains a long-term representation $u_l$ from a user ID embedding, a short-term sequential representation $u_s$ from an LSTM over the click sequence initialized with $\{w_1^{abs}, \dots, w_P^{abs}\}$0, and an attention-based representation $\{w_1^{abs}, \dots, w_P^{abs}\}$1 over the click history conditioned on $\{w_1^{abs}, \dots, w_P^{abs}\}$2. Its output is a final user representation that combines long-term, short-term, and candidate-aware historical evidence.

The Click Predictor uses the user embedding $\{w_1^{abs}, \dots, w_P^{abs}\}$3 and candidate news embedding $\{w_1^{abs}, \dots, w_P^{abs}\}$4 to predict click probability $\{w_1^{abs}, \dots, w_P^{abs}\}$5. Two variants are implemented. The first is a dot-product scorer, $\{w_1^{abs}, \dots, w_P^{abs}\}$6. The second is a neural predictor, $\{w_1^{abs}, \dots, w_P^{abs}\}$7, where $\{w_1^{abs}, \dots, w_P^{abs}\}$8 is an MLP. Training uses Noise Contrastive Estimation with sampled negatives.

This modular structure is important for interpretation. News representation, user representation, and user–item interaction are not collapsed into a single undifferentiated encoder. Instead, the model separates heterogeneous item views, user time scales, and the final matching function, which makes the design comparatively transparent and extensible within the constraints discussed in the paper.

3. Attentive multi-view news encoding

The news encoder follows the NAML style of multi-view attentive fusion, but it replaces shallow word embeddings with DistilBERT-based contextual token embeddings and uses the abstract instead of the full body for efficiency and content filtering (Nguyen et al., 27 Jul 2025).

For title and abstract tokens, DistilBERT is used as a frozen encoder to produce contextual semantic vectors. If $\{w_1^{abs}, \dots, w_P^{abs}\}$9 and $v_c$0 denote the DistilBERT embeddings of title and abstract tokens, these embeddings are then passed to CNN and attention layers. For titles, the CNN layer extracts local patterns:

$v_c$1

followed by word-level attention:

$v_c$2

and the final title representation is

$v_c$3

The abstract encoder mirrors the title encoder but uses its own CNN and attention parameters:

$v_c$4

$v_c$5

Category and subcategory are treated as high-level labels. They are embedded and then projected nonlinearly:

$v_c$6

$v_c$7

These vectors inject topic-level semantics that complement the textual views.

The four view-specific representations $v_c$8 are fused by view-level attention, and the final news vector is

$v_c$9

The significance of this component lies in its explicit treatment of heterogeneity. Titles and abstracts provide fine-grained lexical and contextual information, while category and subcategory provide compact topic-level signals. The model therefore does not assume that any single field is sufficient. The ablation results reported in the paper support this design choice: removing category/subcategory embeddings decreases performance, and removing textual embeddings decreases performance further.

4. Long- and short-term user representation and click prediction

User modeling is built around LSTUR’s basic idea of combining stable user identity information with sequential click behavior, and it is enhanced with candidate-aware attention over history (Nguyen et al., 27 Jul 2025).

The long-term user representation is a trainable ID embedding:

$v_{sc}$0

This is intended to capture stable preferences and is updated during training.

The short-term representation is computed with an LSTM over the clicked news sequence:

$v_{sc}$1

Unlike original LSTUR, which uses a GRU, Co-NAML-LSTUR uses an LSTM to exploit potentially richer temporal dependencies. The LSTM is initialized with the long-term embedding, in the LSTUR-ini style:

$v_{sc}$2

The data block states that this tends to work better than simple concatenation, as found in LSTUR and adopted here.

Beyond sequence modeling, the model adds candidate-aware attention over history. For candidate news vector $v_{sc}$3, each historical item $v_{sc}$4 receives an attention weight

$v_{sc}$5

where $v_{sc}$6 is a feedforward network. The attention-based user embedding is

$v_{sc}$7

The paper’s final formula is

$v_{sc}$8

and it explicitly notes that “$v_{sc}$9” is used informally; in practice, it can be treated as elementwise product, concatenation and projection, or another simple fusion. Conceptually, the final user representation incorporates long-term profile, short-term sequential state, and candidate-aware history.

For click prediction, the model implements both a dot-product predictor and a DKN-style neural predictor. The dot-product form is computationally simple. The neural predictor,

$r \in \mathbb{R}^d$0

permits non-linear and cross-feature interactions between user and news vectors. The paper reports that the neural click predictor gives significantly better ranking performance, at the cost of increased computation.

This component clarifies the model’s treatment of temporal preference structure. User interest is not assumed to be static, and relevance is not assumed to be representable by linear similarity alone. The reported results suggest that both assumptions are too restrictive for the benchmark setting considered.

5. Optimization, datasets, and evaluation protocol

Training uses Noise Contrastive Estimation with negative sampling (Nguyen et al., 27 Jul 2025). For each clicked positive candidate news $r \in \mathbb{R}^d$1, the model samples $r \in \mathbb{R}^d$2 negative candidates $r \in \mathbb{R}^d$3 from news not clicked by that user and computes

$r \in \mathbb{R}^d$4

with loss

$r \in \mathbb{R}^d$5

This is described as essentially a softmax over one positive plus $r \in \mathbb{R}^d$6 negatives per impression. The reported hyperparameters are: $r \in \mathbb{R}^d$7, Adam with learning rate $r \in \mathbb{R}^d$8, batch size $r \in \mathbb{R}^d$9, $\{r_1,\dots,r_k\}$0 epochs, dropout $\{r_1,\dots,r_k\}$1, and frozen DistilBERT.

The main benchmark is MIND. The paper reports the following statistics for MIND-small: #News $\{r_1,\dots,r_k\}$2, #Categories $\{r_1,\dots,r_k\}$3, #Impressions $\{r_1,\dots,r_k\}$4, #Clicks $\{r_1,\dots,r_k\}$5. For MIND-large it reports: #News $\{r_1,\dots,r_k\}$6, #Categories $\{r_1,\dots,r_k\}$7, #Impressions $\{r_1,\dots,r_k\}$8, #Clicks $\{r_1,\dots,r_k\}$9. To mitigate resource constraints, the authors additionally construct MINDtiny by sampling from MIND-large. They select high-activity users with at least $u$0 clicks to preserve long behavior sequences and select popular news clicked on average more than $u$1 times to concentrate on frequently interacted items. MINDtiny has #News $u$2, #Users $u$3, #Categories $u$4, #Subcategories $u$5, average title length $u$6 words, and average abstract length $u$7 words.

The training and evaluation protocol is also specific. Training combines MIND-small and MINDtiny for model learning. Evaluation is then conducted separately on MIND-small and MIND-large. User history uses up to the most recent $u$8 clicked news per user. Text inputs use titles and abstracts, truncated where necessary, together with categories and subcategories where supported.

The baselines include feature-based models LibFM and DeepFM, and neural recommenders DKN, NPA, NAML, LSTUR, NRMS, HiFiArk, TANR, HieRec, and MINER. The evaluation metrics are AUC, MRR, nDCG@5, and nDCG@10. The implementation uses a single NVIDIA RTX A6000 and requires about $u$9 hours for full experiments. The code is publicly available at https://github.com/MinhNguyenDS/Co-NAML-LSTUR.

6. Empirical results, ablations, and positioning

On MIND-small, Co-NAML-LSTUR reports AUC $r_c$0, MRR $r_c$1, nDCG@5 $r_c$2, and nDCG@10 $r_c$3. The paper states that it clearly outperforms classical baselines and most neural baselines. Relative to NRMS, the reported AUC gain is $r_c$4; relative to LSTUR, the AUC gain is $r_c$5 and MRR is higher ($r_c$6 vs. $r_c$7); relative to NAML, the AUC gain is $r_c$8 and nDCG@10 increases by $r_c$9. MINER achieves the best numbers overall on MIND-small, with AUC $u_l$0 and nDCG@10 $u_l$1, while Co-NAML-LSTUR is reported as consistently second-best and has a moderate parameter count of $u_l$2M versus LSTUR’s $u_l$3M (Nguyen et al., 27 Jul 2025).

On MIND-large, Co-NAML-LSTUR reports AUC $u_l$4, MRR $u_l$5, nDCG@5 $u_l$6, and nDCG@10 $u_l$7. It obtains the second-best results on all metrics, trailing only MINER. The reported comparisons are: versus NRMS, $u_l$8 AUC; versus LSTUR, $u_l$9 AUC together with higher MRR and nDCG; versus NAML, $u_s$0 AUC. The paper interprets this as showing strong generalization from MIND-small + MINDtiny training to the much larger MIND-large test set.

The ablation study on MIND-small validation isolates the contribution of individual components. Full Co-NAML-LSTUR gives AUC $u_s$1, MRR $u_s$2, and nDCG@10 $u_s$3. Without category/subcategory embeddings, AUC drops to $u_s$4 and nDCG@10 to $u_s$5. Without word embeddings, AUC falls to $u_s$6 and nDCG@10 to $u_s$7. A GloVe-based version of Co-NAML-LSTUR without BERT yields AUC $u_s$8 and nDCG@10 $u_s$9; removing category in that configuration gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$00, and removing word embeddings gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$01. The paper therefore states that BERT-based contextual embeddings give a large gain, with AUC increasing from $\{w_1^{abs}, \dots, w_P^{abs}\}$02 to $\{w_1^{abs}, \dots, w_P^{abs}\}$03 when BERT is used. A baseline NAML + BERT without LSTUR-style user modeling reaches AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$04 and nDCG@10 $\{w_1^{abs}, \dots, w_P^{abs}\}$05, which remains worse than the full combined model.

The click predictor comparison further differentiates architectural choices. For NAML, dot-product prediction gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$06 and nDCG@10 $\{w_1^{abs}, \dots, w_P^{abs}\}$07, while the neural predictor gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$08 and nDCG@10 $\{w_1^{abs}, \dots, w_P^{abs}\}$09. For Co-NAML-LSTUR, dot-product gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$10 and nDCG@10 $\{w_1^{abs}, \dots, w_P^{abs}\}$11, whereas the neural predictor gives AUC $\{w_1^{abs}, \dots, w_P^{abs}\}$12 and nDCG@10 $\{w_1^{abs}, \dots, w_P^{abs}\}$13. The reported gain for Co-NAML-LSTUR is $\{w_1^{abs}, \dots, w_P^{abs}\}$14 AUC, with roughly $\{w_1^{abs}, \dots, w_P^{abs}\}$15 training time as the trade-off.

The case study included in the paper describes a user with varied interests in entertainment, lifestyle, and food. The top-5 recommendations from Co-NAML-LSTUR concentrate on categories the user frequently clicks, such as tv-celebrity and movies, while still surfacing relevant articles from less frequent categories such as general news, and they include an article the user actually clicked. This is presented as evidence that the model can capture multi-faceted preferences rather than overfitting to a single topic.

In the paper’s own positioning, Co-NAML-LSTUR is not just “NAML + LSTUR stacked together.” Its stated contributions are to unify multi-view attentive news encoding and dual-scale user modeling, enhance semantic modeling with DistilBERT embeddings, strengthen user–news interaction modeling through a DNN-based predictor, and provide a modular framework whose components can be independently improved. The discussion also implies several limitations and future directions. DistilBERT is frozen rather than fine-tuned; only textual and categorical views are used; user modeling remains single-vector at prediction time; and hardware constraints limited ablations to validation splits and relatively shallow networks. A plausible implication is that future extensions could investigate stronger PLMs, multi-modal news representations, or multi-interest user representations while preserving the same modular decomposition.