TBGRecall: Session-Wise Generative Retrieval

• TBGRecall is a session-wise generative retrieval framework that predicts unordered next-session items using a decoder-only Transformer.
• It reformulates conventional autoregressive methods by appending an empty context token to generate a representative embedding for ANN search.
• The system integrates limited-history pre-training, stochastic partial incremental training, and MoE to achieve scalable e-commerce recommendation gains.

TBGRecall is a generative retrieval framework for e-commerce recommendation that reformulates retrieval around Next Session Prediction (NSP) rather than standard autoregressive item generation. It is presented as a decoder-only Transformer that consumes a session-wise sequence of tokens, appends a new empty context token for the next session, and uses the final hidden state of that token as the retrieval query for approximate nearest-neighbor search in the item-embedding space. The stated motivation is that conventional autoregressive approaches introduce sequential dependencies that impede efficient retrieval and are inherently unsuitable for generating multiple items without positional constraints within a single request session; TBGRecall is proposed specifically to remove those within-session dependencies while preserving a generative modeling perspective (Liang et al., 16 Aug 2025).

1. Problem formulation and conceptual scope

TBGRecall is designed for recommendation scenarios in which a user’s recent behavior can be segmented into sessions and the retrieval target is an unordered set of relevant items for the next request context. Its core reformulation is session-wise: each user history is partitioned into multiple sessions, and prediction is directed at the items in the next session rather than the next token in a globally ordered sequence.

This framing distinguishes TBGRecall from token-by-token autoregressive generation. In the stated formulation, the model does not generate a ranked sequence of future items one position at a time. Instead, it produces a single context representation for the next session and scores candidate items against that representation. A common misconception is therefore to read TBGRecall as a standard next-item language-modeling system; the description instead defines it as a session-wise generative retrieval paradigm that “breaks unwanted within-session autoregressive dependencies.”

The framework is positioned for large-scale e-commerce retrieval. The paper explicitly combines architectural reformulation, task design, training methodology, and deployment-oriented optimization, including limited historical data pre-training, stochastic partial incremental training, TSN, MSP, and MoE.

2. Sequence architecture and tokenization

The model architecture is a decoder-only Transformer operating over a session-wise token sequence. A user’s behavior is segmented into sessions, and each session $k$ is represented by a special context token $c^{(k)}$ followed by item tokens $i_1^{(k)}, \dots, i_{N_k}^{(k)}$. At inference time, a new empty context token $c^{(K+1)}$ is appended, and its final hidden state is used as the retrieval query.

The input sequence over the last $K$ sessions is written as

$$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$

Each token, whether a context token or an item token, is represented by the sum of four embeddings:

$$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$

The four components are specified as follows. $\mathbf{e}_{\mathrm{id}}$ is the item-ID embedding, or a learned vector for session tokens. $\mathbf{e}_{\mathrm{act}}$ is an action embedding indicating click or exposure. $\mathbf{e}_{\mathrm{side}}$ is a side-information embedding, including category, price, and seller. $c^{(k)}$0 is a scenario, or session, embedding.

After the full sequence passes through the Transformer, the appended context token yields

$c^{(k)}$1

For a candidate item $c^{(k)}$2 with static token embedding $c^{(k)}$3, the retrieval score is

$c^{(k)}$4

This is converted into a probability over the item pool $c^{(k)}$5:

$c^{(k)}$6

Because retrieval returns an unordered set of items, the formulation also gives a joint probability under an independence assumption:

$c^{(k)}$7

Items are retrieved by ANN search in the item-embedding space using the context embedding. This suggests that TBGRecall treats generation and retrieval as a single embedding-space problem rather than as sequential decoding of an output list.

3. Next Session Prediction and attention structure

The defining task is Next Session Prediction (NSP). The model is trained to predict the items in session $c^{(k)}$8, which is operationalized by moving the new context token $c^{(k)}$9 close to future positive items and far from sampled negatives.

The attention pattern is modified in two ways. First, a session mask is applied on top of the usual causal mask so that within each session $i_1^{(k)}, \dots, i_{N_k}^{(k)}$0, item tokens $i_1^{(k)}, \dots, i_{N_k}^{(k)}$1 do not attend to each other. They attend only to all previous sessions’ tokens and to their own context token $i_1^{(k)}, \dots, i_{N_k}^{(k)}$2. Second, the model uses session-wise RoPE (sw-RoPE), under which all tokens in the same session share the same positional encoding. The stated purpose is to enforce that order within a session carries no semantic meaning.

The objective begins with a contrastive or NCE term. Let $i_1^{(k)}, \dots, i_{N_k}^{(k)}$3 be the positive items shown in session $i_1^{(k)}, \dots, i_{N_k}^{(k)}$4, and $i_1^{(k)}, \dots, i_{N_k}^{(k)}$5 a random set of negative items sampled from the full pool. For each $i_1^{(k)}, \dots, i_{N_k}^{(k)}$6,

$i_1^{(k)}, \dots, i_{N_k}^{(k)}$7

The session-level NCE loss is

$i_1^{(k)}, \dots, i_{N_k}^{(k)}$8

Two cascade losses are added to emphasize clicked and purchased items. With $i_1^{(k)}, \dots, i_{N_k}^{(k)}$9 and $c^{(K+1)}$0 denoting the clicked and purchased subsets,

$c^{(K+1)}$1

$c^{(K+1)}$2

To balance across application scenarios $c^{(K+1)}$3, the losses are normalized and summed:

$c^{(K+1)}$4

where $c^{(K+1)}$5 is the number of sessions in scenario $c^{(K+1)}$6.

This task design is also the main reason TBGRecall differs from ordinary next-item prediction. The target is a future session-level set, and the loss explicitly encodes exposure, click, and pay signals.

4. Training pipeline and optimization mechanisms

The training pipeline begins with limited-history pre-training. TBGRecall is first trained on a fixed historical snapshot of user sequences, sampling from the last $c^{(K+1)}$7 sessions per user to learn general behavior patterns. The pre-training objective is the same $c^{(K+1)}$8, restricted to a large historical dataset covering many months.

The second stage is stochastic partial incremental training (PIT). The motivation is operational: the full model and data are too large to retrain daily. Users are split randomly into $c^{(K+1)}$9 buckets. On day $K$0, bucket $K$1 is selected, and training uses only that bucket’s data from the last 10 days. Because each bucket cycles once every $K$2 days, all user-day pairs are eventually seen, while each daily run uses only $K$3 of the data. The reported training time is approximately $K$4 h, compared with $K$5 d for full data, while still incorporating recent user signals. The description further states that PIT matches the performance of an all-day daily-incremental strategy using $K$6 more GPUs.

Several additional optimization techniques are included. Negative Sampling uses hard negatives from the full item pool for the NCE loss. Token-Specific Network (TSN) applies separate linear projections and a tiny token-type-aware network for context tokens versus item tokens, with the stated effect of closing the domain gap without adding inference cost. Multi-Session Prediction (MSP) is inspired by multi-token prediction in LLMs and predicts several future context tokens in parallel, increasing signal and long-range dependencies. Mixture-of-Experts (MoE) replaces the feed-forward layers with a sparse MoE containing 24 experts and routes 2 experts per token at inference. Sequence Truncation & Curriculum caps total token length, for example at 5120, and can schedule from shorter to longer sessions to stabilize early training, with stochastic length in HSTU cited as the reference pattern.

Taken together, these mechanisms indicate that TBGRecall is not presented only as a modeling proposal. It is also a training and systems recipe for high-throughput retrieval.

5. Scaling behavior

The scaling analysis trains TBGRecall variants with hidden dimensions $K$7, with and without MoE, and measures HR@4000 against parameter count $K$8 (Liang et al., 16 Aug 2025). On a log-log plot of parameters versus performance gap, the reported trend is an approximate power law:

$K$9

with $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$0–0.3 as an empirical fit.

The paper interprets this as evidence that larger models consistently yield better retrieval, and states that gains continue to accrue even at industrial scale, around $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$1 B parameters with MoE. A plausible implication is that the session-wise NSP formulation does not saturate immediately when scaled, which is important for industrial recommendation settings where retrieval models are often constrained by both compute and latency.

6. Benchmarks, deployment results, and stated implications

The evaluation metric is Hit Rate@K (HR@K) for $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$2 (Liang et al., 16 Aug 2025).

On the public benchmark RecFlow, the reported results are: HSTU achieves HR@20 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$3, HR@100 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$4, HR@500 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$5, HR@1000 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$6, HR@2000 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$7, and HR@4000 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$8; TBGRecall achieves HR@20 $$\mathcal{Q} = \bigl[ c^{(1)},\, i^{(1)}_1,\dots,i^{(1)}_{N_1},\; c^{(2)},\,\dots,\; c^{(K)},\,i^{(K)}_1,\dots,i^{(K)}_{N_K},\; c^{(K+1)} \bigr].$$9, HR@100 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$0, HR@500 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$1, HR@1000 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$2, HR@2000 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$3, and HR@4000 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$4. The accompanying summary states that TBGRecall improves over the previous best, HSTU, by roughly $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$5–$$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$6 percentage points across $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$7.

On the Taobao industrial-scale dataset, the production dual-tower baseline Online(DT) records HR@20 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$8, HR@100 $$\mathbf{h} = \mathbf{e}_{\mathrm{id}} + \mathbf{e}_{\mathrm{act}} + \mathbf{e}_{\mathrm{side}} + \mathbf{e}_{\mathrm{ctx}}.$$9, HR@500 $\mathbf{e}_{\mathrm{id}}$0, HR@1000 $\mathbf{e}_{\mathrm{id}}$1, HR@2000 $\mathbf{e}_{\mathrm{id}}$2, and HR@4000 $\mathbf{e}_{\mathrm{id}}$3. TBGRecall records HR@20 $\mathbf{e}_{\mathrm{id}}$4, HR@100 $\mathbf{e}_{\mathrm{id}}$5, HR@500 $\mathbf{e}_{\mathrm{id}}$6, HR@1000 $\mathbf{e}_{\mathrm{id}}$7, HR@2000 $\mathbf{e}_{\mathrm{id}}$8, and HR@4000 $\mathbf{e}_{\mathrm{id}}$9. The text characterizes this as a $\mathbf{e}_{\mathrm{act}}$0–$\mathbf{e}_{\mathrm{act}}$1 percentage point gain at large $\mathbf{e}_{\mathrm{act}}$2 against a production daily-trained dual-tower baseline.

The partial incremental training ablation reports four settings. Normal (full) uses duration $\mathbf{e}_{\mathrm{act}}$3, $\mathbf{e}_{\mathrm{act}}$4 GPUs, does not use the latest data, and achieves HR@4000 $\mathbf{e}_{\mathrm{act}}$5. Incremental (Incre) uses duration $\mathbf{e}_{\mathrm{act}}$6, $\mathbf{e}_{\mathrm{act}}$7 GPUs, uses the latest data, and achieves HR@4000 $\mathbf{e}_{\mathrm{act}}$8. Stoch. Length (SL) uses duration $\mathbf{e}_{\mathrm{act}}$9, $\mathbf{e}_{\mathrm{side}}$0 GPUs, does not use the latest data, and achieves HR@4000 $\mathbf{e}_{\mathrm{side}}$1. PIT (ours) uses duration $\mathbf{e}_{\mathrm{side}}$2, $\mathbf{e}_{\mathrm{side}}$3 GPUs, uses the latest data, and achieves HR@4000 $\mathbf{e}_{\mathrm{side}}$4. The stated conclusion is that PIT uses one-tenth the compute of Incremental but matches its performance.

The online A/B test allocates 5% of Taobao homepage “Guess You Like” traffic over 7 days. The reported PVR is 23.94%. The measured effects are transaction count +0.60% and transaction amount +2.16%, both statistically significant.

The paper’s stated main contributions are threefold: introduction of Next Session Prediction (NSP) as a session-wise generative retrieval paradigm; design of a complete training pipeline tailored for large-scale retrieval; and demonstration of strong scaling laws and real-world gains on Taobao, including +2.16% GMV. The future directions listed are extension to multi-modal contexts such as images, text, and video; investigation of adaptive session masking for partial ordering; better cold-start strategies by combining generative ID tokens with content tokens; and theoretical analysis of contrastive versus autoregressive losses in retrieval.

These claims also clarify two important interpretive points. First, TBGRecall is not presented as abandoning generative modeling; it is presented as a bridge between LLM-style generative modeling and high-throughput e-commerce retrieval. Second, it does not model within-session item order as semantically meaningful; sw-RoPE and the session mask explicitly suppress that assumption.