TubiFM: Unified Streaming Ranking

• TubiFM is a unified ranking system that models item, carousel, and search ranking as prompted next-token prediction over serialized user stories.
• It leverages a shared grammar to integrate heterogeneous signals from watch events, search queries, and session context in one end-to-end framework.
• The productionized Llama 3.2 1B model demonstrates improved HR@K metrics and reduced latency, outperforming task-specific baselines.

TubiFM is a unified ranking system for streaming discovery that models item ranking, carousel ranking, and search ranking within a single large-language-model framework. Rather than training separate production models for each surface, it represents a viewer’s cross-surface behavior as a serialized “user story” and expresses heterogeneous ranking tasks as prompted next-token prediction over a shared grammar. In the reported instantiation, TubiFM is a finetuned Llama 3.2 1B model trained on user stories containing user attributes, sessions, watch events with surface and carousel context, and search events, including search-as-you-type prefixes and queries that do not lead to a watch (Salle et al., 22 May 2026).

1. Problem formulation and system objective

TubiFM addresses three coupled ranking tasks in streaming discovery: item ranking, carousel ranking, and search ranking. The paper frames these tasks as arising from the same viewer journey but being conventionally handled by separate models and serving stacks. That separation is described as fragmenting complementary signals across and within sessions: searches can reveal topical intent even when they do not lead to a watch, searches may express rewatch or continuation intent, and watches provide feedback about which carousels and future search results should be prominent (Salle et al., 22 May 2026).

The central design choice is to collapse these signals into one modeling interface. Instead of using task-specific architectures or heads, TubiFM uses prompted next-token prediction over a single sequence representation. This makes item, carousel, and search ranking different task prompts applied to the same prefix, rather than distinct model families. A plausible implication is that the system treats discovery not as a set of loosely coupled recommenders, but as a single sequential inference problem in which watches, searches, session boundaries, and placement context are jointly informative.

The paper identifies three main contributions. First, it introduces the unified “user story” representation. Second, it defines a shared grammar over interleaved pretrained language tokens and domain-specific event tokens. Third, it presents a single productionized LLM ranker that outperforms specialist baselines offline across item, carousel, and search, while yielding significant online gains for search and carousel ranking and statistically neutral item ranking relative to a mature production stack (Salle et al., 22 May 2026).

2. User stories and the shared grammar

A user story is an ordered, flat token sequence. Its components are a user attribute header, session markers, and chronological interaction events. The attribute header contains coarse fields such as country and device serialized as ordinary text tokens. Sessions are marked by <|session|>, with a new session created if inactivity exceeds 1 hour and sessions capped at 12 hours. Each session also includes elapsed time since the previous session and day-of-week. Within sessions, watch events encode hour-of-day, originating surface, carousel identity, item identifier, and viewing duration, while search events encode hour-of-day and the raw query text for every search, including search-as-you-type prefixes and queries that do not lead to watches (Salle et al., 22 May 2026).

This serialization is governed by a formal grammar. Let $\Sigma_L$ be pretrained language tokens and $\Sigma_D$ domain tokens; then a user story is a sequence $s \in (\Sigma_L \cup \Sigma_D)^*$. The domain token categories include structural markers such as <|begin_sessions|> and <|session|>, event heads such as <|search|> and <|watch|>, temporal fields such as hour=H, day=D, elapsed=\Delta h, and duration=M m, surface tokens such as <|surface=home|> and <|surface=search|>, carousel tokens such as <|carousel(name)|>, <|carousel()|>, and <|carousel(MASK)|>, and atomic item tokens of the form <|id(ID|Title)|> together with <|id(UNK)|> for unknown or new items (Salle et al., 22 May 2026).

Prompts define the task by appending a structurally valid event head. For item ranking, the prompt appends a watch head and asks the model to predict the next item token; for carousel ranking, it appends the next surface context and predicts the next carousel token; for search ranking, it appends query tokens under <|search|> and a follow-up watch head under <|surface=search|>, then predicts the watched item token. Because all three tasks operate over the same grammar, the model can condition on the same serialized history while changing only the prompt (Salle et al., 22 May 2026).

The paper argues that this linearized representation exposes rewatch, continuation, and new discovery signals implicitly. The model observes prior watches, durations, surfaces, intervening searches, unmatched queries, and session gaps, then learns to interpret them through autoregressive prediction. This suggests that intent classification is not implemented as an explicit submodule, but emerges from next-token prediction over structured behavioral text.

3. Model architecture, vocabulary, and training procedure

TubiFM uses Llama 3.2 1B as a decoder-only Transformer backbone and finetunes it on user stories. The tokenizer is extended with atomic domain tokens for event types, fields, surfaces, carousels, and item IDs, covering approximately ~100k titles and ~1k carousels. Positional encoding and architecture remain unchanged from stock Llama, and the system introduces no task-specific heads or architectural modifications (Salle et al., 22 May 2026).

The training mixture contains two corpora. The primary corpus is the user-story corpus, built from ~20M viewers, with average story length ~560 tokens and ~11B serialized tokens before repeated sampling. Stories are truncated to context length 1024. The auxiliary catalog corpus consists of token-to-text alignment statements such as "<|id(#|title)|> has title {title}" and "<|carousel(name)|> has name {name}", with a mixture ratio of 20 : 1 between user stories and the catalog corpus. The stated purpose of this auxiliary corpus is to connect new domain tokens to language semantics, which is especially described as helpful for search (Salle et al., 22 May 2026).

Training uses standard autoregressive cross-entropy:

$$L(\theta) = -\sum_{t=1}^{T} \log p_\theta(x_t \mid x_{<t}),$$

with optimization settings of 120k macro-steps, sequence length 1024, per-device batch size 4, gradient accumulation 4, 8×H100 GPUs, bf16, gradient clipping 1.0, weight decay 0.033, an Adam-style optimizer, and learning rate 10^{-5} after 1000 warmup steps; total training time is reported as ~22 hours (Salle et al., 22 May 2026).

Two masking mechanisms are built into training. With probability $p_m = 0.1$, the carousel token is replaced by <|carousel(MASK)|>, supporting container-independent ranking. With probability $p_u = 0.001$, an item token may be replaced by <|id(UNK)|>, to tolerate catalog churn and new entries until the next token refresh. The paper presents both choices as serving-oriented robustness interventions rather than auxiliary pretext tasks (Salle et al., 22 May 2026).

An ablation on initialization compares Llama-based initialization against random initialization and reports gains from pretraining across tasks, for example item HR@8 0.5817 vs. 0.5670 and search HR@8 0.5673 vs. 0.5535. The paper also reports that coarse user attributes have relatively small effects, while session segmentation and elapsed/day fields materially help all tasks (Salle et al., 22 May 2026).

4. Ranking formulation and empirical performance

TubiFM ranks by scoring atomic domain tokens in a single forward pass. For a candidate $c$ with atomic token $\tau(c)$ and a user-story prefix $\pi$ including the task prompt, the next-token probability is

$$p_\theta(\tau(c)\mid \pi)=\mathrm{softmax}_\tau\, z_\theta(\pi)[\tau(c)],$$

and the ranking score is

$\Sigma_D$0

Because items and carousels are atomic tokens, there is no need to aggregate log-probabilities across multiple tokens. The paper reports ranking by raw logits or probabilities and does not report explicit calibration or temperature (Salle et al., 22 May 2026).

Offline evaluation is performed on a held-out 1% user split with HR@K and NDCG@K at K∈{8,50,100}. Item and carousel baselines are SASRec and HSTU; search baselines are BM25, Qwen3-4B Embeddings, and a Finetuned Sentence-Transformer. TubiFM is evaluated both as a task-specific finetune and as a unified model (Salle et al., 22 May 2026).

Task	Baseline highlight	TubiFM (unified)
Item ranking	HSTU: HR@8=0.4121, NDCG@8=0.3105	HR@8=0.5817, HR@50=0.7788, HR@100=0.8375; NDCG@8=0.4599, NDCG@50=0.5059, NDCG@100=0.5155
Carousel ranking	HSTU: HR@8=0.7724, NDCG@8=0.5362	HR@8=0.8343, HR@50=0.9859, HR@100=0.9961; NDCG@8=0.6366, NDCG@50=0.6747, NDCG@100=0.6764
Search ranking	BM25: HR@8=0.4637, NDCG@8=0.3732	HR@8=0.5673, HR@50=0.7635, HR@100=0.8243; NDCG@8=0.4480, NDCG@50=0.4937, NDCG@100=0.5036

The unified model also outperforms task-specific TubiFM finetunes in all three tasks. For example, task-specific item ranking reports HR@8=0.4975 and NDCG@8=0.3950, while the unified model reaches HR@8=0.5817 and NDCG@8=0.4599. The paper explicitly interprets this pattern as evidence that joint modeling of watches, carousels, and searches is beneficial (Salle et al., 22 May 2026).

Online A/B tests run for one week on millions of users with viewer-level randomization and significance via CURE at p\<0.05. Search total viewing time increases by +3.9%, with especially large gains on tail and long queries, including +20% [TVT](https://www.emergentmind.com/topics/transfer-vae-training-tvt) uplift on those segments. Carousel TVT increases by +0.30%, concentrated in top-position carousels. Item TVT is +0.14%, reported as statistically neutral versus a mature production stack. The paper defines total viewing time as

$\Sigma_D$1

These results suggest that the main measurable online benefit is strongest where search intent disambiguation and cross-surface context matter most (Salle et al., 22 May 2026).

5. Production serving and systems implications

TubiFM is explicitly presented as a productionized ranker. Online serving uses L40S GPUs, and the reported p99 request-to-ranked-list latency is reduced from ~500 ms in production baselines to ~200 ms across item, carousel, and search ranking. The paper attributes this to three factors: single forward-pass scoring over atomic item and carousel tokens, reduction of cross-component orchestration and feature plumbing by using one unified model, and dynamic batching during inference (Salle et al., 22 May 2026).

The serving formulation is tightly coupled to the vocabulary design. Because items and carousels are represented as atomic identifiers, the system can score the full domain vocabulary at the target position in one pass. The paper contrasts this with multi-stage recall plus rerank pipelines and with multi-token generation, noting that no beam search or multi-token decoding is required. A plausible implication is that the modeling choice of atomic IDs is not only a representational decision but a systems optimization that makes full-catalog ranking compatible with practical latency budgets.

The prompt interface also allows multiple ranking modes without model changes. Item ranking can be container-independent by masking carousel identity or contextual by specifying <|surface=…|><|carousel(…)|>. Carousel ranking predicts the next carousel token from surface context. Search ranking appends the raw query and a follow-up watch head from search, then predicts the watched item. This suggests that task switching occurs at the serialization layer rather than through specialized deploy-time routing to different architectures (Salle et al., 22 May 2026).

The paper does not provide explicit throughput or memory figures. It states only that, in practice, a 1B-parameter model with atomic-ID vocabularies and dynamic batching offers favorable latency–cost trade-offs for production ranking compared with multi-stage pipelines, especially when the catalog can be scored in one pass (Salle et al., 22 May 2026).

6. Baselines, limitations, and prospective extensions

The baseline set clarifies TubiFM’s positioning. For item and carousel ranking, SASRec is described as causal self-attention over ID sequences and HSTU as a time-aware sequential Transformer; both are characterized as task-specific and unable to leverage searches, surfaces, and sessions without bespoke feature engineering. For search, the baselines are BM25, zero-shot Qwen3-4B embeddings, and a supervised dense sentence-transformer. The paper attributes TubiFM’s offline gains to three factors: joint modeling of cross-task evidence, atomic identifier scoring over the full catalog or carousel space, and the catalog corpus that links domain tokens to language semantics (Salle et al., 22 May 2026).

The paper also states several limitations. Offline labels derive from watches, so implicit-feedback bias and exposure bias remain. Search evaluation uses watched-after-search positives and ignores queries without watches. Cold-start users with extremely sparse histories may receive weaker personalization, although attribute priors and language priors partially mitigate this. New items are mapped to UNK until token refresh, which can reduce fine-grained search precision. Privacy considerations remain, since coarse user attributes and raw query tokens require careful logging and tokenization to avoid sensitive data leakage. Finally, domain drift is an ongoing concern; the paper notes that daily retraining tracks shifts, but systematic drift could still degrade performance between refreshes (Salle et al., 22 May 2026).

Future directions identified in the paper include larger models or distillations under similar latency budgets, richer token schemas such as fine-grained content facets and episode or series structure, better temporal representations, and hybrid retrieval-augmented methods including constrained decoding or memory modules while preserving the unified user-story interface. This suggests that the “user story” abstraction is intended as a reusable interface rather than a one-off architecture: adding signals may require adding tokens rather than rebuilding task-specific models (Salle et al., 22 May 2026).