--- title: 'TubiFM: Unified Streaming Ranking' url: https://www.emergentmind.com/topics/tubifm type: topic --- # TubiFM: Unified Streaming Ranking TubiFM is a unified ranking system for streaming discovery that models item ranking, carousel ranking, and search ranking within a single large-language-model framework. Rather than training separate production models for each surface, it represents a viewer’s cross-surface behavior as a serialized “user story” and expresses heterogeneous ranking tasks as prompted next-token prediction over a shared grammar. In the reported instantiation, TubiFM is a finetuned Llama 3.2 1B model trained on user stories containing user attributes, sessions, watch events with surface and carousel context, and search events, including search-as-you-type prefixes and queries that do not lead to a watch [2605.23702]. ## 1. Problem formulation and system objective TubiFM addresses three coupled ranking tasks in streaming discovery: item ranking, carousel ranking, and search ranking. The paper frames these tasks as arising from the same viewer journey but being conventionally handled by separate models and serving stacks. That separation is described as fragmenting complementary signals across and within sessions: searches can reveal topical intent even when they do not lead to a watch, searches may express rewatch or continuation intent, and watches provide feedback about which carousels and future search results should be prominent [2605.23702]. The central design choice is to collapse these signals into one modeling interface. Instead of using task-specific architectures or heads, TubiFM uses prompted next-token prediction over a single sequence representation. This makes item, carousel, and search ranking different task prompts applied to the same prefix, rather than distinct model families. A plausible implication is that the system treats discovery not as a set of loosely coupled recommenders, but as a single sequential inference problem in which watches, searches, session boundaries, and placement context are jointly informative. The paper identifies three main contributions. First, it introduces the unified “user story” representation. Second, it defines a shared grammar over interleaved pretrained language tokens and domain-specific event tokens. Third, it presents a single productionized LLM ranker that outperforms specialist baselines offline across item, carousel, and search, while yielding significant online gains for search and carousel ranking and statistically neutral item ranking relative to a mature production stack [2605.23702]. ## 2. User stories and the shared grammar A user story is an ordered, flat token sequence. Its components are a user attribute header, session markers, and chronological interaction events. The attribute header contains coarse fields such as country and device serialized as ordinary text tokens. Sessions are marked by `<|session|>`, with a new session created if inactivity exceeds 1 hour and sessions capped at 12 hours. Each session also includes elapsed time since the previous session and day-of-week. Within sessions, watch events encode hour-of-day, originating surface, carousel identity, item identifier, and viewing duration, while search events encode hour-of-day and the raw query text for every search, including search-as-you-type prefixes and queries that do not lead to watches [2605.23702]. This serialization is governed by a formal grammar. Let $\Sigma_L$ be pretrained language tokens and $\Sigma_D$ domain tokens; then a user story is a sequence $s \in (\Sigma_L \cup \Sigma_D)^*$. The domain token categories include structural markers such as `<|begin_sessions|>` and `<|session|>`, event heads such as `<|search|>` and `<|watch|>`, temporal fields such as `hour=H`, `day=D`, `elapsed=\Delta h`, and `duration=M m`, surface tokens such as `<|surface=home|>` and `<|surface=search|>`, carousel tokens such as `<|carousel(name)|>`, `<|carousel()|>`, and `<|carousel(MASK)|>`, and atomic item tokens of the form `<|id(ID|Title)|>` together with `<|id(UNK)|>` for unknown or new items [2605.23702]. Prompts define the task by appending a structurally valid event head. For item ranking, the prompt appends a watch head and asks the model to predict the next item token; for carousel ranking, it appends the next surface context and predicts the next carousel token; for search ranking, it appends query tokens under `<|search|>` and a follow-up watch head under `<|surface=search|>`, then predicts the watched item token. Because all three tasks operate over the same grammar, the model can condition on the same serialized history while changing only the prompt [2605.23702]. The paper argues that this linearized representation exposes rewatch, continuation, and new discovery signals implicitly. The model observes prior watches, durations, surfaces, intervening searches, unmatched queries, and session gaps, then learns to interpret them through autoregressive prediction. This suggests that intent classification is not implemented as an explicit submodule, but emerges from next-token prediction over structured behavioral text. ## 3. Model architecture, vocabulary, and training procedure TubiFM uses Llama 3.2 1B as a decoder-only Transformer backbone and finetunes it on user stories. The tokenizer is extended with atomic domain tokens for event types, fields, surfaces, carousels, and item IDs, covering approximately `~100k titles` and `~1k carousels`. Positional encoding and architecture remain unchanged from stock Llama, and the system introduces no task-specific heads or architectural modifications [2605.23702]. The training mixture contains two corpora. The primary corpus is the user-story corpus, built from `~20M viewers`, with average story length `~560 tokens` and `~11B serialized tokens before repeated sampling`. Stories are truncated to context length `1024`. The auxiliary catalog corpus consists of token-to-text alignment statements such as `"<|id(#|title)|> has title {title}"` and `"<|carousel(name)|> has name {name}"`, with a mixture ratio of `20 : 1` between user stories and the catalog corpus. The stated purpose of this auxiliary corpus is to connect new domain tokens to language semantics, which is especially described as helpful for search [2605.23702]. Training uses standard autoregressive cross-entropy: $$ L(\theta) = -\sum_{t=1}^{T} \log p_\theta(x_t \mid x_{`, supporting container-independent ranking. With probability $p_u = 0.001$, an item token may be replaced by `<|id(UNK)|>`, to tolerate catalog churn and new entries until the next token refresh. The paper presents both choices as serving-oriented robustness interventions rather than auxiliary pretext tasks [2605.23702]. An ablation on initialization compares Llama-based initialization against random initialization and reports gains from pretraining across tasks, for example item `HR@8 0.5817 vs. 0.5670` and search `HR@8 0.5673 vs. 0.5535`. The paper also reports that coarse user attributes have relatively small effects, while session segmentation and elapsed/day fields materially help all tasks [2605.23702]. ## 4. Ranking formulation and empirical performance TubiFM ranks by scoring atomic domain tokens in a single forward pass. For a candidate $c$ with atomic token $\tau(c)$ and a user-story prefix $\pi$ including the task prompt, the next-token probability is $$ p_\theta(\tau(c)\mid \pi)=\mathrm{softmax}_\tau\, z_\theta(\pi)[\tau(c)], $$ and the ranking score is $$ s(c)=\log p_\theta(\tau(c)\mid \pi). $$ Because items and carousels are atomic tokens, there is no need to aggregate log-probabilities across multiple tokens. The paper reports ranking by raw logits or probabilities and does not report explicit calibration or temperature [2605.23702]. Offline evaluation is performed on a held-out `1% user split` with `HR@K` and `NDCG@K` at `K∈{8,50,100}`. Item and carousel baselines are SASRec and HSTU; search baselines are BM25, Qwen3-4B Embeddings, and a Finetuned Sentence-Transformer. TubiFM is evaluated both as a task-specific finetune and as a unified model [2605.23702]. | Task | Baseline highlight | TubiFM (unified) | |---|---|---| | Item ranking | HSTU: `HR@8=0.4121`, `NDCG@8=0.3105` | `HR@8=0.5817`, `HR@50=0.7788`, `HR@100=0.8375`; `NDCG@8=0.4599`, `NDCG@50=0.5059`, `NDCG@100=0.5155` | | Carousel ranking | HSTU: `HR@8=0.7724`, `NDCG@8=0.5362` | `HR@8=0.8343`, `HR@50=0.9859`, `HR@100=0.9961`; `NDCG@8=0.6366`, `NDCG@50=0.6747`, `NDCG@100=0.6764` | | Search ranking | BM25: `HR@8=0.4637`, `NDCG@8=0.3732` | `HR@8=0.5673`, `HR@50=0.7635`, `HR@100=0.8243`; `NDCG@8=0.4480`, `NDCG@50=0.4937`, `NDCG@100=0.5036` | The unified model also outperforms task-specific TubiFM finetunes in all three tasks. For example, task-specific item ranking reports `HR@8=0.4975` and `NDCG@8=0.3950`, while the unified model reaches `HR@8=0.5817` and `NDCG@8=0.4599`. The paper explicitly interprets this pattern as evidence that joint modeling of watches, carousels, and searches is beneficial [2605.23702]. Online A/B tests run for one week on millions of users with viewer-level randomization and significance via CURE at `p<0.05`. Search total viewing time increases by `+3.9%`, with especially large gains on tail and long queries, including `+20% TVT uplift on those segments`. Carousel TVT increases by `+0.30%`, concentrated in top-position carousels. Item TVT is `+0.14%`, reported as statistically neutral versus a mature production stack. The paper defines total viewing time as $$ \mathrm{TVT}=\sum_{u\in U}\sum_{s\in S(u)} \mathrm{duration}_{u,s}. $$ These results suggest that the main measurable online benefit is strongest where search intent disambiguation and cross-surface context matter most [2605.23702]. ## 5. Production serving and systems implications TubiFM is explicitly presented as a productionized ranker. Online serving uses L40S GPUs, and the reported `p99` request-to-ranked-list latency is reduced from `~500 ms` in production baselines to `~200 ms` across item, carousel, and search ranking. The paper attributes this to three factors: single forward-pass scoring over atomic item and carousel tokens, reduction of cross-component orchestration and feature plumbing by using one unified model, and dynamic batching during inference [2605.23702]. The serving formulation is tightly coupled to the vocabulary design. Because items and carousels are represented as atomic identifiers, the system can score the full domain vocabulary at the target position in one pass. The paper contrasts this with multi-stage recall plus rerank pipelines and with multi-token generation, noting that no beam search or multi-token decoding is required. A plausible implication is that the modeling choice of atomic IDs is not only a representational decision but a systems optimization that makes full-catalog ranking compatible with practical latency budgets. The prompt interface also allows multiple ranking modes without model changes. Item ranking can be container-independent by masking carousel identity or contextual by specifying `<|surface=…|><|carousel(…)|>`. Carousel ranking predicts the next carousel token from surface context. Search ranking appends the raw query and a follow-up watch head from search, then predicts the watched item. This suggests that task switching occurs at the serialization layer rather than through specialized deploy-time routing to different architectures [2605.23702]. The paper does not provide explicit throughput or memory figures. It states only that, in practice, a `1B-parameter` model with atomic-ID vocabularies and dynamic batching offers favorable latency–cost trade-offs for production ranking compared with multi-stage pipelines, especially when the catalog can be scored in one pass [2605.23702]. ## 6. Baselines, limitations, and prospective extensions The baseline set clarifies TubiFM’s positioning. For item and carousel ranking, SASRec is described as causal self-attention over ID sequences and HSTU as a time-aware sequential Transformer; both are characterized as task-specific and unable to leverage searches, surfaces, and sessions without bespoke feature engineering. For search, the baselines are BM25, zero-shot Qwen3-4B embeddings, and a supervised dense sentence-transformer. The paper attributes TubiFM’s offline gains to three factors: joint modeling of cross-task evidence, atomic identifier scoring over the full catalog or carousel space, and the catalog corpus that links domain tokens to language semantics [2605.23702]. The paper also states several limitations. Offline labels derive from watches, so implicit-feedback bias and exposure bias remain. Search evaluation uses watched-after-search positives and ignores queries without watches. Cold-start users with extremely sparse histories may receive weaker personalization, although attribute priors and language priors partially mitigate this. New items are mapped to `UNK` until token refresh, which can reduce fine-grained search precision. Privacy considerations remain, since coarse user attributes and raw query tokens require careful logging and tokenization to avoid sensitive data leakage. Finally, domain drift is an ongoing concern; the paper notes that daily retraining tracks shifts, but systematic drift could still degrade performance between refreshes [2605.23702]. Future directions identified in the paper include larger models or distillations under similar latency budgets, richer token schemas such as fine-grained content facets and episode or series structure, better temporal representations, and hybrid retrieval-augmented methods including constrained decoding or memory modules while preserving the unified user-story interface. This suggests that the “user story” abstraction is intended as a reusable interface rather than a one-off architecture: adding signals may require adding tokens rather than rebuilding task-specific models [2605.23702]. Source: https://www.emergentmind.com/topics/tubifm