Hierarchical Sequential Transducers (HSTU)

• HSTU is an attention-based sequential transducer that reformulates recommendation tasks as a unified sequence modeling of user actions, content items, and contextual features.
• It employs a novel SiLU-based attention mechanism with explicit temporal bias to effectively handle high-cardinality, non-stationary streaming data.
• The architecture scales to trillions of parameters, delivering significant improvements in ranking, retrieval, and inference efficiency over traditional recommendation models.

Hierarchical Sequential Transducers (HSTU) are attention-based sequential transducers for generative recommendation. Introduced by Zhai et al. within the “Generative Recommenders” framework, HSTU reformulates ranking and retrieval as sequential transduction tasks over a unified stream of user actions, content items, and categorical context, and is explicitly designed for high cardinality, heterogeneous, non-stationary streaming recommendation data. In reported industrial deployments, HSTU-based Generative Recommenders scale to 1.5 trillion parameters, exhibit empirical power-law scaling in training compute, and have been deployed on surfaces serving billions of users (Zhai et al., 2024).

1. Sequential transduction formulation

HSTU is defined in the setting of sequential transduction. A chronologically ordered input sequence

$x_0, x_1,\ldots,x_{n-1}, \quad x_i \in \mathbb{X}$

is mapped to an output sequence

$$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$

where $y_i=\varnothing$ denotes positions without a target. In recommendation, contents $\Phi_0,\ldots,\Phi_{n_c-1}$ and user actions $a_0,\ldots,a_{n_c-1}$ are merged into a single time series of tokens $x_0,\ldots,x_{n-1}$. Ranking is cast through an interleaved formulation such as

$$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$

with outputs defined only at content-token positions, while retrieval is cast over tuples $(\Phi_i,a_i)$ and positive-only targets (Zhai et al., 2024).

This formulation addresses several industrial constraints simultaneously. The motivating regime is characterized by extreme cardinality and dynamic vocabularies, heterogeneous sparse and dense features, non-stationary streaming data, and interaction histories reaching $10^4$–$10^5$ events. The original HSTU work positions this reformulation against conventional DLRMs, arguing that many industrial recommendation systems saturate in quality as compute grows, whereas a generative sequential architecture can amortize encoder computation across multiple targets per history and recover LLM-like scaling behavior (Zhai et al., 2024).

2. Layer mechanics and attention design

A canonical HSTU layer maps an input sequence representation $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$0 to an output $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$1 in three stages: $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$2

$$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$3

$$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$4

Here $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$5 and $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$6 are linear layers, $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$7 and $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$8 are SiLU nonlinearities, $$y_0, y_1,\ldots,y_{n-1}, \quad y_i \in \mathbb{X}\cup\{\varnothing\},$$9 is layer normalization, $y_i=\varnothing$0 is elementwise multiplication, and $y_i=\varnothing$1 is a relative attention bias encoding positional and temporal differences between tokens (Zhai et al., 2024).

This mechanism differs materially from a standard Transformer. HSTU replaces softmax attention with a pointwise SiLU transformation over $y_i=\varnothing$2, then normalizes the pooled values and gates them multiplicatively by $y_i=\varnothing$3. In the original formulation, this preserves absolute intensity information that softmax normalization can suppress, which is presented as important for non-stationary, large-vocabulary recommendation data. The architecture also reduces feed-forward overhead relative to standard Transformer blocks by merging attention output and gating into a lighter pointwise transformation (Zhai et al., 2024).

Later system-oriented descriptions preserve the same core structure while making the residualized block explicit. One such formulation writes

$y_i=\varnothing$4

$y_i=\varnothing$5

$y_i=\varnothing$6

$y_i=\varnothing$7

$y_i=\varnothing$8

with $y_i=\varnothing$9 as the causal mask. In context-parallel implementations, the attention score also includes a customized timestamp bias generated from bucketized time differences, yielding a temporal-aware HSTUAttention of the form

$\Phi_0,\ldots,\Phi_{n_c-1}$0

This establishes HSTU as a Transformer-like sequence model, but one with SiLU-based attention, explicit temporal biasing, and jagged-tensor-oriented implementation assumptions (Dong et al., 23 Jul 2025).

3. Input representation, generative training, and inference

In the original Generative Recommender formulation, the input is a unified time series built from interleaved content and action tokens together with merged categorical features such as demographics or followed creators. Slowly changing categorical features are compressed by segments and merged into the main sequence along the time axis so that total sequence length remains roughly $\Phi_0,\ldots,\Phi_{n_c-1}$1. Numerical features are omitted from the explicit sequence in this formulation; the stated design assumption is that sufficiently expressive sequential transduction over longer histories can implicitly recover the corresponding statistics (Zhai et al., 2024).

Training is autoregressive. For an interleaved sequence,

$\Phi_0,\ldots,\Phi_{n_c-1}$2

the generative model factorizes as

$\Phi_0,\ldots,\Phi_{n_c-1}$3

In ranking, the model predicts action probabilities at content positions; in retrieval, it predicts future content items given the evolving user state. In the context-parallel description of the GR framework, this appears equivalently as

$\Phi_0,\ldots,\Phi_{n_c-1}$4

with the HSTU encoder producing the conditional representation $\Phi_0,\ldots,\Phi_{n_c-1}$5 and normalized entropy used as a quality indicator (Dong et al., 23 Jul 2025).

Inference for ranking is target-aware. A candidate content token is appended to the user history, HSTU runs with a causal mask, and the hidden state at the candidate position is sent to a ranking head that predicts multiple engagement events. The serving system described in the original work uses M-FALCON, in which candidates are grouped into microbatches and the user-history $\Phi_0,\ldots,\Phi_{n_c-1}$6 cache is reused across microbatches; masking prevents cross-attention among candidates inside the same microbatch, so each candidate depends only on the user history (Zhai et al., 2024).

4. Efficiency, scaling laws, and systems co-design

HSTU was introduced partly as an efficiency intervention. Even without changing the $\Phi_0,\ldots,\Phi_{n_c-1}$7 asymptotics of self-attention, the original design reduces constant factors through fused $\Phi_0,\ldots,\Phi_{n_c-1}$8 projections, lighter pointwise transformation, and a ragged grouped-GEMM attention kernel. It also introduces Stochastic Length, which samples shorter subsequences during training so that effective complexity becomes

$\Phi_0,\ldots,\Phi_{n_c-1}$9

In the reported H100 benchmarks at sequence lengths up to 8192, HSTU is 5.3x to 15.2x faster than FlashAttention2-based Transformers in training and up to 5.6x faster in inference. The same work reports activation memory of roughly $a_0,\ldots,a_{n_c-1}$0 units per layer for HSTU versus at least $a_0,\ldots,a_{n_c-1}$1 in a typical Transformer configuration, enabling deeper stacks under the same memory budget. On public sequential recommendation datasets, HSTU-large improves NDCG@10 by 18.1% on ML-1M, 30.0% on ML-20M, and 65.8% on Amazon Books relative to SASRec; at industrial scale, HSTU-based GRs are reported at 1.5 trillion parameters and deliver online gains of +12.4% main engagement and +4.4% consumption when replacing a production DLRM ranking source (Zhai et al., 2024).

Scaling sequence length produced a distinct systems line of work. Context parallelism for HSTU attention with jagged tensor support distributes sequence computation across GPUs without changing HSTU’s mathematical formulation. On 80GB H100 GPUs, the reported maximum supported sequence length rises from 3072 without context parallelism to 16384 with CP size 8, a 5.3x increase. The implementation replaces AllGather with AllToAll for jagged tensor segments, reducing peak memory by more than 60% and improving throughput by more than 2x examples per second; load-balanced mini-chunking adds a further 37% throughput improvement, and avoiding host-device synchronization on sequence-length retrieval adds about 2% QPS. When combined with DDP at cp_size=2 and doubled batch size, the reported scaling factor is 1.55x relative to a 24K QPS baseline (Dong et al., 23 Jul 2025).

ULTRA-HSTU extends the architecture through model and system co-design. It redesigns the sequence as item–action fusion,

$a_0,\ldots,a_{n_c-1}$2

introduces Semi-Local Attention with complexity

$a_0,\ldots,a_{n_c-1}$3

uses Attention Truncation so later layers operate only on the most recent $a_0,\ldots,a_{n_c-1}$4 positions, and deploys Mixture of Transducers over different behavior streams. In the reported industrial benchmark at $a_0,\ldots,a_{n_c-1}$5 and 18 layers, ULTRA-HSTU reaches $a_0,\ldots,a_{n_c-1}$6, Train TFLOP $a_0,\ldots,a_{n_c-1}$7 versus $a_0,\ldots,a_{n_c-1}$8 for HSTU, and Inference TFLOP $a_0,\ldots,a_{n_c-1}$9 versus $x_0,\ldots,x_{n-1}$0. Fitted scaling curves attribute 5.3x higher training scaling efficiency and 21.4x higher inference scaling efficiency to ULTRA-HSTU. The model is described as fully deployed at Meta, serving billions of users daily, with reported online uplifts of +4.11% on a consumption metric, +2.27%, +8.2%, and +4.34% on engagement metrics, and +0.217% and +0.037% on top-line metrics (Ding et al., 19 Feb 2026).

5. Variants, augmentations, and backbone reuse

Later work frequently treats HSTU as a reusable backbone rather than a closed architecture. One direction compresses old history while preserving HSTU as the decoder-style generative recommender. In the personalization framework of PerSRec, a long user interaction history is segmented, learnable tokens $x_0,\ldots,x_{n-1}$1 are inserted after earlier segments, and a modified attention mask forces cross-segment communication to pass through those tokens. The underlying HSTU block is left unchanged, the next-item prediction objective is preserved, and positions corresponding to learnable tokens are excluded from the loss. For the HSTU setting with $x_0,\ldots,x_{n-1}$2, $x_0,\ldots,x_{n-1}$3, $x_0,\ldots,x_{n-1}$4, and $x_0,\ldots,x_{n-1}$5, the theoretical inference-compute ratio is reported as $x_0,\ldots,x_{n-1}$6. On MerRec, HSTU 256 yields R@10 $x_0,\ldots,x_{n-1}$7 and N@10 $x_0,\ldots,x_{n-1}$8, HSTU 1280 yields R@10 $x_0,\ldots,x_{n-1}$9 and N@10 $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$0, and Personalized HSTU yields R@10 $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$1 and N@10 $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$2. Train time is 376 sec/epoch for Personalized HSTU versus 360 sec/epoch for HSTU 1280, a reported +4.4%, while inference time remains nearly identical at 8.70 versus 8.78 sec/epoch. On EB-NeRD, Personalized HSTU reaches R@50 $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$3 and N@50 $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$4, above HSTU 500 at 61.80% and 36.08% (Zhang et al., 7 Jan 2026).

A second direction enriches HSTU’s item representation with external semantics while leaving the transducer unchanged. HSTU-BLaIR augments the input embedding with a projected text embedding,

$$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$5

followed by

$$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$6

The reported configuration uses four transformer blocks and four attention heads, keeps BLaIR embeddings fixed, and trains all models for 100 epochs with sampled softmax and text-informed negative sampling. On Amazon Video Games, HR@10 progresses from 0.1315 for HSTU to 0.1328 for HSTU-OpenAI and 0.1353 for HSTU-BLaIR; NDCG@10 moves from 0.0741 to 0.0742 to 0.0760. On Office Products, HR@10 is 0.0395 for HSTU, 0.0477 for HSTU-OpenAI, and 0.0484 for HSTU-BLaIR, with corresponding NDCG@10 values 0.0223, 0.0269, and 0.0271. In this usage, HSTU functions as a stable generative backbone onto which richer semantic embeddings are projected (Liu, 13 Apr 2025).

Variant	Modification	Reported effect
Personalized HSTU	Segment compression with learnable tokens and modified attention mask	Matches or slightly exceeds full-history HSTU on MerRec and EB-NeRD
HSTU-BLaIR	Add projected contrastive text embeddings to item embeddings	Improves HR@10 and NDCG@10 over both HSTU and HSTU-OpenAI on the reported datasets

6. Competing interpretations of “hierarchy” and current research directions

A central conceptual dispute concerns what “hierarchical” means in HSTU. In one later taxonomy, HSTU is treated as the canonical flat-sequence generative recommender: a Transformer-like model operating on the entire user history as a flat sequence of item or action tokens with dense self-attention. From that perspective, HSTU’s success is said to rest on a “flat-sequence” assumption that neglects explicit user $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$7 sessions $$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$8 items structure. The claimed limitations are threefold: failure to capture the temporal hierarchy of session-based engagement, computational inefficiency of dense attention over long histories, and noise from semantically sparse histories. HPGR is proposed as a response by adding a Session Enhancement Module, Masked Item Modeling pre-training, time-aware positional encoding with candidate insertion, and Preference-Guided Sparse Attention based on Top-$$x_i=\Phi_0,a_0,\Phi_1,a_1,\ldots,\Phi_{n_c-1},a_{n_c-1},$$9 preference anchors and a Gaussian mask. On the sampled APPGallery dataset, the reported AUC values are 0.8220 for HSTU, 0.8253 for MTGR, 0.8327 for HPGR Base, 0.8348 for Base + PGSA, 0.8374 for Base + SEM, and 0.8377 for Full HPGR; on a full-dataset online A/B test, HPGR reports +1.99% eCPM uplift versus a production DLRM (Chen et al., 1 Mar 2026).

This critique has clarified a recurring ambiguity in the HSTU literature. In the original and systems-oriented lines, “hierarchy” refers primarily to multi-layer composition, multi-scale temporal processing, heterogeneous feature aggregation, or later topology choices such as Attention Truncation and Mixture of Transducers. In the HPGR line, by contrast, HSTU is explicitly not hierarchical in the sense of an explicit session tree. This suggests that the term “hierarchical” in HSTU is architectural rather than ontological: it describes how the transducer processes sequence information, not necessarily how user behavior is explicitly decomposed into user-, session-, and item-level structures.

Several open problems remain active. The original HSTU work notes that the architecture remains quadratic in worst-case sequence length despite Stochastic Length; it also identifies fully generative listwise recommendation, richer long-term memory, cross-modal context integration, and theoretical understanding of pointwise attention as unresolved directions. Subsequent work indicates two broad trajectories: making HSTU-style self-attention tractable at ultra-long context through sparse masks, truncation, and parallelism, and injecting stronger structural priors such as session hierarchy or preference-aware sparsity into the HSTU backbone (Zhai et al., 2024).