ULTRA-HSTU: Generative Transduction Model

• ULTRA-HSTU is a large-scale sequential transduction architecture that encodes ultra-long user histories as unified token sequences.
• It integrates attention sparsification and system-level co-optimization to achieve high efficiency and superior recommendation quality.
• The model reformulates recommendation as a fully generative, autoregressive sequence transduction problem, offering practical benefits over traditional DLRMs.

ULTRA-HSTU is a large-scale sequential transduction architecture designed for generative recommendation systems, with a primary focus on handling high-cardinality, ultra-long user histories in streaming recommendation tasks. By integrating modeling, input construction, attention sparsification, and system-level co-optimization, ULTRA-HSTU achieves unprecedented parameter scale, efficiency, and recommendation quality in industry-scale deployments (Zhai et al., 2024, Ding et al., 19 Feb 2026).

1. Generative Sequential Transduction Reformulation

ULTRA-HSTU reconceptualizes recommendation as a fully generative, autoregressive sequence transduction problem. Unlike traditional Deep Learning Recommendation Models (DLRMs) that rely on pointwise scoring over engineered dense/sparse feature sets and emit <user,item,label> triplets, ULTRA-HSTU encodes the entire user interaction history as a unified sequence of tokens: $<$Content₀, Action₀, Content₁, Action₁, …$>,$ interleaved with categorical side-features