UserU: Industrial-Scale User Understanding Dataset

• UserU is a multi-modal pre-training dataset that integrates dynamic Alipay user behaviors with semantic query supervision.
• It employs dual strategies—future behavior prediction and synthetic QA—to construct scenario-adaptive user embeddings.
• A hierarchical encoding framework transforms 90-day multi-modal profiles into detailed tokens, improving downstream task performance.

Searching arXiv for the specified paper and closely related work on UserU / Query-as-Anchor. UserU is an industrial-scale pre-training dataset introduced as part of the “Query as Anchor” framework for scenario-adaptive user representation via LLMs. It is designed to align heterogeneous user behaviors with user understanding semantics through two complementary supervision sources: future behavior prediction and query–answer supervision. In the underlying formulation, UserU is not treated as a raw behavioral log repository, but as a structured data engine that couples multi-modal behavioral sequences from the Alipay ecosystem with natural-language queries and answers, thereby supplying both behavioral priors and semantic priors for user embedding learning (Yuan et al., 16 Feb 2026).

1. Definition and scope

UserU is defined as “the User Understanding (UserU) Pretraining Dataset, which integrates dynamic, context-aware user behavior with task adaptability for real-world applications” (Yuan et al., 16 Feb 2026). Its stated function is to align “heterogeneous user behaviors with user understanding semantics via future behavior prediction and QA-based supervision,” thereby supporting user representation learning with both predictive and semantic structure (Yuan et al., 16 Feb 2026).

The dataset is constructed around multi-modal user profiles from Alipay. For each user, the profile is denoted as

$$\mathbf{u}_i = \{ Bill_i, Mini_i, Spm_i, App_i, Search_i, Tabular_i \} \in \mathcal{U}.$$

This profile aggregates a 90-day behavioral window and serves as the basis for both pre-training tasks in UserU (Yuan et al., 16 Feb 2026).

The paper situates UserU within an industrial problem setting in which a single user representation must be sufficiently universal to transfer across downstream tasks, yet sufficiently scenario-adaptive to respond differently under different business queries. In that framing, existing static, task-agnostic embeddings are described as inadequate for reconciling divergent downstream requirements within unified vector spaces, especially under multi-source, heterogeneous, noisy, and sparse behavioral logs (Yuan et al., 16 Feb 2026). This suggests that UserU is intended as an intermediate representation layer between raw platform activity and query-conditioned user semantics.

2. Data sources, profile structure, and modalities

UserU is built entirely from the Alipay ecosystem and draws on six explicitly listed modalities: PayBill transactions, Mini Program interactions, SPM paths, App list signals, Search queries, and Tabular features (Yuan et al., 16 Feb 2026). The data therefore spans payments, app navigation, search behavior, and structured attributes within a single user profile.

The paper describes this as an industrial-scale construction. It states that $\mathcal{D}_{\text{train}}$ contains approximately $1.024 \times 10^8$ training pairs, and that pre-training uses a global batch size of 2,048 and up to 50k steps, corresponding to roughly 102.4 million samples (Yuan et al., 16 Feb 2026). It also characterizes the deployment setting as covering a very large user population and notes that each user is represented by a 90-day multi-modal behavioral profile (Yuan et al., 16 Feb 2026).

At the representation level, UserU does not directly present raw logs to the LLM. Instead, each modality is converted into dense embeddings by modality-specific gte-base encoders, after which event-level, modality-level, and user-level tokens are formed through a hierarchical coarse-to-fine encoding pipeline (Yuan et al., 16 Feb 2026). The resulting structured prefix is denoted $\mathbf{e}_i$ and is later combined with a query for query-anchored representation learning.

The modality schema can be summarized as follows.

Modality	Description in UserU
PayBill	payments and transaction logs
Mini Program	interactions with mini-apps inside Alipay
SPM	navigation paths across in-app components
App list	app-level list signals
Search	on-platform search queries
Tabular features	structured numeric and categorical features

The paper also states that the hierarchy is designed so that “the LLM can attend to either fine-grained events or high-level behavior summaries conditioned on the query, while remaining compatible with its native embedding space” (Yuan et al., 16 Feb 2026). A plausible implication is that UserU is organized not only to preserve behavioral detail, but also to make that detail query-selectable at inference time.

3. Dataset construction and supervision design

UserU comprises two main datasets: a behavior-based interaction dataset $\mathcal{D}_{\text{future}}$ for future behavior prediction, and a synthetic query–answer dataset $\mathcal{D}_{\text{uqa}}$ for user understanding (Yuan et al., 16 Feb 2026). The combination is central to the paper’s claim that UserU aligns behavioral sequences with user understanding semantics.

For $\mathcal{D}_{\text{future}}$, the input is a three-month user profile $\mathbf{u}_i$. The target is a structured future summary $a_i^{\mathrm{future}}$, derived from subsequent interactions beyond the 90-day observation window. Future logs are aggregated into temporal bins and action categories, and actions are selected using both frequency and diversity to form a representative subset (Yuan et al., 16 Feb 2026). The associated query is a fixed template, exemplified by the prompt “What are the user's most likely actions in the next period?” (Yuan et al., 16 Feb 2026). The resulting dataset is written as

$$\mathcal{D}_{\text{future}} = \{(\mathbf{u}_i \oplus q^{\mathrm{future}},\, a_i^{\mathrm{future}})\}_{i=1}^{N}.$$

For $\mathcal{D}_{\text{train}}$0, the paper constructs a synthetic UserQA corpus. It first creates a seed topic pool $\mathcal{D}_{\text{train}}$1 containing 72 life-related user-understanding topics using Qwen-Max, with examples including financial planning and health management (Yuan et al., 16 Feb 2026). For each user profile $\mathcal{D}_{\text{train}}$2, an LLM retrieves the top-10 most relevant topics from $\mathcal{D}_{\text{train}}$3, instantiates a naturalistic query $\mathcal{D}_{\text{train}}$4, and then generates an answer $\mathcal{D}_{\text{train}}$5 conditioned on $\mathcal{D}_{\text{train}}$6 (Yuan et al., 16 Feb 2026). A reflection or verification stage subsequently revises unsupported or inconsistent statements to improve faithfulness (Yuan et al., 16 Feb 2026). The resulting dataset is written as

$\mathcal{D}_{\text{train}}$7

The common rendering template supplied to the LLM is:

$\mathbf{e}_i$9

Within this template, e_i is the structured embedding prefix, the query is appended at the end, and <USER_EMB> marks the position from which a unified user embedding is extracted (Yuan et al., 16 Feb 2026).

4. Hierarchical encoding and representation interface

UserU relies on a hierarchical tokenization interface that mediates between multi-modal industrial logs and the LLM. For each modality $\mathcal{D}_{\text{train}}$8, raw event embeddings $\mathcal{D}_{\text{train}}$9 are adapted through modality-specific MLPs: $1.024 \times 10^8$0 These event-level tokens are mean-pooled within each modality: $1.024 \times 10^8$1 and then passed through a shared modality adapter: $1.024 \times 10^8$2 A user-level representation $1.024 \times 10^8$3 is then obtained by aggregating the modality embeddings through a user adapter (Yuan et al., 16 Feb 2026).

The final hierarchical token sequence is denoted $1.024 \times 10^8$4. It concatenates user-level, modality-level, and event-level information into a prefix supplied to the LLM before the query (Yuan et al., 16 Feb 2026). Although the paper notes formatting glitches in the displayed notation, its intent is explicit: the sequence includes a user token, followed by modality tokens, followed by event tokens (Yuan et al., 16 Feb 2026).

The placement of the query at the sequence terminus is architecturally significant. The model first processes the user tokens and then the query, allowing the decoder to attend across the prefix conditioned on the downstream question (Yuan et al., 16 Feb 2026). This suggests that the hierarchy is not merely a compression scheme; it is a mechanism for selective retrieval over user evidence at multiple granularities.

5. Role in Query-as-Anchor pre-training

UserU serves as the pre-training substrate for the Q-Anchor Embedding model, which uses a dual-tower LLM architecture with shared weights (Yuan et al., 16 Feb 2026). One tower, the Anchor Tower, processes $1.024 \times 10^8$5 to produce a query-conditioned user embedding

$1.024 \times 10^8$6

while the Semantic Tower processes the answer $1.024 \times 10^8$7 to produce

$1.024 \times 10^8$8

Because the towers share parameters, the anchor and semantic outputs are learned within a common latent space (Yuan et al., 16 Feb 2026).

The first pre-training objective is a query-conditioned InfoNCE loss: $1.024 \times 10^8$9 The denominator incorporates not only anchor–answer negatives but also cross-user anchor and answer terms, together with a margin-based mask $\mathbf{e}_i$0 to remove false negatives when another sample is too similar to the positive pair (Yuan et al., 16 Feb 2026). The paper states that this loss aligns query-conditioned user embeddings with answer embeddings while improving semantic discriminability.

The second objective is Next-Token Prediction: $\mathbf{e}_i$1 where $\mathbf{e}_i$2 are the answer tokens of $\mathbf{e}_i$3 (Yuan et al., 16 Feb 2026). This objective requires the anchor pathway to autoregressively reconstruct the answer from the user prefix and query, thereby enforcing token-level grounding.

The total pre-training loss is

$\mathbf{e}_i$4

The paper interprets the two terms as complementary: $\mathbf{e}_i$5 induces global discriminative structure, whereas $\mathbf{e}_i$6 supplies token-level semantic grounding (Yuan et al., 16 Feb 2026). In that sense, UserU is simultaneously a representation dataset and a supervision scaffold for aligning behavior, query, and answer spaces.

6. Scenario adaptation, evaluation, and deployment significance

The paper argues that UserU supplies the universal pre-training prior required for downstream scenario adaptation. After pre-training, the model is adapted with Cluster-based Soft Prompt Tuning, which freezes the LLM backbone and hierarchical encoder while learning scenario-specific soft prompt tokens and class prototypes (Yuan et al., 16 Feb 2026). The reported effect is to transform the universal user space into a scenario-specific discriminative embedding space without discarding the semantics learned from UserU.

On 10 Alipay real-world binary classification tasks, grouped into User Engagement, Risk, and Marketing Sensitivity, user embeddings are evaluated via linear probing with AUC and KS (Yuan et al., 16 Feb 2026). The paper reports the following headline comparisons: Llama-Embed-Nemotron-8B achieves AUC 0.7488; FOUND achieves AUC 0.7832; Q-Anchor (Base) achieves AUC 0.8104; and Q-Anchor (Prompt Tuned) achieves AUC 0.8225 with KS 0.5267 (Yuan et al., 16 Feb 2026). It also reports that adding prompt tuning improves average AUC from 0.8104 to 0.8225 and KS from 0.5044 to 0.5267 (Yuan et al., 16 Feb 2026).

Ablation results in the paper attribute a substantial portion of this performance to UserU pre-training. Removing contrastive loss reduces AUC from 0.8104 to 0.7667 and KS from 0.5044 to 0.4215; removing NTP yields a smaller decline, with AUC dropping from 0.8104 to 0.8061; omitting the margin mask causes a small drop; and removing pre-training entirely while keeping prompt tuning reduces AUC from 0.8225 to 0.7782 and KS from 0.5267 to 0.4679 (Yuan et al., 16 Feb 2026). The paper explicitly states that “pretraining is not a mere optimization aid but a foundational requirement for distilling robust behavioral priors” (Yuan et al., 16 Feb 2026).

The same paper further reports that scaling UserU data from 20.48M to 102.4M pairs yields consistent gains, whereas scaling model size from 0.5B to 3B does not (Yuan et al., 16 Feb 2026). This suggests that, within the reported setup, the structure and quantity of user-aligned supervision in UserU matter more than model scale alone.

Deployment considerations are also bound to the UserU design. Because the query is appended at the sequence terminus, the model can precompute and store the KV-cache for the prefix $\mathbf{e}_i$7, then reuse it across many scenario queries (Yuan et al., 16 Feb 2026). The paper states that the per-query incremental complexity is $\mathbf{e}_i$8, with the prefix cost amortized across scenarios (Yuan et al., 16 Feb 2026). Offline embedding refresh is performed daily on a 100×L20 GPU cluster using delta updates, while pre-training uses Qwen2.5-0.5B Instruct with LoRA on 64 A100-80G GPUs (Yuan et al., 16 Feb 2026). In production A/B testing, the paper reports improvements in IVR cash-reserve outreach and credit delinquency risk identification, including +12.5% drawdown rate, +5.3% average outstanding balance per user, +4.2% product visit rate, +17.7% drawdown-page visits, and +1.96% KS in the risk pipeline (Yuan et al., 16 Feb 2026). These results are presented as evidence that UserU-backed query-anchored embeddings are practically useful at Alipay scale.

7. Interpretive significance, assumptions, and limitations

Within the paper’s own framing, UserU is the component that allows Query-as-Anchor to function “like a user-LLM rather than a generic text encoder” (Yuan et al., 16 Feb 2026). That characterization follows from the dataset’s pairing of user behavior with future summaries and user-understanding QA, rather than from generic language modeling alone. A plausible implication is that UserU changes the effective supervision target from generic semantic similarity to structured semantic interpretation of users.

The construction nevertheless imposes explicit assumptions and limitations. UserU is domain-specific, being built entirely from Alipay logs and therefore reflecting Alipay-style financial and lifestyle behaviors (Yuan et al., 16 Feb 2026). The paper also notes possible bias and coverage issues arising from the demographics, platform usage patterns, and long-tail distributions present in those logs (Yuan et al., 16 Feb 2026). In modality terms, UserU is limited to what exists inside the Alipay ecosystem; it does not include images, and natural text is comparatively constrained beyond search and generated QA (Yuan et al., 16 Feb 2026).

Another limitation concerns the semantics themselves. Much of the user-understanding supervision is synthetic, being produced through LLM-generated QA with a reflection step for faithfulness (Yuan et al., 16 Feb 2026). This suggests that UserU depends not only on behavioral coverage but also on the quality of topic selection, query generation, answer generation, and post hoc verification.

The paper also identifies a “scaling paradox”: larger models do not necessarily produce better embeddings when trained on the same UserU data (Yuan et al., 16 Feb 2026). It proposes future work on gradient recovery techniques, adaptive parameter tuning, and better data/parameter trade-offs (Yuan et al., 16 Feb 2026). Additional directions include enriching UserU with more modalities, richer annotations, refined QA generation and reflection pipelines, and possible extension to other domains beyond Alipay’s financial and lifestyle setting (Yuan et al., 16 Feb 2026).

In summary, UserU denotes an industrial-scale user-understanding pre-training dataset that transforms 90-day multi-modal Alipay behavioral profiles into a hierarchical token interface paired with future behavior summaries and synthetic user-understanding QA. Its function is to provide the behavioral and semantic priors required for query-anchored user representation learning, and the reported evidence associates it with both offline benchmark improvements and online production gains in Alipay (Yuan et al., 16 Feb 2026).