Field-Aware Masked Auto-Encoding (FAMAE)

• FAMAE is a representation learning component that incorporates structured feature fields to generate predictive-sufficient item embeddings for sequential recommendations.
• It employs a masked auto-encoding strategy tailored to structured domains to capture cross-field dependencies and reduce uncertainty in sequential prediction.
• Integrated within the ReSID framework, FAMAE outperforms baselines by over 10% and cuts tokenization costs by up to 122x, improving both efficiency and accuracy.

Field-Aware Masked Auto-Encoding (FAMAE) is a component within the ReSID framework, designed for generative recommender systems that utilize Semantic ID (SID)-based sequential recommendation. FAMAE focuses on learning predictive-sufficient item representations from structured features, specifically addressing the limitations of semantic-centric pipelines where item embeddings are learned from foundation models and discretized using generic quantization schemes. By integrating field-awareness into the masked auto-encoding process, FAMAE aligns representation learning with the objectives of sequential recommendation, aiming to improve collaborative prediction and efficient sequence modeling (Liang et al., 2 Feb 2026).

1. Background and Motivation

SID-based generative recommendation leverages compact, discrete item codes to enable scalable and efficient autoregressive modeling of recommender sequences. Conventional approaches typically involve two disjoint stages: obtaining continuous semantic embeddings from foundation models and applying generic quantization to produce discrete SIDs. This pipeline is suboptimal as semantic embeddings may not optimally capture collaborative signals relevant to recommendation, and generic quantization can lead to inefficiencies due to misalignment with sequential prediction tasks. FAMAE is motivated by the need to tightly couple representation learning with the SIDs’ downstream use in autoregressive generative models, thus ensuring that the learned codes are predictive-sufficient and structured according to the needs of the recommender system (Liang et al., 2 Feb 2026).

2. Field Awareness in Representation Learning

FAMAE explicitly incorporates awareness of structured feature fields into the auto-encoding process. In the context of recommender systems, "fields" refer to distinct categories or sources of item features (e.g., categorical content information, collaborative context, item metadata). By accounting for the differing semantic and predictive properties of these fields, FAMAE enables the model to better preserve, disentangle, and leverage field-specific information during encoding. This field-awareness is designed to address the inadequacies of uniform encodings that obscure or dilute field-level signal, and to promote representations that are more coherent and interpretable for downstream collaborative prediction (Liang et al., 2 Feb 2026). This suggests that field-aware design strengthens the alignment between input structure and the encoded representations for autoregressive modeling.

3. Masked Auto-Encoding Objective

FAMAE employs a masked auto-encoding strategy to learn its item representations. Masked auto-encoding involves selectively masking portions of the input structured feature fields and tasking the encoder-decoder model to reconstruct the missing information. This process compels the encoder to capture cross-field dependencies and contextually informative signals, with the goal of producing representations that are robust and predictive-sufficient for sequence modeling. FAMAE’s masking approach is specifically tailored to the structured domains of recommender features, as opposed to generic masking in other domains. A plausible implication is that this tailored masking enables the model to optimize for the reduction of uncertainty under sequential autoregressive prediction, a property not generally ensured by standard auto-encoding (Liang et al., 2 Feb 2026).

4. Integration within the ReSID Framework

In the ReSID architecture, FAMAE operates as the initial stage dedicated to encoding items into continuous spaces that are optimized for both collaborative and structured feature prediction. These item representations are then handed off to the subsequent Globally Aligned Orthogonal Quantization (GAOQ) component, which discretizes the representations into compact and autoregressively predictable SIDs. The integration ensures a seamless flow from feature-aware, collaborative-optimized continuous codes to quantized sequences that reduce semantic ambiguity and prefix-conditional uncertainty, directly supporting efficient autoregressive generative recommendation (Liang et al., 2 Feb 2026).

5. Empirical Performance and Impact

The effectiveness of FAMAE within the ReSID framework has been demonstrated through theoretical analysis and comprehensive empirical evaluation across ten datasets. ReSID, comprising FAMAE and GAOQ, consistently outperforms strong sequential and SID-based generative baselines by an average of over 10%, while achieving a tokenization cost reduction of up to 122x. These results validate the value of FAMAE’s field-aware and predictive-sufficient representation learning in enhancing both the efficiency and accuracy of generative recommender systems (Liang et al., 2 Feb 2026). This suggests that field-aware masked auto-encoding addresses critical bottlenecks in scaling autoregressive recommendation and representation quantization.

6. Relationship to Prior Art and Distinctiveness

FAMAE distinguishes itself from existing semantic-centric and quantization approaches by reframing the representation learning process through field-awareness and collaborative predictiveness, rather than semantic similarity alone. Unlike generic quantization schemes that prioritize broad semantic encoding, FAMAE optimizes representations for sequential predictability and information sufficiency aligned with generative recommendation objectives. This field-cognizant and prediction-oriented paradigm responds directly to deficiencies identified in prior pipelines, positioning FAMAE as a principled advance tailored to the requirements of modern SID-based recommender systems (Liang et al., 2 Feb 2026).