Data-Efficient Personalization

• Data-efficient personalization methods are techniques that adapt models to individual users while minimizing data, compute, and communication requirements.
• They balance user-specific adaptation with global generalization using metrics like weighted loss functions and low-rank user modeling.
• Practical frameworks span federated updates, on-device strategies, and resource-aware training to achieve scalable and privacy-preserving personalization.

Data-efficient personalization methods aim to deliver effective, user-specific model adaptation while minimizing the amount of data, compute, and communication required. These techniques span foundational metrics for personalization, black-box topic modeling, representation learning with theoretical guarantees, parameter- and memory-efficient federated updates, and on-device, resource-aware adaptation protocols. Approaches delivering strong results balance statistical robustness, generalization, model interpretability, privacy, and computational efficiency. The field is marked by methodological diversity, rigorous theoretical analysis, and practical frameworks tailored to both central and federated/decentralized environments.

1. Personalization Metrics and Overfitting–Generalization Trade-offs

A foundational dimension of data-efficient personalization is quantifying the trade-off between fitting user-specific data and preserving generalization across the population. One principled metric formalizes overall loss as

$$L_\text{total} = \alpha\,L(X_i, M_i) + (1-\alpha)\,L(D, M_i)$$

where $L(X_i, M_i)$ denotes error on user $i$’s own data, $L(D, M_i)$ the error on a global dataset, and $\alpha \in [0,1]$ modulates the emphasis on personalization versus regularization (Brasher et al., 2018). This captures a key tension: aggressive personalization ($\alpha \to 1$) can induce overfitting, especially with sparse data; enforcing global performance ($\alpha \to 0$) regularizes but can limit individual specificity.

This metric allows tuning $\alpha$ to fit practical objectives and enables analytical model selection via break-even thresholds: $$\alpha = \frac{g_1 - g_0}{(p_0 - p_1) - (g_0 - g_1)}$$ where $p_\cdot$ and $g_\cdot$ are local and global loss metrics for two candidate models. The metric is privacy-compatible, supports decentralized architectures, and frames subsequent developments in federated, meta-learning, and adaptive pipeline designs.

2. Black-Box Topic-Level Personalization with Latent Topic Models

The Latent Topic Personalization (LTP) framework decomposes online service platform (OSP) personalization into distinct blocks: a topic model reminiscent of LDA, and a permutation-based ranking model for outputs (Majumder et al., 2012). The user is characterized by a personalization vector $\eta$ (where $\eta_k$ indicates affinity for topic $k$), inferred solely from differences in observed personalized versus “vanilla” outputs. The core mechanics are as follows:

• Topic block: Each candidate item (e.g., a URL) is annotated by a topic-map $\theta_i$, a multinomial over topics; topics are Dirichlet priors.
• Personalization block: Given the vanilla ranking $\sigma$ and personalized output $T$, a latent switch $z$ determines whether reranking arises from user topics. If $z=1$, reranking is modeled by a score function $g$ combining the vanilla order and $\eta^\top\theta_d$ for each item.
• Variational inference: Approximate posteriors for $\eta$, $z$, and ranking variables are learned via LTP-INF; LTP-EM alternates variational steps with parameter reestimation.

LTP is empirically validated with $R$-precision up to 84% in extracting personalized topics (using AlterEgo synthetic and Google data), and is notable for: (1) inferring topic sensitivities used by opaque OSPs, (2) requiring only platform output, (3) enabling privacy-centric audits of personalization.

3. Representation Learning: Low-Rank plus Sparse User Modeling

Sample-efficient user adaptation is achieved by parameterizing each user’s model as

$\Theta^{(i)} = U^\ast w^{(i)} + b^{(i)}$

where $U^\ast$ is a shared low-rank basis, $w^{(i)}$ user-specific combination weights, and $b^{(i)}$ a user-specific $k$-sparse correction term (Pal et al., 2022). The method—solved via alternating minimization with hard thresholding (AMHT-LRS)—provides several advantages:

• Reduces memory/storage costs: only $O(r + k)$ extra parameters per user.
• Sample complexity is nearly linear in $k$, cubic in $r$; the method provably recovers global and user idiosyncrasies under a Gaussian linear model.
• Differential privacy is achieved by noise-adding and clipping during global subspace updates, providing user-level privacy guarantees without substantial degradation in accuracy.

This model is especially applicable to large-scale recommendation and other multi-user systems with severe data sparsity.

4. Parameter- and Memory-Efficient Personalization in Federated and Decentralized Settings

Data-efficient personalization in federated learning is a major research strand. Key frameworks include:

• FedPer (Base plus Personalization Layer): The architecture shares robust “base” representation layers across clients and fine-tunes lightweight heads for each local distribution (Arivazhagan et al., 2019). This structure mitigates statistical heterogeneity, improves local task accuracy, and requires minimal data per client.
• FLoRAL (Federated Low-Rank Adaptive Learning): Here, client models are parameterized by

$w^k = u + \sum_{c=1}^C \pi^k_c a_c$

with $u$ the global weight vector, $a_c$ a set of $C$ shared low-rank adaptors, and $\pi^k$ a client-specific mixture. Memory is sharply reduced (per-client storage: $O(C)$ router coefficients), and overfitting is constrained by regularizing each client’s adaptation to lie in a low-dimensional shared subspace (Almansoori et al., 4 Oct 2024). The approach achieves better accuracy than full model mixtures or per-client local adaptors, and theory confirms improved gradient variance reduction in federated optimization.

• FedMCP (Model-Contrastive Personalization with Adapters): Inserting lightweight global and private adapters in frozen PLM backbones, only global adapters are aggregated, and a model-contrastive loss (using CKA similarity) enforces complementary universal and user-specific representations (Zhao et al., 28 Aug 2024). Storage and transmission per client is reduced to $\sim$1% of model size.
• Bayesian Federated Learning with Efficient Second-Order Optimization: Each client infers a local Gaussian posterior (mean $m_k$, diagonal covariance $\sigma_k^2$), updated through an implicit Hessian estimator (IVON). Global aggregation is Fisher-weighted, and a hierarchical prior enables client personalization (Pal et al., 27 Nov 2024). This yields accurate uncertainty estimates, improved predictive accuracy, and low communication overhead, with high data efficiency especially on heterogeneous data.
• Mixture-of-Experts with Joint Gating (FedJETs/DDOME): A pool of expert models is dynamically selected per client by a learned gating function conditioned on a shared fixed expert for embedding; only relevant specialized experts are communicated and trained for each client, reducing overhead and improving accuracy in the presence of distributional heterogeneity (Farhat et al., 2023).

5. On-Device Adaptation and Resource-Aware Training Strategies

Several methods address practical limitations of on-device personalization, constrained by edge resource budgets:

• Quantized Diffusion Personalization with Zeroth-Order Optimization: Full model quantization (e.g., INT8) is paired with forward-pass-only optimization for token adaptation, thus avoiding both backpropagation and memory-intensive dequantization (Seo et al., 19 Mar 2025). Subspace Gradient projection (PCA-based denoising within the trajectory of token updates) and partial uniform timestep sampling concentrate optimization on signal-carrying dimensions and critical training steps. This yields up to $8.2\times$ lower VRAM requirements while maintaining competitive image/text alignment.
• Edge FPGA Training Accelerator: EF-Train couples channel-level parallelism, continuous memory allocation, and loop reordering for efficient DNN training on FPGAs, enabling mini-batch personalization with minimal memory and high throughput (up to 46.99 GFLOPS, 6.09 GFLOPS/W) (Tang et al., 2022). The design supports real-time adaptation for dynamic environments such as UAVs, robotics, and medical devices.
• Hybrid Cloud-Device Data Augmentation: CDCDA-PLM applies cloud-based LLMs to synthesize high-quality personal data (using semantic and diversity filters), then fine-tunes a small on-device model using LoRA adapters and both real and synthetic data (Zhong et al., 29 Aug 2025). This approach supports privacy-aware, scalable, and resource-constrained latent adaptation while maintaining on-device independence during inference.
• Explainable Model Selection for On-Device Personalization: XPerT expedites personalization by selecting pre-existing fine-tuned LLMs whose explainable style drift (encapsulated in orthogonal basis vectors) is closest to the local user profile. On-device fine-tuning is then limited to a small update; overall bandwidth and compute cost is sharply reduced (computation savings up to 83%, data efficiency gain up to 51%) (Wang et al., 15 Apr 2025).

6. Personalization in Structured Prediction and Recommendation

Specific approaches target rich user modeling for sequential and set-based prediction tasks:

• Simple Transformers for Real-Time Personalization: Single self-attention transformer layers can (provably) represent complex user choice phenomena such as variety, complementarity, and “halo” effects. An efficient two-phase algorithm (ANN retrieval, then low-rank BQP optimization) achieves near-optimality in sub-linear time, with empirical gains (14.1% accuracy over classical methods) demonstrated for Spotify and Trivago datasets (An et al., 1 Mar 2025).
• Efficient LLM Personalization with Embedding-to-Prefix: External user embeddings are projected via lightweight MLPs into “soft” prefix tokens inserted into frozen LLMs; personal context is thus injected with zero overhead in token space and with a minimal weight parameter footprint (Huber et al., 16 May 2025). Empirical improvements cover dialogue, headline, and music/podcast recommendation settings.
• Rule-Based Steering for Large LMs: Small models are trained to map user inputs to concise, natural-language preference rules, which are concatenated to prompt input for a larger LM—forming a “steering wheel” that achieves efficient personalization even with very limited user data (Shashidhar et al., 30 Sep 2024).
• Hierarchical Representation and Collaborative Refinement: Persona-DB restructures user history into semantically dense, hierarchical personas (distilled/induced + cache), enabling compact retrieval and collaborative knowledge injection from similar users to address cold-start scenarios, with superior performance in LLM response prediction and high data efficiency (Sun et al., 16 Feb 2024).

7. Clinical Applications and Uncertainty-Guided Sampling

Personalization methods are increasingly tailored for inclusivity and clinical contexts:

• ASR Personalization for Non-Normative Speech: A targeted oversampling strategy leverages per-phoneme uncertainty estimates (Phoneme Difficulty Score, derived from MC Dropout ensemble predictions, entropy, and agreement) to drive data-efficient adaptation on speakers with impairments (Pokel et al., 23 Sep 2025). The method is validated against expert logopedic (speech therapy) ratings and achieves substantive error rate reductions in UA-Speech and BF-Sprache datasets. The alignment between phoneme-level uncertainty and clinical difficulty assessments underlines the framework’s potential utility in assistive and diagnostic applications.

Summary Table: Representative Methods and Features

Method/Framework	Key Principle	Data/Parameter Efficiency Mechanism
LTP (Majumder et al., 2012)	Topic vector extraction from output diff	Black-box inference, variational EM
FedPer (Arivazhagan et al., 2019)	Base + personalized head layers	Head-only local updates, federated aggregation
FLoRAL (Almansoori et al., 4 Oct 2024)	Low-rank adaptor mixtures, federated	Mixture routers, LoRA adaptors
AMHT-LRS (Pal et al., 2022)	Low-rank + sparse meta-learning	Alternating minimization, DP extension
Persona-DB (Sun et al., 16 Feb 2024)	Hierarchical persona, collaborative join	Compact abstraction, cross-user refinement
CDCDA-PLM (Zhong et al., 29 Aug 2025)	Cloud-augmented on-device tuning	Synthetic data, LoRA PEFT, filtering
Simple Transformers (An et al., 1 Mar 2025)	Single self-attention, fast search	Sub-linear algorithm, set interactions
XPerT (Wang et al., 15 Apr 2025)	Explainable personalized LLM selection	Latent vector search, on-device PEFT
FedMCP (Zhao et al., 28 Aug 2024)	Model-contrastive federated PEFT	Adapter modules, CKA similarity
ASR PhDScore (Pokel et al., 23 Sep 2025)	Phoneme uncertainty, oversampling	MCD entropy, expert-aligned sampling

Outlook

Data-efficient personalization methods have matured into a rigorous toolkit for adapting models to user-specific distributions under data, memory, computational, and privacy constraints. They enable robust and practical model adaptation for centralized, federated, on-device, and assistive domains. Research continues to advance generalization theory, privacy mechanisms, adaptive data augmentation, and automatic selection of adaptation pathways, underpinning scalable, responsible personalization in real-world AI systems.