Personalization & Multi-Task Learning

• Personalization and multi-task learning are methods that adapt models to individual users by blending global, group, and user-specific parameters, as demonstrated in recommender systems and federated networks.
• They utilize hierarchical decompositions and custom loss functions, like AUC loss, to effectively address data scarcity and task heterogeneity.
• Scalable optimization techniques, including blockwise proximal methods and consensus ADMM, ensure robust convergence and privacy in applications such as dialogue and spatio-temporal forecasting.

Personalization and multi-task learning are tightly interwoven in modern machine learning, especially where user heterogeneity, data scarcity, and complex task relationships drive demand for models capable of per-user or per-context adaptation. Recent research formulates personalization as a multi-level multi-task problem, designs scalable optimization algorithms, analyzes theoretical generalization and convergence, and empirically demonstrates gains across domains such as attribute prediction, federated learning, recommendation, reinforcement learning, dialog, and spatio-temporal modeling.

1. Conceptual Foundations and Multi-Level Decomposition

Personalization is the process of tailoring a model, prediction, or policy to the preferences, context, or behavior of an individual user or subpopulation. Multi-task learning (MTL) is a paradigm in which parameters are shared across related tasks to enable statistical strength-sharing while retaining some degree of task-specific modeling. The connection is formalized by treating each user's learning problem as a specific "task," so personalization reduces to learning a model ensemble with both global (shared) and local (personalized) components.

A canonical illustration is the hierarchical parameter decomposition for personalized attribute prediction (Yang et al., 2019). For $U$ users and $d$-dimensional features, each task $i$ is assigned a linear predictor $f^{(i)}(x)=W^{(i)\top}x$, with

$W^{(i)} = \theta + G^{(i)} + P^{(i)}$

where:

• $\theta \in \mathbb{R}^d$: global consensus vector shared by all.
• $G\in\mathbb{R}^{d\times U}$: group-level factors, regularized to induce co-clustering (via capped trace-norm).
• $P\in\mathbb{R}^{d\times U}$: individual-specific deviations, penalized for column sparsity.

Regularization on $\theta$, $G$, and $P$ controls the tradeoff between global sharing (consensus), medium-scale sharing (group/cluster), and strong personalization (per-user). The global, group, and personalized terms provide a principled means to interpolate between universality and complete individualization (Yang et al., 2019).

Parallel decompositions appear in recommender systems (per-user/item embeddings with personalized task weights) (Yang et al., 2024), federated learning (global plus per-client model partitions) (Mills et al., 2020, Almansoori et al., 2024), tensorized MTL for personalized high-dimensional modeling (shared low-rank tensor plus per-task residuals) (Konyar et al., 21 Aug 2025), and decentralized graph-based personalization (Mortaheb et al., 2022, Odeyomi et al., 31 Aug 2025).

2. Objective Functions and Task-Specific Loss Formulation

Personalization often requires models to optimize for metrics that directly reflect user-specific (or task-specific) objectives. In personalized attribute learning, the relevant metric is frequently the Area Under the ROC Curve (AUC), rather than classical pointwise accuracy (Yang et al., 2019, Yang et al., 2020). To this end, the empirical AUC loss for user $i$ aggregates over all positive-negative sample pairs: $$\ell_{AUC}^{(i)} = \frac{1}{n_{+,i} n_{-,i}} \sum_{x_p \in S_{+,i}} \sum_{x_q \in S_{-,i}} (1 - W^{(i)\top}(x_p - x_q))^2$$ Efficient computation leverages Laplacian-based formulations for scalable evaluation of loss and gradient (Yang et al., 2019, Yang et al., 2020).

In federated or decentralized settings, the multi-task loss is a weighted sum over users/tasks, possibly with task- or data-volume-based reweighting: $$L(\{w, \theta_i\}_{i=1}^N) = \sum_{i=1}^N \alpha_i \ell_i(w, \theta_i)$$ where $\theta_i$ are per-client private parameters (e.g., non-federated BatchNorm in FL), $w$ global parameters, and $\ell_i$ the loss on user $i$'s data (Mills et al., 2020).

Gradient-based task weighting, personalized at the embedding or user/item level (e.g., via per-task $\ell_2$ gradient norms and softmaxed task weight scheduling), enables both alignment with primary objectives and robustness to auxiliary task imbalance (Yang et al., 2024).

Auxiliary tasks (e.g., knowledge tracing in education (Nasrin et al., 5 Jul 2025), persona reconstruction in dialogue (Lee et al., 2021), or auxiliary self-supervised objectives in vision/language tasks (Lee et al., 30 Sep 2025)) are often incorporated in a multi-task regime to further boost personalization signal, provided loss contributions are properly balanced.

3. Optimization Algorithms and Scalability

Optimization for personalized multi-task objectives requires addressing blockwise regularization, nonconvexity (especially due to grouping and co-clustering), and large-scale model updates distributed across (possibly heterogeneous) clients.

Blockwise proximal gradient methods with backtracking line search are standard in convex settings with structured regularizers. For group-level trace-norm regularization, the G-subproblem admits closed-form solutions via generalized singular value thresholding (SVD followed by elementwise shrinkage on the singular values) (Yang et al., 2019). For collaborative task-feature grouping, the bipartite graph Laplacian structure admits a convex relaxation based on the sum of the bottom $k$ eigenvalues, leading to globally convergent block coordinate descent (Yang et al., 2020).

In the federated domain, MTFL modifies FL algorithms to support personalized parameter partitions, e.g., non-federated private layers (BatchNorm) local to users, while remaining parameters are aggregated as in FedAvg or FedAvg-Adam. Each FL round proceeds by local updates followed by weighted averaging, preserving compatibility with classical FL optimizers while providing seamless personalization (Mills et al., 2020).

Consensus-ADMM methods decouple global and local model updates, enforcing soft proximity constraints (e.g., quadratic regularization $\|w_i-w_0\|^2$ between local and global weight vectors), and yield closed-form iterate updates and scalability to hundreds of clients (Ponomarenko-Timofeev et al., 2023).

Decentralized or peer-to-peer MTL leverages dynamic communication graphs based on pairwise task similarity ("transference" calculated from exchanged gradients) to continuously rewire aggregation patterns so that only mutually beneficial tasks exchange updates, mitigating negative transfer in heterogeneous environments (Mortaheb et al., 2022, Odeyomi et al., 31 Aug 2025).

4. Generalization, Convergence, and Privacy Guarantees

Theoretical guarantees in personalized multi-task learning span generalization error, convergence rates, and, increasingly, privacy bounds.

Rademacher complexity analysis, in conjunction with trace-norm and column-sparsity control, provides a bound on the average per-task AUC loss in hierarchical decomposition models (Yang et al., 2019). Under standard smoothness, strong convexity, and bounded variance, blockwise proximal gradient and decentralized updates converge to stationary points at sublinear or linear rates, with explicit rates provided for step sizes and network parameters (Yang et al., 2019, Yang et al., 2020, Mortaheb et al., 2022).

In federated and privacy-sensitive settings, mean-regularized MTL achieves personalized models under silo-specific sample-level differential privacy, offering closed-form derivations of optimal personalization parameter settings as a function of privacy noise and heterogeneity (Liu et al., 2022). Recent advances categorize privacy models for MTL and meta-learning (joint DP, billboard, 1-out-of-$t$), showing that personalized MTL is strictly less sample-inefficient than DP metalearning in high dimension, with lower bounds established via fingerprinting codes and tight upper bounds constructed by analytical noise calibration (Aliakbarpour et al., 2024).

Optimization convergence and statistical efficiency are established in federated low-rank personalization frameworks (e.g., FLoRAL's clustered router-based SGD), and in neural meta-RL with personalized regularization, with sublinear rates and explicit trade-offs in hyperparameter schedules (Almansoori et al., 2024, Ji et al., 2023).

5. Application Domains and Empirical Results

Personalized MTL is extensively validated on practical problems spanning heterogeneous attribute prediction, federated learning, recommendation, reinforcement learning, dialogue, education, medical time-series, and spatio-temporal forecasting.

• In attribute prediction, hierarchical and co-clustered multi-task models deliver significant AUC improvements over consensus-only or baseline MTL methods, and raise bottom-quartile per-user accuracy (Yang et al., 2019, Yang et al., 2020).
• In federated classification and regression, personalized MTL achieves faster convergence and better per-client test accuracy under non-IID heterogeneity, both in DNNs via private BN patches (Mills et al., 2020), and in federated SVMs via consensus-ADMM (Ponomarenko-Timofeev et al., 2023).
• In recommendation, personalized multi-task gradient-level integration (PMTRec) increases NDCG and Recall metrics by 3–10% across diverse datasets, particularly benefiting cold-start users/items (Yang et al., 2024).
• Multi-task and meta-RL applications demonstrate that personalized committees or per-task policies dramatically outperform standard multi-task or meta-learning baselines on both zero-shot and few-shot adaptation for highly diverse task distributions (Ge et al., 26 Feb 2025, Ji et al., 2023).
• In spatio-temporal learning, a multi-task backbone with per-task prompts and targeted freezing of stable attention weights enables robust adaptation to changing urban domains and fast cold starts with preserved task uniqueness (Yi et al., 2024).
• In education, multi-task LSTM architectures that fuse recommendation and knowledge tracing objectives achieve +6% accuracy over single-task models and improve trace AUC scores (Nasrin et al., 5 Jul 2025).
• In privacy-preserving cross-silo FL, mean-regularized MTL dominates both local-only and global-only approaches under strong DP, with Pareto-optimal $\lambda^*$ adapting automatically to privacy noise and heterogeneity (Liu et al., 2022).
• In decentralized and cyber-physical systems, trust-aware, peer-to-peer multi-task learning algorithms enable honest clients to maintain sublinear regret even in the presence of a Byzantine majority, providing robust personalization under high adversarial load (Odeyomi et al., 31 Aug 2025).

6. Extensions, Limitations, and Theoretical Insights

Extensions include tensorized MTL for multiway data (identifying shared and personalized structure via Tucker decomposition) (Konyar et al., 21 Aug 2025), multi-modal and multi-task foundation models in federated settings with blockwise replacement and post-hoc knowledge distillation (Lee et al., 30 Sep 2025), and rolling adaptation schemes that alternate between learning shared invariants and refining per-task idiosyncrasies (Yi et al., 2024).

Major limitations span the need for cross-validation over regularization and subspace hyperparameters, computational overhead for singular value or tensor factorization, sensitivity to auxiliary task and regularization weighting, and the assumption of task-label availability for clustering or committee construction.

Theoretical frontiers concern the interplay of personalization granularity, sample complexity, and privacy: recent research establishes formal separations and optimality criteria for private personalization, the benefits of clustering or committee structures for local adaptation, and the variance reduction achievable via grouped weight sharing in federated settings (Aliakbarpour et al., 2024, Almansoori et al., 2024, Ge et al., 26 Feb 2025).

7. Summary Table: Core Personalized Multi-Task Models

Paper / Setting	Personalization Mechanism	Optimization / Guarantee
(Yang et al., 2019) Attribute learning	Multi-level (global, group, user-specific)	Blockwise PGD, SVD for group term, AUC bound
(Mills et al., 2020) Federated DNNs	Private BN layers; shared parameters	FL + FedAvg(/Adam); UA, rounds, privacy
(Yang et al., 2020) Co-group MTL	Task-feature bipartite block regularization	Convex relaxation, global convergence
(Yang et al., 2024) RecSys MTL	Per-user/item gradient-level task weighting	Personalized task weights, focusing, balancing
(Ponomarenko-Timofeev et al., 2023) Federated SVM	Local offset vs. global weight	Consensus ADMM, closed-form updates, privacy mask
(Almansoori et al., 2024) FLoRAL (Fed. Low-rank)	Client-specific adaptors in LoRA pool	Router + SGD, variance reduction, convergence
(Ge et al., 26 Feb 2025) RL committees	Policy-set covering distinct task types	Greedy clustering, sample complexity guarantees
(Konyar et al., 21 Aug 2025) Tensorized MTL	Low-rank Tucker, task-residuals	Block-descent GLMs, interpretability, simulation/real
(Aliakbarpour et al., 2024) DP MTL/Meta-learn	Privacy threat taxonomy, separation theorems	Explicit noise/sample complexity separation

• Learning Personalized Attribute Preference via Multi-task AUC Optimization (Yang et al., 2019)
• Multi-Task Federated Learning for Personalised Deep Neural Networks in Edge Computing (Mills et al., 2020)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction (Yang et al., 2020)
• Personalized Multi-task Training for Recommender System (Yang et al., 2024)
• Multi-Task Model Personalization for Federated Supervised SVM in Heterogeneous Networks (Ponomarenko-Timofeev et al., 2023)
• Collaborative and Efficient Personalization with Mixtures of Adaptors (Almansoori et al., 2024)
• Learning Policy Committees for Effective Personalization in MDPs with Diverse Tasks (Ge et al., 26 Feb 2025)
• Tensorized Multi-Task Learning for Personalized Modeling of Heterogeneous Individuals with High-Dimensional Data (Konyar et al., 21 Aug 2025)
• Privacy in Metalearning and Multitask Learning: Modeling and Separations (Aliakbarpour et al., 2024)
• Online Decentralized Federated Multi-task Learning With Trustworthiness in Cyber-Physical Systems (Odeyomi et al., 31 Aug 2025)
• Get Rid of Isolation: A Continuous Multi-task Spatio-Temporal Learning Framework (Yi et al., 2024)