Joint Training in Recommender Systems

• Joint training approach is a method that simultaneously optimizes multiple objectives, such as ranking accuracy, adversarial robustness, and fairness in recommender systems.
• It employs adversarial losses, minimax games, and alternating updates to balance primary performance with secondary traits like robustness and bias control.
• Empirical results show notable gains in metrics (e.g., +11.2% HR@100) while highlighting trade-offs like increased popularity bias that require careful hyperparameter tuning.

A joint training approach in recommender systems and machine learning refers to the simultaneous optimization of multiple, typically interacting, objectives or modules—often by means of a minimax or multi-objective framework—rather than optimizing each component in isolation. In the context of recommender systems, joint training paradigms have been prominently applied to incorporate adversarial robustness, fairness constraints, or robustness to distributional bias alongside standard accuracy, resulting in models that offer improved performance along secondary axes (e.g., fairness, robustness, or diversity) while controlling degradation in primary performance metrics. Recent lines of research instantiate joint training especially for pairwise ranking tasks via adversarial augmentation and adversarial fairness enforcement, employing minimax games or adversarial losses embedded into the training workflow of matrix factorization or deep neural recommenders.

1. Mathematical Formulation of Joint Training Objectives

Joint training is instantiated as a simultaneous optimization over model parameters with respect to multiple, often competing, losses. In adversarial personalized ranking, for example, the model parameters $\Theta$ for a base recommender—such as matrix-factorization latent factors—are learned by minimizing a composite loss that couples the primary ranking loss (e.g., BPR) with one or more secondary objectives. The canonical forms are:

Adversarial Robustness Joint Objective:

For Adversarial Personalized Ranking (APR), the minimax/robust joint training problem is

$$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$

where $L_{\mathrm{BPR}}$ is the Bayesian Personalized Ranking loss and the second term adversarially penalizes models that are fragile to bounded perturbations $\Delta$ of the parameters (He et al., 2018, Anelli et al., 2021). This minimax structure is characteristic of joint adversarial training.

Fairness-aware Joint Objective:

For Debiased Personalized Ranking (DPR), jointly enforcing ranking fairness, the minimax joint training is

$$\min_{\Theta} \max_{\Psi} \sum_{(u,i,j)} L_{\mathrm{BPR}}(u,i,j; \Theta) + \alpha L_{\mathrm{Adv}}(\Theta; \Psi) + \beta L_{KL}(\Theta) + \lambda_{\Theta}\|\Theta\|^2_F,$$

where $L_{\mathrm{Adv}}$ is an adversarial loss against a group-predicting discriminator (parameterized by $\Psi$), $L_{KL}$ is a KL-divergence term for score normalization, and $\alpha, \beta$ balance the trade-off (Zhu et al., 2021).

This class of joint training frameworks enables balancing trade-offs between primary recommendation accuracy and vital secondary properties such as fairness or adversarial robustness, with explicit, tunable weighting.

2. Components and Workflow in Joint Training

The design of joint training for recommendation, especially via adversarial learning, involves multiple coordinated modules:

• Generator (Primary Model): Learns user/item embeddings or scoring functions to optimize the main ranking objective (e.g., BPR loss $L_{\mathrm{BPR}}$).
• Adversary (Discriminator or Perturbation Module): In APR, synthesizes worst-case parameter perturbations to expose model fragility; in DPR, attempts to recover group labels from model predictions, enabling group-invariant representation induction (He et al., 2018, Zhu et al., 2021).
• Loss Coupling: The generator is penalized not only for ranking errors, but also for vulnerability to adversary success or group bias. The magnitude of secondary losses is governed by hyperparameters (e.g., $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$0 for adversarial strength, $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$1 for fairness regularization).
• Mini-batch Alternating Updates: Joint optimization typically cycles between adversary and generator updates per mini-batch, akin to GAN training, until convergence (Zhu et al., 2021).

Table: Core Components in Joint Adversarial Training

Module	Role	Example Loss Term
Generator	Minimize ranking error, robustify model	$$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$2
Adversary	Maximize group recoverability / ranking error post-perturbation	$$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$3 (DPR), $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$4 (APR)
Coordinator	Controls update schedule / trade-offs	Hyperparameters ($$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$5, $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$6)

3. Joint Training for Adversarial Robustness (APR)

The APR framework augments the BPR loss for matrix factorization and neural recommenders with an explicit adversarial defense term. Training proceeds by:

• Sampling user–item–negative triples (as in BPR).
• Computing adversarial perturbations $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$7 of bounded norm via the fast-gradient sign method:

$$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$8

• Minimizing the sum of the standard and adversarial loss with respect to model parameters $$\min_{\Theta}\left[L_{\mathrm{BPR}}(\Theta) + \alpha \max_{\|\Delta\|\leq \epsilon} L_{\mathrm{BPR}}(\Theta+\Delta)\right],$$9 (He et al., 2018, Anelli et al., 2021).

Joint training in APR has shown substantial empirical gains (+11.2% relative HR@100/NDCG@100), yielding robust models that outperform BPR-MF, NeuMF, and IRGAN baselines (He et al., 2018).

However, joint training with APR induces adverse side effects: empirical and theoretical studies find that APR amplifies popularity bias and reduces novelty/coverage of recommendations, as the adversarial regularization disproportionately reinforces head items with abundant positive feedback (Anelli et al., 2021). This effect is enlarged on datasets with highly tailed feedback distributions.

4. Joint Training for Fairness in Ranking (DPR)

The joint training framework in DPR targets fairness metrics in personalized ranking, introducing an adversarial minimax structure to enforce group-invariant score distributions:

• Fairness metrics:
  • RSP@k: Relative std/mean of group exposure in top-$L_{\mathrm{BPR}}$0 recommendations.
  • REO@k: Relative std/mean of group true-positive rates in top-$L_{\mathrm{BPR}}$1.
• Discriminator: MLP-based, predicts item group from predicted score. The generator attempts to obfuscate group membership, minimizing the discriminator's classification ability.
• Objective coupling: The minimax loss couples BPR ranking accuracy, adversarial fairness regularization (with weight $L_{\mathrm{BPR}}$2), and a per-user KL normalization of scores ($L_{\mathrm{BPR}}$3).
• Training: Alternating mini-batch updates to generator and discriminator (Zhu et al., 2021).

Empirical evaluations demonstrate that DPR achieves dramatic reductions in group fairness disparities—RSP@15 reduced by 76.9%, REO@15 by 72.4%—at a modest cost to F1@15 ($L_{\mathrm{BPR}}$4 for RSP, negligible or improved for REO) (Zhu et al., 2021). Gains over non-adversarial baselines (FATR, Reg-RSP, Reg-REO) are substantial.

5. Interactions, Trade-offs, and Hyperparameter Tuning

The effectiveness of joint training depends critically on the tuning of trade-off hyperparameters, notably:

• $L_{\mathrm{BPR}}$5: Strength of adversarial/fairness regularization. Higher values yield greater fairness or robustness but decrease standard ranking accuracy. Empirically, $L_{\mathrm{BPR}}$6 (for RSP) or $L_{\mathrm{BPR}}$7 (for REO) provide strong fairness/diversity improvements for DPR (Zhu et al., 2021). In APR, $L_{\mathrm{BPR}}$8 best set around 1 for balanced accuracy–robustness trade-off (He et al., 2018).
• $L_{\mathrm{BPR}}$9: Weight on KL score normalization (DPR)—too high values degrade recommendation quality (Zhu et al., 2021).
• Perturbation budget $\Delta$0 (APR): ~0.5 provides best robustness–accuracy compromise; lower or higher degrades performance (He et al., 2018, Anelli et al., 2021).

A central property of joint training approaches in this domain is the explicit, quantifiable trade-off between the primary objective and adversarial/fairness goals. Increased secondary-regularization improves the targeted property (robustness, group fairness) at the cost of canonical recommendation accuracy or diversity metrics.

6. Computational Aspects and Scalability

Joint adversarial/fairness training introduces additional computational overhead:

• Each parameter or prediction update involves an extra forward–backward pass:
  • APR: One evaluation at $\Delta$1, one at $\Delta$2.
  • DPR: Discriminator forward–backward per scored sample.
• In practice, the overhead is modest: training times rise by a small constant over baseline BPR due to mini-batched updates and compact MLP discriminators (Zhu et al., 2021, He et al., 2018).
• Complexity remains linear in the number of observations and embedding size, matching the base learning algorithms plus additive adversarial complexity.

7. Implications, Limitations, and Extensions

Joint training enables the synthesis of recommender models with explicit control over non-standard desiderata such as adversarial robustness and group fairness. However, such frameworks expose complex trade-offs in the behavior and societal impact of deployed systems.

• In APR, despite robust improvements to accuracy and generalization, unwanted amplification of popularity bias and declines in novelty and coverage must be anticipated and, if needed, mitigated (e.g., with long-tail regularizers or specialized sampling) (Anelli et al., 2021).
• In DPR, dramatic fairness gains are consistently achievable with modest to negligible accuracy cost, provided trade-off parameter search is robust (Zhu et al., 2021).
• The joint training framework generalizes to a spectrum of recommender architectures, including neural collaborative filtering, graph-based recommenders, and attention models, by injecting adversarial or fairness-oriented losses at the embedding, hidden, or output layers (He et al., 2018).
• Theoretical convergence and stability are inherited from underlying adversarial learning theory; care is nonetheless required to avoid overfitting in the discriminator or instability in minimax optimization (Zhu et al., 2021).

A plausible implication is that future recommender systems will increasingly adopt joint training frameworks for multi-criteria optimization, but nuanced curation of trade-offs will remain an essential part of practical deployment and alignment with societal values.