---
title: Scheduled Multi-Task Learning (SML)
url: https://www.emergentmind.com/topics/scheduled-multi-task-learning-sml
type: topic
---

# Scheduled Multi-Task Learning (SML)

Scheduled Multi-task Learning (SML) is a family of methodologies for multi-task learning (MTL) in which the order, frequency, and relative emphasis of different tasks are dynamically controlled according to an explicit or implicit schedule. SML frameworks are designed to maximize performance on the main task by leveraging auxiliary tasks selectively—either by task weighting, adaptive sampling, sequential group updates, or meta-learning policies. This approach stands in contrast to conventional multi-task learning, where task contributions are fixed or determined via static weighting, and represents a convergence of ideas from curriculum learning, meta-learning, and gradient-based task selection.

## 1. Core Principles and Definitions

The central objective in SML is to learn a shared representation or parameterization $\theta$ (or multi-headed variants with some task-specific parameters) such that the main task’s loss $L_{\text{main}}(\theta)$ is minimized, with strategic incorporation of auxiliary task losses $\{L_k(\theta)\}_{k=1}^K$. The key innovation in SML frameworks lies in dynamically modulating each task’s contribution via a learned or pre-defined schedule, schedule-adaptive mixing coefficients, or meta-policies—ensuring that task interference is mitigated and positive transfer is maximized.

The most general form of the instantaneous SML objective can be written as:

$$
L_{\text{total}}(\theta; t) = L_{\text{main}}(\theta) + \sum_{k=1}^K \lambda_k(t) L_k(\theta)
$$

where $\lambda_k(t)$ are (possibly time-varying or data-dependent) scheduling weights, aligning the optimization trajectory to favor the main task as dictated by an explicit schedule, adaptive policy, or meta-scheduler [2205.03766, 1804.08915, 1909.06434, 2007.05290].

## 2. Scheduling Strategies

SML encompasses a broad spectrum of scheduling strategies:

- **Static schedules**: The contribution of each task is fixed throughout training, e.g. constant mixing ratios or round-robin task selection [1804.08915].
- **Pre-determined dynamic schedules**: Schedules shift over time following a hand-designed policy (e.g., exponentially increasing weight on the main task or sigmoidal ramp-up) [1804.08915].
- **Gradient-based scheduling (gradient surgery)**: At every step, one projects each auxiliary task’s gradient $g_k$ onto the main task’s gradient $g_{\text{main}}$ to retain only the portion aligned (or apply a regularizer if anti-aligned), followed by global time-dependent scaling [2205.03766].
- **Adaptive sampling and meta-scheduling**: Task selection or weighting is adjusted online based on relative task progress, validation metrics, or a learnable policy—often formulated as a bandit, reinforcement learning, or bi-level optimization problem [2007.05290, 1702.06053, 1909.06434].
- **Affinity-based grouping and sequential updates**: Tasks are partitioned into affinity clusters and groups are updated sequentially within each batch, reducing negative transfer and enabling more effective adaptation of task-specific parameters [2502.11986].

A summary of representative SML scheduling strategies:

| Approach                            | Scheduling Mechanism                               | Key Reference     |
|:------------------------------------ |:------------------------------------------------- |:----------------- |
| Constant, Exponential, Sigmoid       | Fixed or monotonically varying coefficients       | [1804.08915]      |
| Gradient surgery + time decay        | Projection, decay schedule for $\alpha(t)$        | [2205.03766]      |
| Adaptive sampling, RL meta-policy    | Bandit, policy-gradients, active task selection   | [1702.06053], [2007.05290] |
| Affinity-based grouping              | Online affinity metrics + sequential updates      | [2502.11986]      |

## 3. Algorithmic Realizations

### Curriculum and Schedule-based Weighting

A general approach is to specify time-varying weights for each task based on an explicit curriculum. For example:

- **Exponential schedule**: $\alpha(t) = 1 - \exp(-\gamma t)$,
- **Sigmoid schedule**: $\alpha(t) = 1/(1 + \exp(-\gamma t))$,

where $\alpha(t)$ defines the main task’s proportion and $1-\alpha(t)$ is spread over auxiliaries [1804.08915]. Used in NMT, such schedules progress from heavy auxiliary focus (linguistic structure) to almost pure translation over epochs, embodying a neural curriculum.

### Gradient-based Scheduling

In the context of neural chat translation, SML uses a three-stage pipeline: (1) generic pre-training, (2) in-domain pre-training with all auxiliary tasks, (3) fine-tuning with continued scheduling. In Stages 2 and 3, when computing the gradient, each auxiliary gradient $g_k$ is projected onto $g_{\text{main}}$, scaled by a global $\alpha(t)$ which decays from 1 to 0:

$$
\Delta\theta \propto g_{\text{main}} + \alpha(t) \sum_{k=1}^K \text{Proj}_{g_{\text{main}}}(g_k)
$$

This ensures only the auxiliary task contributions aligned with the main task are accumulated, and the regularizer effect is retained for misaligned tasks. $\alpha(t)$ is linearly decayed within each stage [2205.03766].

### Affinity-based Grouping and Sequential Updates

SML by selective group update divides tasks into dynamic clusters based on online affinity metrics such as Proximal Inter-Task Affinity (PIA). After each mini-step for a group $G$, the reduction in loss for each other task $k$ is measured, and groups are adaptively merged or split. Within a batch, each group is updated sequentially, so that interference is reduced and task-specific information is better preserved:

- Batches of tasks with strong positive affinity are sequentially co-updated.
- Hyperparameters: affinity decay $\beta$, learning rate $\eta$, with theoretical conditions ensuring convergence to Pareto stationary points [2502.11986].

### Adaptive and Learnable Scheduling

Meta-scheduling or adaptive SML uses task performance signals (e.g., validation BLEU or loss) to adapt task weights or select the next training task:

- **Adaptive sampling**: Probability for task $i$ at a checkpoint is $p_i \propto 1/(S_i^\alpha + \varepsilon)$ where $S_i$ is the relative score versus baseline [1909.06434].
- **Active RL/bandit scheduling**: Task selection is cast as a contextual bandit or full RL problem, with meta-controllers that sample tasks based on observed or inferred gaps to target performance, uncertainty bonuses (UCB), or explicit rewards maximizing multitask performance [1702.06053, 2007.05290].
- **Self-paced MTL**: Tasks and instances are prioritized jointly by a self-paced regularizer, beginning with “easier” tasks/instances and gradually moving to harder ones, with group sparsity controlling task-level prioritization [1604.01474].

## 4. Empirical Findings and Impact

SML methods yield substantial, statistically significant improvements over conventional MTL and single-task learning across diverse application domains:

- **Neural Machine Translation**: SML systematic schedules (esp. exponential) yield +0.7 BLEU gains over strong NMT baselines, and +1.5–2 BLEU gains when augmented with in-domain pre-training and gradient-based scheduling for chat translation [2205.03766, 1804.08915].
- **Vision Multi-task Benchmarks**: Affinity-grouped SML methods achieve up to 80% relative improvement on aggregate performance (multi-task metric $\Delta_m$) compared to joint-gradient baseline and prior state-of-the-art MTL optimizers [2502.11986].
- **Sequence Learning with Temporally Correlated Tasks**: Bi-level and RL-based SML outperform uniform and curriculum transfer on both simultaneous translation and time-series (forecasting) benchmarks, yielding up to +3 BLEU and +0.05 RankIC relative gains [2007.05290].
- **Atari Reinforcement Learning**: Active sampling SML mechanisms double the multitask normalized mean reward compared to uniform sampling baselines, with robustness across 6–21 task setups [1702.06053].
- **Robustness and Ablations**: Ablation studies confirm the importance of scheduler dynamics, group affinity, inverse-projection regularization, and time-dependent weighting schedules. Removing adaptive scheduling uniformly decreases main task performance and exacerbates negative transfer [2205.03766, 2502.11986, 1909.06434].

## 5. Theoretical Frameworks and Analysis

Several SML variants provide explicit theoretical guarantees:

- **Gradient-Alignment and Loss Reduction**: Affinity-based sequential updates provably yield better gradient alignment and lower main-task loss compared to joint updates, for convex losses and small learning rates [2502.11986].
- **Convergence**: Under standard Lipschitz-gradient assumptions, sequential group updates converge to Pareto-optimal solutions across tasks [2502.11986].
- **Meta-learned Schedules**: Bi-level optimization is formalized with the outer loop targeting main-task validation loss and the inner loop adapting the parameters via scheduled task sampling, using REINFORCE gradients to train the scheduler policy [2007.05290].
- **Self-paced Regimes**: SPMTL’s group sparse regularizer ensures “easy” tasks/instances are learned first, with convergence guaranteed by block coordinate descent [1604.01474].

## 6. Limitations and Open Issues

- **Computational Overhead**: Gradient-based scheduling and meta-learning approaches require multiple backprops or extra forward passes, although group-based SML can reduce memory and compute to nearly $O(1)$ in the number of tasks [2502.11986].
- **Schedule Design and Hyperparameter Sensitivity**: Hand-tuned schedules can outperform learned schedules in specific settings, but meta-learned schedules generalize better as the task count and task diversity grow [1909.06434].
- **Sensitivity to Task Definition**: Effectiveness relies on auxiliary tasks being positively correlated with the main task. Poorly chosen or noisy auxiliary tasks can induce negative transfer even with SML [2502.11986].
- **Scalability**: Grouping strategies and meta-policies scale to $O(10^2)$ tasks but may require adaptation for extreme multitask settings or with very large task-specific parameter sets.

## 7. Connections and Future Directions

SML is closely related to curriculum learning, multi-objective optimization, adaptive loss weighting, and meta-learning. Recent work situates SML as unifying pre-training, standard MTL, and fine-tuning within a single procedural framework [1804.08915, 2205.03766]. A plausible implication is that further integration with large language model pre-training, more expressive meta-schedulers (e.g., transformer-based), and automatic curriculum discovery will continue to expand the domain of applicability.

References:
- "Scheduled Multi-task Learning for Neural Chat Translation" [2205.03766]
- "Selective Task Group Updates for Multi-Task Optimization" [2502.11986]
- "Adaptive Scheduling for Multi-Task Learning" [1909.06434]
- "Scheduled Multi-Task Learning: From Syntax to Translation" [1804.08915]
- "Temporally Correlated Task Scheduling for Sequence Learning" [2007.05290]
- "Learning to Multi-Task by Active Sampling" [1702.06053]
- "Self-Paced Multi-Task Learning" [1604.01474]

Source: https://www.emergentmind.com/topics/scheduled-multi-task-learning-sml