---
title: 'CAGrad: Conflict-Averse Gradient Descent'
url: https://www.emergentmind.com/topics/conflict-averse-gradient-descent-cagrad
type: topic
---

# CAGrad: Conflict-Averse Gradient Descent

Conflict-Averse Gradient Descent (CAGrad) is a convex-optimization–based approach for combining multiple gradient signals—typically from either model ensembles in offline model-based optimization (MBO) or multiple tasks in multi-task learning—in a manner that systematically avoids destructive interference between objectives while maintaining convergence guarantees to the average objective. The method provides a principled mechanism to interpolate between plain averaging of gradients and Pareto-optimal multi-objective updates, governed by a single hyperparameter.

## 1. Formulation and Objective

The CAGrad update is defined, for either an ensemble of models or multiple loss functions, by constructing an update direction that maximizes the minimum improvement across tasks/models (worst-case linear improvement), yet remains close to the average gradient. Denoting the ensemble or set of tasks by $i = 1, \ldots, m$, and $g_i$ the individual gradients, the average gradient is $g_0 = \frac{1}{m} \sum_{i=1}^m g_i$. All update directions $d$ satisfy the proximity constraint $\|d - g_0\|_2 \le c \|g_0\|_2$, with $c \in [0,1)$ controlling the level of allowable deviation from the average.

The primal optimization is:
\[
\text{maximize}_{d \in \mathbb{R}^d, t \in \mathbb{R}} \;\; t 
\quad \text{subject to} \quad 
\langle d, g_i \rangle \ge t,\;\;\forall i;\;\; \|d - g_0\|_2 \leq c \|g_0\|_2.
\]

Strong duality yields an equivalent dual problem in the simplex-parameterized weight vector $w \in \Delta^m$:
\[
\text{minimize}_{w \succeq 0, \sum w_i=1} \;\; g_w^T g_0 + \sqrt{\phi}\|g_w\|_2,
\]
where $g_w = \sum_{i=1}^m w_i g_i$ and $\phi = c^2 \|g_0\|_2^2$. Once $w^*$ is found, the update is $d^* = g_0 + \frac{\sqrt{\phi}}{\|g_{w^*}\|_2} g_{w^*}$, or equivalently $d^* = g_0 + \frac{c \|g_0\|_2}{\|g_{w^*}\|_2} g_{w^*}$ [2303.17934, 2110.14048].

## 2. Theoretical Guarantees and Special Cases

CAGrad strictly preserves convergence to stationary points of the average objective—$F_{\text{avg}}(x) = \frac{1}{m} \sum_i \hat f_i(x)$ or $L_0(\theta) = \frac{1}{K} \sum_i L_i(\theta)$—for any $c \in [0,1)$. Formally, under standard Lipschitz gradient and smoothness assumptions, iterates converge to $\nabla F_{\text{avg}}(x^*)=0$, and the convergence rate of average norm-squared-gradient is inversely proportional to $(1-c^2)$ [2303.17934, 2110.14048].

Adjusting $c$ allows seamless interpolation between:

- $c=0$: $d^*=g_0$—the mean gradient (gradient descent/ascent)
- $c \to 1^-$: updates approach maximum robustness to conflicting gradients, resembling the Multiple Gradient Descent Algorithm (MGDA)
- $c \to \infty$ (theoretical): unconstrained Pareto-optimal directions

Thus, CAGrad encompasses plain GD and MGDA as limits [2110.14048].

## 3. Algorithm and Implementation

A single CAGrad iteration involves:

1. Computing the $m$ individual gradients $g_i$
2. Calculating $g_0$ and $\phi = c^2 \|g_0\|_2^2$
3. Solving the dual quadratic program in $w$ via standard constrained optimization routines (since $m$ is typically small)
4. Recovering $d^*$ from $w^*$
5. Updating $x$ (for MBO) or $\theta$ (for MTL) via $x_{k+1}=x_k+\alpha d^*$ or $\theta_{t+1}=\theta_t-\alpha d^*$

This routine is computationally efficient, with overhead per iteration scaling as $O(m^3)$ due to the small dual variable dimension [2303.17934].

Typical settings use $c$ in $[0.2,0.5]$; step sizes $\alpha$ are task-specific and require sufficient smallness to maintain the local approximation $\langle g_i,d\rangle \approx (1/\alpha)(\hat f_i(x+\alpha d)-\hat f_i(x))$ [2303.17934]. For continuous domains, input variables should be normalized. In discrete domains, CAGrad is applied in “soft” one-hot spaces and discretized via coordinate-wise argmax.

## 4. Context within Ensemble and Multi-Objective Optimization

CAGrad belongs to a family of gradient aggregation strategies for ensembles and multi-objective optimization. Key alternatives include:

| Scheme            | Update Direction                        |
|-------------------|-----------------------------------------|
| Mean gradient     | $g_0 = (1/m)\sum_i g_i$                |
| Minimum gradient  | $\nabla_x(\min_i \hat f_i(x))$         |
| MGDA              | $\arg\max_d [\min_i \langle d,g_i\rangle - (1/2)\|d\|^2]$ |
| CAGrad            | As described by the constrained max-min |

The mean gradient is vulnerable to pathologies caused by gradient conflict, leading to oscillatory or uninformative steps. The minimum-gradient method is highly conservative—reliably reducing the worst-case but prone to instability and slow progress. MGDA achieves Pareto-stationarity but lacks explicit control over deviation from the average objective. CAGrad maintains a tunable tether to the average gradient while enforcing worst-case improvement, synthesizing robust trade-offs [2303.17934].

## 5. Empirical Evaluation and Applications

CAGrad has been evaluated across diverse domains:

- **Offline Model-Based Optimization**: On five black-box design tasks with ensembles of proxy models, CAGrad outperformed mean and minimum aggregation in both average and median ground-truth scores, matching MGDA on maxima, while exhibiting greater stability on discrete search spaces and faster convergence on average objectives [2303.17934].
  
- **Multi-Task Learning**: On synthetic, supervised vision, reinforcement learning, and semi-supervised benchmarks, CAGrad demonstrated robust balancing of average loss and worst-case improvement. In particular, it yielded minimal performance drops on tasks typically neglected by alternative methods, and achieved state-of-the-art results, e.g., $\sim$83% success rate on Meta-World MT10 multi-task RL (vs. 72% for PCGrad), and improved semi-supervised accuracy by $\sim$1–2% over strong baselines [2110.14048].

A plausible implication is that CAGrad is particularly effective in settings with significant gradient conflict across objectives, and where robust average performance is critical.

## 6. Hyperparameter Selection and Practical Considerations

The conflict-aversion hyperparameter $c$ occupies the range $[0,1)$ and modulates the average/worst-case trade-off. Lower values prioritize fidelity to the average, whereas larger (but $<1$) values allow more assertive corrections for conflict. Theory requires $c<1$ for convergence to the average optimum; typical empirical values are in $[0.2,0.5]$ [2110.14048, 2303.17934]. Step size selection parallels standard practice in gradient methods, with specific adjustment to ensure local approximation validity.

In implementation, the dual QP overhead is negligible for small $m$. For very large ensembles or task counts, sub-sampling or approximate dual solvers may provide substantial speed-ups with modest performance cost [2110.14048].

## 7. Relationship to Broader Research and Extensions

CAGrad’s design reflects the broader multi-objective optimization literature, unifying convergence to average-objective optima (as in standard GD) and Pareto-stationarity (as in MGDA) within a single convex-analytic framework. It advances beyond heuristic gradient conflict mitigation strategies by providing theoretical guarantees and a tunable, interpretable mechanism for managing gradient disagreement.

Both in ensemble MBO and multi-task learning, CAGrad operationalizes a practical balance between optimism and pessimism in the face of model or task uncertainty, leading to empirically strong and theoretically justified performance envelopes [2303.17934, 2110.14048].

Source: https://www.emergentmind.com/topics/conflict-averse-gradient-descent-cagrad