Projecting Conflicting Gradient

Updated 16 May 2026

Projecting conflicting gradients are an optimization approach that detects and neutralizes opposing gradient directions to prevent harmful interference in multi-objective systems.
It employs orthogonal projection methods, such as those used in PCGrad and SafeGrad, to adjust gradients so that each update contributes beneficially to all objectives.
Empirical studies show that this technique improves convergence rates and task performance in multi-task learning, reinforcement learning, and fairness-sensitive applications.

A projecting conflicting gradient is an optimization technique that detects directly opposed gradient directions arising from multiple objectives or tasks and modifies the optimization step by removing components of one gradient that would actively harm progress on another. This operation is foundational in multi-objective machine learning, reinforcement learning, and robust fine-tuning of LLMs, providing a rigorous solution to inter-objective interference, especially in safety-critical or fairness-sensitive optimization schemes.

1. Formal Definition and Detection of Gradient Conflict

A gradient conflict occurs when the gradient vectors of two objectives, $g_i$ and $g_j$ , are oriented in such a way that progress on one directly harms the other. The canonical test is

$g_i^\top g_j < 0$

or, equivalently, cosine similarity

$\cos\phi_{ij} = \frac{g_i^\top g_j}{\|g_i\|\|g_j\|} < 0.$

This condition implies that following $g_j$ would increase the value of the loss associated with $g_i$ , and vice versa, leading to undesirable oscillations, deadlocks, or stalling in plain-vanilla gradient descent, especially when the conflicting directions have similar magnitudes or under large curvature in the loss landscape (Yu et al., 2020, Wang et al., 2 Feb 2025). In multi-task settings, conflict prevalence is a key limiter of joint training gains.

2. Orthogonal Projection Approach: Core Methodology

Orthogonal projection resolves a two-way gradient conflict by removing from $g_i$ its component along $g_j$ , leaving only the part that is neutral to the conflicting objective. The update is

$g_i^{\text{proj}} = g_i - \frac{g_i^\top g_j}{\|g_j\|^2}\, g_j$

which ensures that the adjusted $g_i^{\text{proj}}$ is orthogonal to $g_j$ 0, i.e., $g_j$ 1. This operation is the algebraic foundation of gradient surgery techniques such as PCGrad (Yu et al., 2020), SafeGrad (Yi et al., 10 Aug 2025), Ortho-LoRA (Yang et al., 14 Jan 2026), FairGrad (Wang et al., 19 Apr 2025), and analogous extensions to subspace (multi-gradient) cases (Zhu et al., 5 Mar 2025).

For $g_j$ 2 tasks (objectives), the generalization requires projecting each $g_j$ 3 onto the orthogonal complement of the subspace spanned by $g_j$ 4:

$g_j$ 5

with $g_j$ 6 the canonical subspace projector, typically computed via Gram-Schmidt or solving linear systems in the Gram matrix of task gradients (Zhu et al., 5 Mar 2025, Wang et al., 19 Apr 2025).

3. Algorithmic Implementations and Notable Variants

Numerous algorithmic frameworks have emerged, implementing projecting conflicting gradient principles for distinct applications and optimization regimes:

PCGrad: Sequential, randomized projection of each task gradient onto the normal plane of others when pairwise conflicts are detected; summed for the final update (Yu et al., 2020).
SafeGrad: Identifies and projects away the user-task component along the safety-alignment gradient in LLM fine-tuning when conflict is detected, ensuring $g_j$ 7 without impeding benign user learning (Yi et al., 10 Aug 2025).
Ortho-LoRA: In multi-task LoRA, pairwise orthogonal projections are independently applied to each LoRA factor, with random task order to minimize systemic bias in conflict removal (Yang et al., 14 Jan 2026).
Gradient Deconfliction via Orthogonal Projection onto Subspaces (GradOPS): Each gradient is projected onto the orthogonal complement of the subspace spanned by others, guaranteeing non-conflict among all pairs and enabling flexible Pareto trade-off reweighting (Zhu et al., 5 Mar 2025).

Typical pseudocode structure includes conflict detection, projection, aggregation (sum or Pareto-weighted), and step update. Computational complexity is dominated by gradient calculation ( $g_j$ 8) and projection operations (up to $g_j$ 9 for naive PCGrad, but often constant-factor overhead in practical settings).

4. Theoretical Guarantees and Optimization Properties

Projecting conflicting gradients guarantees, under standard smoothness assumptions, that each task gradient contributes a nonnegative descent direction for its objective, eliminating destructive interference. For subspace projections, $g_i^\top g_j < 0$ 0 for all $g_i^\top g_j < 0$ 1, ensuring that no update increases any constituent loss to first order (Zhu et al., 5 Mar 2025, Wang et al., 19 Apr 2025). Convergence proofs establish that iterates approach Pareto-stationary points—the condition that no common direction yields simultaneous descent in all objectives:

$g_i^\top g_j < 0$ 2

Global convergence rates for these methods match those of standard stochastic gradient descent, and empirically, convergence is accelerated in the presence of persistent conflict. Variants such as SafeGrad prove that the orthogonal projection prevents safety loss regression in LLMs even under high proportions of poison data (Yi et al., 10 Aug 2025).

Projecting conflicting gradients is foundational to a family of multi-objective optimization techniques:

Method	Principle	Conflict Handling	Trade-off Control	Distinctive Feature
PCGrad	Pairwise projection	Pairwise neutralization	Randomized order	Simple, versatile (Yu et al., 2020)
GradOPS	Full subspace orthogonal projection	Complete decorrelation	Weight parameter $g_i^\top g_j < 0$ 3	No need to solve QPs (Zhu et al., 5 Mar 2025)
RACO-Clip	Clipped conflict-averse combination	Pareto anchor + clipping	User weights $g_i^\top g_j < 0$ 4, clipping	Strictly respects user-specified trade
SafeGrad	Conflict-aware in safety/task LLM	User-task vs align proj.	Alignment weight $g_i^\top g_j < 0$ 5	Provable safety invariance (Yi et al., 10 Aug 2025)
Ortho-LoRA	LoRA factorwise projection	Factor-specific	Random shuffle	Low-rank MTL, minimal overhead (Yang et al., 14 Jan 2026)

Hard-projection approaches can be computationally expensive for many tasks ( $g_i^\top g_j < 0$ 6), motivating stochastic or compressed forms (e.g., sublinear filtering in CONGRAD (Li et al., 31 Mar 2025)) and dynamic trade-off variants (e.g., cone-constrained optimization in CONICGRAD (Hassanpour et al., 31 Jan 2025)).

Alternatives such as CAGrad, MGDA, IMTL-G, or HRGrad employ different geometric, dual, or rotational strategies to resolve multi-way conflict, each with distinct regularity, trade-off optimality, and computational/robustness profiles (Hassanpour et al., 31 Jan 2025, Liang, 27 Apr 2026).

6. Empirical Impact and Applications

Empirical validations span multi-task supervised learning, reinforcement learning, LLM fine-tuning, 3D scene representation, and fairness-aware healthcare AI.

Multi-task SL and RL: PCGrad, GradOPS, and CONICGRAD consistently achieve higher task accuracy, lower mean rank, and better negative transfer avoidance than naively summed or weighted updates (Yu et al., 2020, Zhu et al., 5 Mar 2025, Hassanpour et al., 31 Jan 2025).
Safe Fine-tuning of LLMs: SafeGrad maintains low harmful scores ( $g_i^\top g_j < 0$ 7 at $g_i^\top g_j < 0$ 8 poison ratio) with no decrement in finetune accuracy, decisively outperforming magnitude-balancing, constraint-annealing, or reward-weighted approaches (Yi et al., 10 Aug 2025).
Low-rank adaptation: Ortho-LoRA effectively recovers the majority of the gap to single-task performance in GLUE, with negligible computational overhead (Yang et al., 14 Jan 2026).
Fairness in Healthcare: FairGrad reduces equalized-odds differences ( $g_i^\top g_j < 0$ 9–48%) at less than $\cos\phi_{ij} = \frac{g_i^\top g_j}{\|g_i\|\|g_j\|} < 0.$ 0 relative loss in AUROC (Wang et al., 19 Apr 2025).
3DGS Rendering: Direction-aware projection in GDAGS improves rendering quality while halving memory usage by focusing density control on conflicting-gradient regions (Zhou et al., 12 Aug 2025).

These gains are realized by ensuring that no task or constraint is antagonized during joint optimization.

7. Current Limits and Open Challenges

Despite their efficacy, projecting conflicting gradients remains computationally intensive for high task counts, and may induce convergence to Pareto-stationary points that are not globally optimal for any single objective. Subspace-projection approaches depend on effective detection and ranking of conflicts, which can become ill-conditioned as tasks proliferate or data distributions diverge.

Projection-based methods often require careful scheduling, reweighting, or approximate projections for scalability, and the trade-off between fairness/robustness and task accuracy remains an open topic for further investigation (Wang et al., 19 Apr 2025, Hassanpour et al., 31 Jan 2025). Moreover, extension to non-Euclidean geometries, continuous task spectra (as in HRGrad (Liang, 27 Apr 2026)), and high-dimensional preference alignment (as in LLMs) represent ongoing research frontiers.

In summary, projecting conflicting gradients is the central paradigm for eliminating destructive interference between objectives in multi-objective, multi-task, and robust optimization. Through geometric projection, it guarantees first-order non-interference and provides a rigorous basis for trade-off control in complex, conflicting settings (Yi et al., 10 Aug 2025, Yu et al., 2020, Zhu et al., 5 Mar 2025, Wang et al., 19 Apr 2025, Yang et al., 14 Jan 2026, Wang et al., 2 Feb 2025).