PHN-EPO: Exact Pareto Optimal Descent

• PHN-EPO is a first-order multi-objective optimization framework that traces exact Pareto-optimal points through a smooth two-phase descent-and-balancing dynamic.
• The method overcomes stagnation and oscillation in traditional gradient approaches by first balancing objectives according to user preferences and then executing a joint descent along the Pareto front.
• PHN-EPO guarantees provable convergence and achieves linear convergence rates under standard conditions, making it applicable to multi-task learning and multi-criteria decision-making.

Exact Pareto Optimal Descent (PHN-EPO) is a first-order optimization framework for multi-objective optimization (MOO) that locates or traces the exact Pareto-optimal point corresponding to a user’s prescribed preferences, even in the presence of non-convex objectives. PHN-EPO overcomes the stagnation and oscillation commonly found in gradient-based methods employing classical Chebyshev scalarization, by introducing a smooth two-phase descent-and-balancing dynamic in objective space. The method ensures provable convergence to the desired Pareto-optimal solution and achieves linear convergence rates when tracing along the Pareto front (PF) under standard regularity conditions. PHN-EPO forms a foundational primitive for multi-task learning (MTL), multi-criteria decision-making (MCDM), and preference-guided optimization in deep networks (Mahapatra et al., 2021).

1. Chebyshev Scalarization and the Exact Pareto-Optimal Criterion

Given $m$ differentiable objectives $f_1(x), \ldots, f_m(x) \geq 0$, for $x \in \mathbb{R}^n$, and a positive preference vector $r \in \mathbb{R}_+^m$, Chebyshev scalarization is defined by

$$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$

Any global minimizer $x^*$ of $g(x;r)$ satisfies the proportionality condition,

$$r_1 f_1(x^*) = r_2 f_2(x^*) = \cdots = r_m f_m(x^*),$$

and $x^*$ is Pareto-optimal; such a solution is called an Exact Pareto-Optimal (EPO) point for $r$. The vector $f(x^*)$ lies on the ray parallel to $r^{-1} = (1/r_1, \ldots, 1/r_m)$ in objective space (Mahapatra et al., 2021).

2. Balancing and Descent in PHN-EPO: The Two-Phase Strategy

PHN-EPO eschews direct minimization of the non-smooth Chebyshev scalarization in favor of a smooth two-stage approach operating in the $m$-dimensional objective space:

• Balance Mode: Reduce a smooth proportionality gauge $\omega(f; r)$ that quantifies the imbalance of $r \circ f$ until objectives are proportional to $r^{-1}$ up to a user tolerance.
• Descent Mode: Once proportionality is achieved, reduce all objectives simultaneously, driving the solution further along the selected PF ray.

Two gauges for imbalance are notably effective:

• Cauchy–Schwarz gauge: $$\omega_C(f; r) = \frac{1}{2}\left(1 - \langle f / \|f\|,\, r^{-1} / \|r^{-1}\| \rangle^2 \right)$$,
• Lagrange-identity gauge: $$\omega_L(f; r) = \frac{\|f\|^2 \|r^{-1}\|^2 - \langle f, r^{-1} \rangle^2}{2 \|r^{-1}\|^2}$$.

The gradient $\nabla_f \omega(f; r)$ serves as the balancing anchor direction $a_{\text{bal}}(f)$ during balance mode; in descent mode the anchor is simply $a_{\text{des}}(f) = f$. This construction guarantees every iteration moves toward proportionality, then toward Pareto improvement.

3. Gradient-Based Search: Quadratic Program Formulation

At each iteration $t$ with parameter $x^t$, objectives $f^t = f(x^t)$, and Jacobian $F = \nabla_x f(x^t) \in \mathbb{R}^{m \times n}$, PHN-EPO constructs a search direction via a constrained quadratic program (QP):

1. Select anchor $a \in \mathbb{R}^m$:
   • Balance: $$a = a_{\text{bal}}(f^t),\; J = \{ \arg \max_j [ r_j f^t_j ] \}$$,
   • Descent: $a = f^t,\; J = \{ 1,\ldots, m \}$.
2. Solve

$$\min_{\|\beta\|_1 \leq 1} \| F F^\top \beta - a \|^2 \quad \text{subject to } \beta^\top F^\top \nabla_x f_j \geq 0,\, \forall j \in J.$$

Set $d_{\text{nd}} = F^\top \beta^\ast$ and update $x^{t+1} = x^t - \eta d_{\text{nd}}$.

This mechanism allows PHN-EPO to blend ascent (for underweighted tasks) and descent to achieve proportionality and subsequently strict descent along the Pareto ray. The $\ell_1$ constraint $\|\beta\|_1 \leq 1$ ensures stable, bounded updates.

4. Algorithmic Variants

PHN-EPO admits two main algorithmic instantiations:

Variant	Initialization	Anchor Schedule	Constraints
Algorithm A	Arbitrary $x^0$	Balance till $\omega \leq \epsilon_1$, then Descent	In balance: max set; in descent: all objectives
Algorithm B	$x^0$ on PF	Alternate anchors every step, converge in balance steps	Optionally add equality constraint for descent along $r^{-1}$

Algorithm A is generic and robust to initialization; Algorithm B is designed for controlled tracing along the PF and supports a single equality constraint that ensures moves are confined exactly to the selected ray direction when required (Mahapatra et al., 2021).

5. Convergence Properties

Under standard conditions (differentiable $f_j$, compact image set, full-rank $F$ off the PF), PHN-EPO is globally convergent:

• The set

$$A^r_{f^t} = \{ f \in O \mid f \preceq \lambda^t r^{-1} \},\quad \lambda^t = \max_j (r_j f^t_j)$$

shrinks strictly with each balance-mode iteration.

• If an exact EPO exists, the sequence converges to it; otherwise, to the Pareto point best aligned with $r^{-1}$.
• On the PF, the proportionality objective contracts linearly for small $\eta$:

$$\omega(f^{t+1}) \leq (1 - c\eta + O(\eta^2)) \omega(f^t),$$

using a Polyak–Łojasiewicz–type structure, Lipschitz smoothness, and angle-bound arguments.

• The typical number of balance-mode steps to reach tolerance $\epsilon$ is $O(\log(1/\epsilon))$ (Mahapatra et al., 2021).

6. Computational Complexity and Scalability

Each PHN-EPO iteration involves:

• Jacobian formation: $O(n m)$,
• Gram-matrix multiplication ($F F^\top$): $O(n m^2)$,
• $\ell_1$-QP in $m$ variables and $O(m)$ constraints: $O(m^3)$ (solved by interior-point or active-set methods).

Total per-iteration complexity is $O(n m^2 + m^3)$, with memory $O(n m + m^2)$. Per-iteration cost scales linearly in decision dimension $n$ and quadratically-cubically in typically small number of tasks $m$.

7. Applications and Extensions

PHN-EPO provides a unified mechanism for:

• Approximating the PF in a posteriori MCDM (via PESA-EPO),
• Interactive preference elicitation (via GP-EPO with GP models over $r^{-1}$),
• Multi-task deep learning with user-specified priorities,
• Real-world deployment in personalized medicine, e-commerce, and hydrometeorology, where it demonstrates efficacy for deep MTL.

Each update involves all gradient directions, thereby avoiding stagnation and oscillation present in naive min-max gradient descent. Extensions to constraint handling are immediate (Mahapatra et al., 2021). A plausible implication is the suitability of PHN-EPO as a generic primitive for vector-valued optimization routines where robust control of user-weighted objectives is essential.