PHN-EPO: Exact Pareto Optimal Descent

• PHN-EPO is a first-order multi-objective optimization framework that traces exact Pareto-optimal points through a smooth two-phase descent-and-balancing dynamic.
• The method overcomes stagnation and oscillation in traditional gradient approaches by first balancing objectives according to user preferences and then executing a joint descent along the Pareto front.
• PHN-EPO guarantees provable convergence and achieves linear convergence rates under standard conditions, making it applicable to multi-task learning and multi-criteria decision-making.

Exact Pareto Optimal Descent (PHN-EPO) is a first-order optimization framework for multi-objective optimization (MOO) that locates or traces the exact Pareto-optimal point corresponding to a user’s prescribed preferences, even in the presence of non-convex objectives. PHN-EPO overcomes the stagnation and oscillation commonly found in gradient-based methods employing classical Chebyshev scalarization, by introducing a smooth two-phase descent-and-balancing dynamic in objective space. The method ensures provable convergence to the desired Pareto-optimal solution and achieves linear convergence rates when tracing along the Pareto front (PF) under standard regularity conditions. PHN-EPO forms a foundational primitive for multi-task learning (MTL), multi-criteria decision-making (MCDM), and preference-guided optimization in deep networks (Mahapatra et al., 2021).

1. Chebyshev Scalarization and the Exact Pareto-Optimal Criterion

Given $m$ differentiable objectives $f_1(x), \ldots, f_m(x) \geq 0$, for $x \in \mathbb{R}^n$, and a positive preference vector $r \in \mathbb{R}_+^m$, Chebyshev scalarization is defined by

$$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$

Any global minimizer $x^*$ of $g(x;r)$ satisfies the proportionality condition,

$$r_1 f_1(x^*) = r_2 f_2(x^*) = \cdots = r_m f_m(x^*),$$

and $x^*$ is Pareto-optimal; such a solution is called an Exact Pareto-Optimal (EPO) point for $r$. The vector $f_1(x), \ldots, f_m(x) \geq 0$0 lies on the ray parallel to $f_1(x), \ldots, f_m(x) \geq 0$1 in objective space (Mahapatra et al., 2021).

2. Balancing and Descent in PHN-EPO: The Two-Phase Strategy

PHN-EPO eschews direct minimization of the non-smooth Chebyshev scalarization in favor of a smooth two-stage approach operating in the $f_1(x), \ldots, f_m(x) \geq 0$2-dimensional objective space:

• Balance Mode: Reduce a smooth proportionality gauge $f_1(x), \ldots, f_m(x) \geq 0$3 that quantifies the imbalance of $f_1(x), \ldots, f_m(x) \geq 0$4 until objectives are proportional to $f_1(x), \ldots, f_m(x) \geq 0$5 up to a user tolerance.
• Descent Mode: Once proportionality is achieved, reduce all objectives simultaneously, driving the solution further along the selected PF ray.

Two gauges for imbalance are notably effective:

• Cauchy–Schwarz gauge: $f_1(x), \ldots, f_m(x) \geq 0$6,
• Lagrange-identity gauge: $f_1(x), \ldots, f_m(x) \geq 0$7.

The gradient $f_1(x), \ldots, f_m(x) \geq 0$8 serves as the balancing anchor direction $f_1(x), \ldots, f_m(x) \geq 0$9 during balance mode; in descent mode the anchor is simply $x \in \mathbb{R}^n$0. This construction guarantees every iteration moves toward proportionality, then toward Pareto improvement.

3. Gradient-Based Search: Quadratic Program Formulation

At each iteration $x \in \mathbb{R}^n$1 with parameter $x \in \mathbb{R}^n$2, objectives $x \in \mathbb{R}^n$3, and Jacobian $x \in \mathbb{R}^n$4, PHN-EPO constructs a search direction via a constrained quadratic program (QP):

1. Select anchor $x \in \mathbb{R}^n$5:
   • Balance: $x \in \mathbb{R}^n$6,
   • Descent: $x \in \mathbb{R}^n$7.
2. Solve

$x \in \mathbb{R}^n$8

Set $x \in \mathbb{R}^n$9 and update $r \in \mathbb{R}_+^m$0.

This mechanism allows PHN-EPO to blend ascent (for underweighted tasks) and descent to achieve proportionality and subsequently strict descent along the Pareto ray. The $r \in \mathbb{R}_+^m$1 constraint $r \in \mathbb{R}_+^m$2 ensures stable, bounded updates.

4. Algorithmic Variants

PHN-EPO admits two main algorithmic instantiations:

Variant	Initialization	Anchor Schedule	Constraints
Algorithm A	Arbitrary $r \in \mathbb{R}_+^m$3	Balance till $r \in \mathbb{R}_+^m$4, then Descent	In balance: max set; in descent: all objectives
Algorithm B	$r \in \mathbb{R}_+^m$5 on PF	Alternate anchors every step, converge in balance steps	Optionally add equality constraint for descent along $r \in \mathbb{R}_+^m$6

Algorithm A is generic and robust to initialization; Algorithm B is designed for controlled tracing along the PF and supports a single equality constraint that ensures moves are confined exactly to the selected ray direction when required (Mahapatra et al., 2021).

5. Convergence Properties

Under standard conditions (differentiable $r \in \mathbb{R}_+^m$7, compact image set, full-rank $r \in \mathbb{R}_+^m$8 off the PF), PHN-EPO is globally convergent:

• The set

$r \in \mathbb{R}_+^m$9

shrinks strictly with each balance-mode iteration.

• If an exact EPO exists, the sequence converges to it; otherwise, to the Pareto point best aligned with $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$0.
• On the PF, the proportionality objective contracts linearly for small $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$1:

$$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$2

using a Polyak–Łojasiewicz–type structure, Lipschitz smoothness, and angle-bound arguments.

• The typical number of balance-mode steps to reach tolerance $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$3 is $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$4 (Mahapatra et al., 2021).

6. Computational Complexity and Scalability

Each PHN-EPO iteration involves:

• Jacobian formation: $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$5,
• Gram-matrix multiplication ($$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$6): $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$7,
• $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$8-QP in $$\min_{x}\; g(x;r), \quad \text{where} \quad g(x;r) := \max_{j=1,\ldots,m} [\, r_j f_j(x) \,].$$9 variables and $x^*$0 constraints: $x^*$1 (solved by interior-point or active-set methods).

Total per-iteration complexity is $x^*$2, with memory $x^*$3. Per-iteration cost scales linearly in decision dimension $x^*$4 and quadratically-cubically in typically small number of tasks $x^*$5.

7. Applications and Extensions

PHN-EPO provides a unified mechanism for:

• Approximating the PF in a posteriori MCDM (via PESA-EPO),
• Interactive preference elicitation (via GP-EPO with GP models over $x^*$6),
• Multi-task deep learning with user-specified priorities,
• Real-world deployment in personalized medicine, e-commerce, and hydrometeorology, where it demonstrates efficacy for deep MTL.

Each update involves all gradient directions, thereby avoiding stagnation and oscillation present in naive min-max gradient descent. Extensions to constraint handling are immediate (Mahapatra et al., 2021). A plausible implication is the suitability of PHN-EPO as a generic primitive for vector-valued optimization routines where robust control of user-weighted objectives is essential.