Three-Way Pareto Frontier

• Three-way Pareto frontier is a generalization of two-objective Pareto optimality, representing trade-offs among three conflicting objectives within a two-dimensional manifold in 3D space.
• It rigorously defines Pareto-optimality using dominance concepts and hypervolume metrics to evaluate solution distribution and ensure efficient coverage of the feasible region.
• The concept underpins algorithms in multi-objective optimization, benchmarking, and online learning, guiding uniform approximations and robust efficiency assessments while balancing conflicting criteria.

A three-way Pareto frontier generalizes the classical two-objective Pareto optimality concept to the case of three objectives or competing criteria. In this context, a solution is said to be Pareto-optimal if no objective can be improved without worsening at least one of the others. The resulting frontier is typically a two-dimensional surface embedded within three-dimensional objective space, separating feasible solutions that are non-dominated from those that are dominated in all objectives. Three-way Pareto analysis arises in multi-objective optimization (continuous, integer, and mixed-integer domains), evolutionary optimization, performance evaluation of machine learning models, online learning (multi-armed bandits), and in algorithmic approaches for uniform or hypervolume-optimal sampling of fronts.

1. Formal Characterization of the Three-Way Pareto Frontier

In three-objective optimization, let each solution $x$ map to an objective vector $F(x) = (f_1(x), f_2(x), f_3(x)) \in \mathbb{R}^3$. A solution $x^*$ is Pareto-optimal if there does not exist $y$ such that $F(y)$ weakly dominates $F(x^*)$ (componentwise $f_i(y) \leq f_i(x^*)$ for all $i$, with strict inequality for at least one component).

The resulting three-way Pareto frontier is the set of all such non-dominated $F(x^*)$, forming a two-dimensional manifold in $\mathbb{R}^3$. In applications such as neural architecture search or bandit regret balancing, this frontier provides a rigorous surface of achievable trade-offs—e.g., between accuracy, latency, and cost, or between regrets for three actions.

Within the context of benchmarking, the three-way frontier is constructed by normalizing all objectives (when objectives are of differing types or orders of magnitude), and defining dominance under the convention of which objectives are to be minimized or maximized. For example, with accuracy ($A$) maximized and latency ($L$) and cost ($C$) minimized, a candidate $(C_i, L_i, A_i)$ dominates $(C_j, L_j, A_j)$ if $C_i \leq C_j$, $L_i \leq L_j$, $A_i \geq A_j$, with at least one strict (Nia et al., 2022).

2. Hypervolume and Distribution of Solutions on 3D Fronts

Hypervolume (HV) is a dominant metric for evaluating discrete approximations of the Pareto frontier in three dimensions, capturing the total volume in objective space dominated by the solution set and bounded by a reference point. For a Pareto set $$A = \{x^{(1)},\dots, x^{(\mu)}\}\subset \mathbb{R}^3$$ with $r \ll 0$,

$$HV(A, r) = \mathrm{Volume}\bigcup_{a\in A}\;[r_1, a_1]\times [r_2, a_2] \times [r_3, a_3]$$

is commonly used as a target for maximizing both the diversity and quality of the set.

Key findings on where to optimally place the $\mu$ solutions on various 3D Pareto fronts include:

• Line-based fronts: For a single-line front with one constant objective (e.g., $f_2=0, f_1+f_3=1$), uniform spacing maximizes HV. For fronts where two objectives increase together, uniform spacing is not HV-optimal, and the optimal positions are non-uniform (Shang et al., 2021).
• Multi-segment (L-shape, triangular rim) fronts: When the front decomposes into orthogonal line segments, optimality is achieved by allocating points uniformly and proportionally across line segments, using 2D theory for each segment.
• Plane-based (triangular simplex) fronts: Uniform grid constructions (DAS) provide locally HV-optimal sets and are globally optimal for small $\mu$ (e.g., $H=1,2$). For higher $\mu$, slight non-uniform adjustments can further improve HV, as demonstrated by local search (e.g., SMS-EMOA). Uniformity is a local optimum under $(\mu+1)$ selection, but not always the global maximum (Shang et al., 2021).

3. Algorithms for Three-Way Pareto Front Approximation

Three-way Pareto frontier approximation requires specialized algorithmic frameworks for different problem settings:

• Black-box multi-objective optimization: Trust-region methods utilizing reference-point selection based on a density function, combined with derivative-free quadratic models and Pascoletti-Serafini scalarization, are effective for producing uniformly distributed non-dominated sets on the three-objective front. The approach iteratively explores sparsely-populated areas, converging to Pareto-critical points and delivering nearly uniform approximations across various testbeds (e.g., DTLZ2, DTLZ7) (Ju et al., 2022).
• Integer and mixed-integer programming: Scalarization models such as weighted sum, $\epsilon$-constraint, Pascoletti-Serafini, and—most powerfully—weighted-constraint (WC) allow grid-based exploration of the Pareto surface. For the three-objective case, the CHIM grid coupled with WC scalarization recovers all weakly efficient points, including for disconnected or non-convex fronts. Theoretical surjectivity of the WC scalarization coupled with grid refinement ensures full coverage as the discretization becomes fine (Burachik et al., 2019).

Scalarization	Convexity Sensitivity	Disconnected Fronts	WC Surjectivity
Weighted sum	Sensitive	Misses portions	No
$\epsilon$-constraint	Moderately robust	Can cover	No
Weighted-constraint	Robust	Full coverage	Yes

Uniformity and coverage of the three-dimensional surface are quantitatively assessed via generational distance (GD) and attained hypervolume.

4. Benchmarking, Randomization, and Statistical Efficiency

In performance evaluation tasks (e.g., for deep neural networks), the three-way Pareto frontier formalizes trade-offs between cost, latency, and accuracy. Beyond classical domination, scalar efficiency scores based on Data Envelopment Analysis (DEA) can be computed by solving a convex program:

$$\min_{\theta,\;\lambda_1,\;\lambda_2}\;\theta \quad \text{s.t.}\; \sum_{l=1}^3 \lambda_{1,l}\,x_{l} \leq \theta\,x_{i,l},~ \lambda_2\,y \geq y_i,~ \sum \lambda = 1,~ \lambda \geq 0$$

where models with $\theta_i=1$ are Pareto-optimal (Nia et al., 2022).

Variability due to stochasticity in measurements (e.g., hardware latencies, training cost fluctuations) is incorporated via a parametric bootstrap, yielding a randomized three-way Pareto frontier. Statistical dominance is established via repeated sampling, and dominance probabilities or confidence intervals for relative efficiency are reported, making the assessment robust to noise and hardware/environmental variability.

Model	$\theta$ (GPU+CPU+Cost)	Variance
MobileNetV3	100	Low
ShuffleNetV2	100	Low
DualPathNet92	<20	Moderate

Trade-offs are explicit: once training cost is included, models such as ShuffleNetV2 may “overtake” others in DEA-based efficiency.

5. The Three-Way Pareto Regret Frontier in Online Learning

The three-objective Pareto frontier extends to sequential decision-making such as bandits. Here, the Pareto regret frontier describes the set of achievable worst-case regret budgets $B_1, B_2, B_3$ for three arms. The fundamental lower bound is

$$\forall\ i:~ B_i \geq c \left( \frac{n}{B_j} + \frac{n}{B_k} \right) \text{ for } i \neq j \neq k$$

and similarly for the others, where $n$ is the time horizon. No algorithm can simultaneously guarantee all regrets substantially below this boundary. A matching upper bound is achieved by the “unbalanced MOSS” strategy, which tunes exploration bias for each arm to hit the prescribed budgets. The admissible set forms the “three-way” regret Pareto frontier (Lattimore, 2015).

In symmetric allocation ($B_1 = B_2 = B_3 = b$), this gives $b \gtrsim \sqrt{2n}$, so increasing preference for one arm disproportionately drives up regret for the others.

6. Practical Implications, Guidance, and Limitations

For multi-objective optimization and benchmarking utilizing the three-way Pareto frontier:

• If the 3D front decomposes into independent line segments (as in "triangular rim" structures), distribute solutions uniformly according to 2D Pareto theory and allocate points based on proportionality for hypervolume maximization.
• For simplex-type (triangular) fronts, uniform constructions offer a strong baseline but are only globally optimal for low resolutions; local search or hypervolume-based ascent is necessary for accurate results at high resolution (Shang et al., 2021).
• In black-box optimization, density-driven exploration is key to maintaining uniform coverage, as direct ordering is unavailable beyond two objectives (Ju et al., 2022).
• DEA-based efficiency scoring, with randomness-aware extensions, is recommended for objective model rankings in the presence of experimental noise, but one must recognize the convexity assumptions and the dependence on measured metrics (Nia et al., 2022).
• In regret balancing, the three-way frontier quantifies the extreme tradeoffs incurred when attempting to simultaneously minimize the regret for several actions; diminishing one necessarily inflates the others in a superlinear fashion (Lattimore, 2015).

A plausible implication is that in three-objective problems, practitioners should always check for decomposability and possible non-uniform hypervolume-maximizing solution sets, and should utilize local optimization or density correction steps when uniformity is not globally optimal.

7. Open Questions and Future Directions

While substantial progress has been achieved in characterizing and optimizing three-way Pareto frontiers, several challenges remain:

• Efficient computation of globally hypervolume-optimal sets on general (non-simplicial, curved, or disconnected) fronts in high-resolution scenarios remains open.
• Systematic development of density or information-theoretic metrics for three-objective uniform distribution, especially in black-box or constrained settings, is ongoing (Ju et al., 2022).
• Extension of statistically robust Pareto benchmarking frameworks to settings with more than three objectives while avoiding domination collapse and interpretability loss.
• The adaptation of regret-Pareto tradeoff theory to adversarial or non-stationary online environments, with generalization to arbitrary subgroups of actions.
• Automated algorithms to detect, exploit, or correct non-uniform hypervolume suboptimality for specific front topologies.

Research in these directions will further clarify the interplay of fairness, diversity, and optimality in multi-criteria decision-making contexts involving three competing objectives.