Singularities in Multi-Objective Optimization and their Crossing during Continuation

Abstract: Continuation methods help trace Pareto sets in multi-objective optimization but are inherently local: a single run traces a single connected branch, requiring multiple restarts to recover disconnected components of Pareto fronts. We show that, for unconstrained bi-objective problems under weighted-sum scalarization, these disconnects can be artifacts of singularities in the scalarization parameter, where the weight $λ$ diverges as the objective gradients become collinear. Recasting Pareto optimality as a nonlinear system, we apply pseudo-arclength continuation to follow the Pareto-critical set, and show that suitable singular reparameterizations allow crossing these singularities in systematically, recovering disconnected branches in a single run. A coordinate-wise projective compactification further provides a unified framework for parameter and decision-space variables. We demonstrate the approach on the ZDT3 benchmark and modifications.

• The paper shows that disconnected Pareto fronts arise from coordinate singularities rather than true topological breaks.
• It introduces angular compactification (reparameterizing λ as tanθ) to cross singularities seamlessly during continuation.
• Furthermore, the study generalizes these techniques to decision-space singularities, reducing manual restarts and enhancing MOO algorithm efficiency.

Singularities in Multi-Objective Optimization and Their Crossing during Continuation

Introduction

Multi-objective optimization (MOO) emerges ubiquitously across scientific domains where trade-offs between competing criteria are intrinsic to the underlying system. Pareto sets, and their projection as the Pareto front in objective space, encode the landscape of optimal compromises. Commonly, practitioners rely on scalarization—most notably, weighted-sum-based approaches or related extensions—to convert the vector-valued objective into a tractable scalar problem. However, such approaches typically recover only the convex portions of the Pareto front and often require repeated initializations and restarts to reconstruct disconnected or non-convex regions. The apparent branch disconnections in Pareto-critical manifolds challenge systematic sampling and have substantial implications for both algorithmic completeness and interpretability in MOO.

This paper analyzes the root causes of these disconnected solution branches under continuation-based methodologies, with a focus on the widely used pseudo-arclength continuation. The central contribution is the demonstration that these disconnects often arise not from genuine topological breaks but instead from coordinate singularities in the scalarization parameterization, such as the divergence of the weight $\lambda$ when objective gradients become collinear. By leveraging angular and projective compactification techniques, the authors show how to cross such singularities systematically, enabling complete traversal of the Pareto-critical manifold—including previously inaccessible, disconnected regions—within a single continuation run. These insights are demonstrated extensively on the ZDT3 benchmark and its modifications, providing a unifying geometric perspective on parameter and decision space singularities.

Background and Problem Formulation

For unconstrained bi-objective optimization, the canonical approach is to minimize $f_1(x)$ and $f_2(x)$ over a feasible set, identifying the Pareto-optimal set where no objective can be improved without degrading the other. Scalarization via weighted sums produces a reduced optimization of the form $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$, associated with the first-order optimality condition $$\lambda \nabla f_1(x) + (1-\lambda) \nabla f_2(x) = 0$$, typically solved using continuation in the augmented $(x, \lambda)$ space. While well established, this method inherently traces only the connected branch reachable from a given initialization. Non-convex and disconnected Pareto fronts, such as those exhibited by the ZDT3 test problem, cause this approach to miss substantial portions of the Pareto set unless manual restarts are performed.

A key technical observation is that singularities in the parameterization—specifically, divergence of the scalarization weight $\lambda$ as objective gradients become collinear—cause apparent discontinuities and fragmentation of the continuation trajectory. These "disconnects" do not correspond to intrinsic gaps in the Pareto-critical set, but rather result from removable coordinate singularities.

Analysis of Parameter Singularities and Compactification

$\lambda$-Singularities and Their Origin

Using the ZDT3 bi-objective formulation, the authors show that tracing the Pareto-critical set in $(x, \lambda)$ space via pseudo-arclength continuation yields multiple disconnected segments Figure 1. These disconnects coincide precisely with loci where the gradients $\nabla f_1(x)$ and $f_1(x)$0 become collinear, resulting in $f_1(x)$1.

Figure 1: Disconnected Pareto front of ZDT3 under classical scalarization, with multiple disconnected solution branches resulting from $f_1(x)$2-singularities.

By considering the explicit form $f_1(x)$3, the divergence point is easily seen to correspond to $f_1(x)$4. Standard continuation fails here because an unbounded parameter cannot be crossed with a finite step.

Angular Compactification: $f_1(x)$5

To regularize the divergence, the authors introduce angular compactification: reparameterizing $f_1(x)$6 as $f_1(x)$7, which maps $f_1(x)$8 to the finite points $f_1(x)$9. This removes the divergence and renders the previously disconnected branches as a single smooth, traversable manifold in $f_2(x)$0 coordinates Figure 2. The coordinate transformation ensures that the leading-order unbounded divergence cancels, and enables pseudo-arclength continuation to pass seamlessly through the $f_2(x)$1-singularity.

Figure 2: Angular compactification ($f_2(x)$2) regularizes the singularity, yielding a single smooth branch in $f_2(x)$3 while the $f_2(x)$4 projection remains fragmented.

Direct Infinity Crossing via Projective Reparameterization

An alternative to angular compactification is to reparameterize near the singularity with $f_2(x)$5, converting the singular ODE into the regular $f_2(x)$6. This transformation permits "crossing" $f_2(x)$7 by integrating through $f_2(x)$8, producing a reflection that continues the solution trajectory on the next branch without manual intervention Figure 3. The strategy is robust and utilizes only local second-derivative information.

Figure 3: Infinity crossing in $f_2(x)$9 for ZDT3 using $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$0 reparameterization; the singularity is crossed smoothly in $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$1.

Extension to Decision Space and General Compactification

The methodology is further generalized to accommodate singularities arising in decision variables, not just in the scalarization parameter. A modified two-dimensional ZDT3 is employed to demonstrate this scenario, where excursions to large $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$2 and repeated $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$3-singularities are induced by oscillatory objective terms and domain extension. The paper introduces a coordinate-wise projective compactification, with each variable assigned an independent projective chart (e.g., $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$4 with proper normalization), preventing the collapse of distinct infinities to a common boundary. This approach preserves the geometric distinction between different types of singularities—enabling simultaneous traversal of both parameter and state space infinities in a manner that maintains solution branch identity Figure 4.

Figure 4: Coordinate-wise projective compactification enables Pareto-critical continuation to traverse both parameter ($$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$5) and state ($$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$6) infinities, recovering the full structure within a single branch.

Supplementary results confirm that this framework applies to a wide class of objectives, higher-dimensional settings, and to both angular and projective compactifications (Figures 5–14), establishing the method's generality and robustness.

Implications and Future Directions

This work provides a unified geometric interpretation of apparent branch disconnects in continuation-based MOO under scalarization. The capacity to traverse parameter and decision variable singularities algorithmically obviates the need for ad hoc manual restarts and multiple initializations, greatly improving the completeness and efficiency of Pareto front exploration in high-stakes applications such as multi-objective optimal control, systems engineering, and computational chemistry.

The formalism not only clarifies the nature of apparent singularities, distinguishing them from true topological breaks, but also enables the extension of continuation-based techniques to non-convex and disconnected regions of the Pareto set previously considered challenging or unreachable. For practical deployment in algorithmic toolchains, this approach reduces the burden on users and removes a significant barrier to the widespread adoption of geometric continuation in MOO.

On the theoretical front, the capacity to traverse coordinate singularities invites further study of genuinely disconnected Pareto-critical manifolds, and their algorithmic identification in cases where topology, rather than coordinate artifacts, causes true branch separation. Additionally, integration of these methods within robust, general-purpose MOO solvers appears promising, particularly for high-dimensional and non-smooth objective landscapes.

Conclusion

The paper rigorously demonstrates that disconnected Pareto fronts seen in standard scalarization-based continuation for MOO are typically artifacts of parameterization singularities—specifically, divergence of the scalarization parameter $$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$7 as objective gradients become collinear. These singularities are shown to be removable using angular compactification ($$g(x; \lambda) = \lambda f_1(x) + (1 - \lambda) f_2(x)$$8) or appropriate projective reparameterization, both yielding single, connected, and smoothly traversable Pareto-critical manifolds. This eliminates the necessity for multiple initializations and manual restarts and recovers the full solution structure, extending naturally to cases with decision-space singularities via coordinate-wise projective compactification. The approach does not resolve truly topologically disconnected Pareto sets, which remain a topic for future inquiry. The methods and conceptual insights provided have substantial implications for advancing both the theory and practice of multi-objective optimization.