NSGA-II: Multi-objective Optimization

• NSGA-II is a multi-objective optimization algorithm that efficiently approximates the Pareto front using evolutionary strategies and diversity preservation techniques.
• It employs fast non-dominated sorting and crowding distance metrics to balance convergence and solution spread in objective space.
• Improved crowding distance modifications enhance performance by better estimating solution density, especially in bi-objective and many-objective settings.

The Non-dominated Sorting Genetic Algorithm II (NSGA-II) is a generational multi-objective evolutionary algorithm that efficiently approximates the Pareto front of multi-objective optimization problems. NSGA-II maintains population diversity via fast non-dominated sorting and a crowding distance metric, selecting individuals for survival with explicit consideration for both convergence and spread in objective space. Since its introduction, NSGA-II has become a standard in multi-objective evolutionary optimization, with numerous algorithmic refinements targeting performance on high-dimensional fronts, distribution improvements, and faster convergence.

1. Core Algorithmic Workflow

NSGA-II operates on a population $P$ of size $N$ through deterministic and stochastic mechanisms in each generation. The main steps are:

1. Population Merging: Parents $P_t$ and offspring $Q_t$ are combined to form $R_t$ of size $2N$.
2. Non-dominated Sorting: $R_t$ is partitioned into Pareto fronts $F_1, F_2, \ldots$ by dominance ranking.
3. Survivor Selection: Successive entire fronts are added to the next parent population $P_{t+1}$ until the population size would be exceeded. If the last added front would overfill, individuals are selected via the crowding distance metric within that front.
4. Variation Operators: The next offspring population $Q_{t+1}$ is generated by binary tournament selection (favoring lower rank, then higher crowding distance), then crossover (e.g., simulated binary) and mutation.

Crowding distance acts as a density estimator within a front, guiding selection toward sparsely populated regions, thereby preserving solution diversity (Chu et al., 2018).

2. Crowding Distance: Rationale and Innovations

The original crowding distance assigns infinite distance to boundary solutions, with interior distances computed as:

$$d_{i} = \sum_{m=1}^{M} \frac{f_{i+1}^{m} - f_{i-1}^{m}}{f_{\max}^{m} - f_{\min}^{m}}$$

where $f_{i}^{m}$ is the $m$-th objective of the $i$-th individual (after sorting the front by $f^m$). This formulation essentially measures the perimeter of the smallest axis-aligned hypercube enclosing a solution and its neighbors (Chu et al., 2018).

However, this approach has notable shortcomings:

• Solutions within the same objective-space “hypercube” have identical distances, insensitive to proximity to the Pareto front.
• No prioritization for solutions closer to the true front, resulting in missed convergence opportunities.
• Only boundary points are explicitly protected from removal.

Improved Crowding Distance (Chu et al., 2018): The forward-difference approach uses

$$d_i^{\mathrm{imp}} = \sum_{m=1}^{M} \frac{f_{i+1}^{m} - f_{i}^{m}}{f_{\max}^{m} - f_{\min}^{m}}$$

(again after sorting each objective), making crowding distances sensitive to Pareto-front proximity. Boundary assignments remain infinite.

Pseudocode

def ComputeImprovedDistance(front_F, M): for i in F: d[i] = 0 for m in range(1, M+1): sort F by f^m ascending d[first] = d[last] = inf for k in 2..|F|-1: i = index of k in sort j = index of (k+1) in sort d[i] += (f_j^m - f_i^m) / (f_max - f_min) return d

Selection within the critical front is performed by descending improved crowding distance.

3. Runtime Behavior: Objective Dimensionality and Selection Pathologies

When $m \geq 3$ objectives, NSGA-II’s performance deteriorates dramatically due to its independent per-objective crowding distance mechanism (Zheng et al., 2022). Specifically, most solutions in $S \subseteq \{0,1\}^n$ (pairwise non-dominated) receive zero crowding distance and are selected almost uniformly at random. Only $O(mn)$ solutions, corresponding to boundary and near-boundary points, get positive crowding distance. This lack of discrimination severely impedes coverage of the Pareto front, resulting in exponential runtime lower bounds: $$T_{\mathrm{min}} = \exp\bigl(\Omega(M)\bigr) = \exp\bigl(\Omega((2n/m)^{m/2})\bigr)$$ for the $m$-objective OneMinMax benchmark, even when $N$ scales linearly with front size (Zheng et al., 2022).

This inefficiency is absent when $m = 2$ because the objectives induce global orderings that are mutually reversed, ensuring each solution in the front is “neighborly” in both coordinates.

4. Empirical and Theoretical Performance

NSGA-II with improved crowding distance has demonstrated consistently faster convergence to the Pareto front across diverse benchmarks (SCH, FON, POL, KUR, ZDT1, ZDT2, ZDT3, ZDT4, ZDT6). Quantitative results include:

• Generalized (Generational) Distance (GD): Reduced GD for improved crowding distance (e.g., ZDT2: $1.50 \times 10^{-4}$ vs. $3.56 \times 10^{-4}$).
• Coverage index $C$: Dominance by improved crowding distance; $$C(S_{\mathrm{imp}}, S_{\mathrm{orig}}) \approx 2 \times C(S_{\mathrm{orig}}, S_{\mathrm{imp}})$$ across all problems.
• Distribution Metrics (SP, $M_2^*$): Comparable or slightly favorable values for improved crowding distance, indicating diversity preservation (Chu et al., 2018).

These results validate that the improved formulation retains the computational cost ($O(Mn\log n)$), remains parameterless, and serves as a seamless replacement in existing NSGA-II implementations.

5. Algorithmic Integration, Complexity, and Practitioner Recommendations

The improved crowding distance requires only a single line modification in the codebase: replace the symmetric difference $(f_{i+1} - f_{i-1})$ with the forward difference $(f_{i+1} - f_i)$ during the accumulation phase. Complexity remains $O(Mn\log n)$ per generation (Chu et al., 2018).

Recommended workflow in population update:

• For every front $F_j$ that potentially overfills the survivor pool, compute improved crowding distances and retain the $N - |P_{t+1}|$ highest scoring individuals.
• All other algorithmic components (non-dominated sorting, binary tournament selection, crossover, and mutation) remain unmodified.

This drop-in adjustment accelerates convergence rates while preserving solution diversity. It is broadly applicable to standard test suites and real-world multi-objective problems.

6. Extensions, Limitations, and Open Problems

While improved crowding distance accelerates bi-objective NSGA-II and yields robust performance in moderate dimensions, sub-exponential runtime cannot be attained for $m \geq 3$ via standard per-objective crowding alone (Zheng et al., 2022). Core difficulties are rooted in the independence of objective-wise sorting, failing to capture joint density for high-dimensional fronts.

Remedies and future research directions include:

• Objective Correlation: Designing diversity metrics that correlate across objectives, e.g., reference-point systems (NSGA-III) or joint hypervolume contributions (SMS-EMOA).
• Efficient Diversity Maintenance: Constructing crowding or density estimators with guaranteed non-vanishing discrimination for large $m$, yet computational complexity $O(n \log n)$ or $O(n^2)$ per generation.
• Algorithm Selection: For many-objective environments, hybrid or reference-based selection schemes are standard practice.

A plausible implication is that practitioners should restrict the use of classic NSGA-II or single-objective crowding for $m \geq 3$ objectives, unless augmented by diversity mechanisms with proven many-objective performance

7. Summary Table: NSGA-II Crowding Distance Formulations

Formulation	Distance Computation	Parameter Sensitivity	Runtime per Gen.
Original (symmetric)	$$\sum_{m}(f_{i+1}^{m} - f_{i-1}^{m})/(f_{\max}^{m} - f_{\min}^{m})$$	None	$O(Mn\log n)$
Improved (forward)	$$\sum_{m}(f_{i+1}^{m} - f_i^m)/(f_{\max}^{m} - f_{\min}^{m})$$	None	$O(Mn\log n)$
Many-objective (not covered)	Separate joint-objective or reference-based approaches	Recommended (>2 obj)	Varies

The improved crowding distance formulation provides a computationally efficient, parameter-free, and robust density estimation strategy for standard multi-objective evolutionary optimization, relevant for up to moderate objective counts (Chu et al., 2018). Limitations at high dimensionality are intrinsic to the algorithmic architecture, motivating ongoing research in alternative diversity maintenance schemes (Zheng et al., 2022).