Novelty Search with Local Competition

• NS-LC is an algorithmic framework for quality-diversity optimization that integrates behavioral novelty with local performance competition.
• It employs spatial or archive-based structures to compare individuals within a behavioral descriptor space, encouraging both exploration and targeted improvement.
• Recent advances like Dominated Novelty Search reformulate NS-LC by eliminating rigid discretization and dynamically tuning local competition parameters.

Novelty Search with Local Competition (NS-LC) is an algorithmic framework for Quality-Diversity (QD) in evolutionary computation that combines the exploration of behavioral novelty with localized performance competition. It operates by enforcing local competition within a behavioral descriptor space—typically via explicit spatial structures such as grids or archives—rewarding individuals that are both novel and outperform their close behavioral neighbors. NS-LC has been foundational to QD methods such as MAP-Elites and its variants but faces limitations due to its dependence on predefined spatial discretization, bounded descriptor ranges, and hard-to-tune locality parameters. Contemporary research has introduced more general formulations, notably Dominated Novelty Search, by reframing local competition as a dynamic fitness transformation, thus circumventing the rigid discretization and thresholding of classic NS-LC implementations (Bahlous-Boldi et al., 1 Feb 2025).

1. Conceptual Foundation

NS-LC augments a standard evolutionary algorithm with two distinctive mechanisms: behavioral novelty and local competition. Novelty is quantified by the sparsity of an individual’s behavioral descriptor within the current population or an archive, while local competition compares an individual’s fitness against others deemed nearby in descriptor space. Traditionally, this dual criterion is operationalized by partitioning the descriptor space into explicit cells (e.g., MAP-Elites) or maintaining an unstructured archive with a threshold-based nearest-neighbor scheme. Individuals are selected based on their ability to outperform local neighbors, incentivizing both exploration of diverse behaviors and exploitation of high performance in each niche.

The mechanism of explicitly partitioning or archiving imposes artificial constraints and parameters, such as:

• Descriptor space boundaries (for grid-based QD)
• Archive cell count or inter-point distance thresholds
• Static or heuristically adapted bounds and thresholds

These constraints can hinder scalability and robustness in scenarios involving discontinuous, high-dimensional, unbounded, or learned descriptor spaces (Bahlous-Boldi et al., 1 Feb 2025).

2. Mathematical Structure of Competition Fitness

The competition step in NS-LC and its modern variant, Dominated Novelty Search (DNS), is formalized as follows. For a population of size $N$, with raw fitness vector $f = (f_1, \dots, f_N)$ and corresponding behavioral descriptor vectors $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$:

• For each solution $i$, identify the set of strictly fitter individuals:

$$\mathcal{D}_i = \{\, j \in \{1, \dots, N\} \mid f_j > f_i \}$$

• Compute descriptor-space distances to these fitter solutions:

$$d_{ij} = \| d_i - d_j \| \quad \forall j \in \mathcal{D}_i$$

• Let $\mathcal{K}_i \subseteq \mathcal{D}_i$ be the indices of the $k$ nearest fitter solutions (if $|\mathcal{D}_i| < k$, use all of $\mathcal{D}_i$).
• Define the dominated novelty competition fitness:

$f = (f_1, \dots, f_N)$0

Individuals on the Pareto front of fitness (those with no strictly fitter neighbors) are preserved; all others are ranked by their average nearest distance to fitter solutions, promoting behavioral diversity relative to top performers (Bahlous-Boldi et al., 1 Feb 2025).

3. Algorithmic Workflow and Distinction from Explicit-Archive QD

Within each evolutionary generation, NS-LC and the DNS reformulation follow these steps:

1. Sample offspring by reproduction.
2. Aggregate with the current population.
3. Evaluate raw fitness and behavioral descriptors for all individuals.
4. Compute the competition fitness $f = (f_1, \dots, f_N)$1 using the above formulation.
5. Select the top $f = (f_1, \dots, f_N)$2 individuals by competition fitness for the next generation.

DNS introduces several advances over traditional NS-LC:

• Eliminates explicit archive or grid maintenance.
• Replaces a static distance threshold or descriptor bounds with a single dynamic parameter ($f = (f_1, \dots, f_N)$3), the nearest-neighbor count.
• Allows local competition to emerge from the evolving population distribution rather than rigid discretization.

This generalization enables straightforward application to arbitrary, discontinuous, unbounded, or high-dimensional descriptor spaces without redesigning spatial containment mechanisms or thresholds (Bahlous-Boldi et al., 1 Feb 2025).

4. Hyperparameters, Practical Tuning, and Domains

The key hyperparameters in DNS and NS-LC are:

• Population size $f = (f_1, \dots, f_N)$4
• Offspring batch size $f = (f_1, \dots, f_N)$5
• Locality parameter $f = (f_1, \dots, f_N)$6 (nearest neighbors for competition)

DNS obviates the need for descriptor bounds, grid resolutions, or static distance thresholds, reducing the algorithm to a small set of interpretable parameters. Empirical results indicate DNS is robust to the choice of $f = (f_1, \dots, f_N)$7 within reasonable ranges (e.g., $f = (f_1, \dots, f_N)$8), in contrast to the sensitive tuning required by hard-threshold-based NS-LC (Bahlous-Boldi et al., 1 Feb 2025).

Experimental domains have included:

• Brax/MuJoCo continuous-control (Walker, Ant, Ant Blocks)
• Maze navigation (with both explicit and unsupervised, autoencoder-learned behavioral descriptors)

5. Empirical Evaluation and Comparative Analysis

In standardized QD benchmarks, DNS demonstrates significant empirical advantages relative to NS-LC, MAP-Elites, and other QD baselines:

Scenario	DNS Advantage	Statistical Result
Well-defined descriptors	Outperforms Threshold/Cluster-Elites	$f = (f_1, \dots, f_N)$9; MAP-Elites on some
High-dimensional	Exceeds all QD baselines	$d = (d_1, \dots, d_N) \subset \mathbb{R}^D$0, scales well
Unsupervised descriptors	Robust, exceeds alternatives	MAP-Elites inapplicable

• On conventional descriptor spaces (Walker, Ant, Ant Blocks), DNS outperforms Threshold-Elites and Cluster-Elites (p < $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$1), and surpasses MAP-Elites where grid-based algorithms are misaligned with space topology.
• In high-dimensional scenarios, MAP-Elites' cell-based representation suffers from exponential inefficiency, whereas DNS scales gracefully.
• For unsupervised, learned descriptor spaces (e.g., autoencoder latents), DNS manages unbounded and discontinuous spaces where grid-based and threshold-based methods are inapplicable or require additional mechanisms.
• DNS performance is empirically invariant to moderate variation in $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$2, eliminating the need for extensive hyperparameter search (Bahlous-Boldi et al., 1 Feb 2025).

6. Implementation Considerations and Computational Cost

DNS’s main computational cost is the pairwise distance matrix required for each generation ($d = (d_1, \dots, d_N) \subset \mathbb{R}^D$3), which may be alleviated for large populations by approximate nearest-neighbor structures such as KD-trees or locality-sensitive hashing. For typical robotics and control tasks, standard practice is $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$4, $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$5, and $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$6. A plausible implication is that for very high-dimensional descriptors and extremely large populations, the algorithmic cost and possible noise in fitness evaluation may become limiting factors. These can be mitigated by distance approximation or smoothing of the fitness front (Bahlous-Boldi et al., 1 Feb 2025).

DNS is designed as a drop-in replacement for previously explicit-archive QD algorithms, requiring only a change to the competition step. The removal of archive/grid infrastructure and parameter tuning simplifies deployment in complex or learned behavioral spaces.

7. Limitations and Future Research Directions

Both NS-LC and DNS, although robust, are subject to limitations:

• Pairwise distance computations may become a bottleneck in large-scale or high-dimensional settings.
• Extreme noise in raw fitness can result in spurious retention of solutions with apparent infinite competition fitness; this is mitigable by smoothing or introducing a fitness margin.
• Static $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$7 may be insufficiently adaptive for tasks with highly dynamic evolutionary landscapes.

Future directions include meta-learning the competition function, dynamically adjusting $d = (d_1, \dots, d_N) \subset \mathbb{R}^D$8, and extending the approach to multi-objective or co-evolutionary QD domains. The reformulation of QD as a genetic algorithm with a dynamic local-competition step provides infrastructure for further theoretical and empirical innovation in scalable Quality-Diversity optimization (Bahlous-Boldi et al., 1 Feb 2025).