ME-MAP-Elites: Multi-Emitter Search

• ME-MAP-Elites is a quality-diversity algorithm that uses multiple heterogeneous emitters to explore and optimize diverse high-performing solutions.
• It employs various emitter types—optimizing, random-direction, and improvement emitters—to generate candidate solutions via tailored local search strategies.
• Dynamic emitter selection using a bandit-based scheduling approach balances exploration and reward to enhance archive coverage and performance metrics.

Multi-Emitter MAP-Elites (ME-MAP-Elites) denotes a class of quality-diversity (QD) algorithms that enhance the classical MAP-Elites framework by deploying multiple, potentially heterogeneous, emitters to accelerate and diversify exploration and exploitation in high-dimensional search spaces. This approach has yielded state-of-the-art empirical results in generating archives of high-performing and diverse solutions across both standard optimization and complex robotics tasks (Cully, 2020, Flageat et al., 2023).

1. Quality-Diversity Framework and Archive Structure

Quality-diversity algorithms simultaneously optimize for both solution performance (fitness) and diversity across a discretized behavioral descriptor space. Formally, let $X\subseteq\mathbb{R}^d$ denote the genotype (parameter) space, $f:X\to\mathbb{R}$ the scalar fitness function, and $b:X\rightarrow\mathcal{B}\subseteq\mathbb{R}^k$ the behavioral descriptor mapping. The descriptor space $\mathcal{B}$ is discretized into $M$ cells, constructing an archive

$\mathcal{A} = \{A_i\;|\;i = 1,\ldots, M\},$

where $A_i$ stores the best-so-far individual $x$ for which $b(x)$ falls into cell $i$, or $f:X\to\mathbb{R}$0 if empty. The fundamental QD objective is, for each $f:X\to\mathbb{R}$1,

$f:X\to\mathbb{R}$2

Evaluation metrics include coverage $f:X\to\mathbb{R}$3, QD-Score $f:X\to\mathbb{R}$4, and Max-Fitness $f:X\to\mathbb{R}$5.

2. Heterogeneous Emitters: Principles and Mechanisms

Standard MAP-Elites employs a single, uniform variation operator, typically Gaussian mutation. By contrast, ME-MAP-Elites instantiates a pool of emitters, each guided by distinct search heuristics or local optimization strategies. This emitter diversity is central to improved QD performance.

Typical emitter types include (Cully, 2020):

• Optimizing emitters: CMA-ES instances optimizing $f:X\to\mathbb{R}$6.
• Random-direction emitters: CMA-ES variants rewarding directional progress in descriptor space, i.e., $f:X\to\mathbb{R}$7 for random $f:X\to\mathbb{R}$8.
• Improvement emitters: CMA-ES optimizing archive addition or quality improvement.
• Random (MAP-Elites-style) emitters: Direct random variations as in standard MAP-Elites.

Emitters propose candidate solutions via sampling and local adaptation, with their offspring evaluated for fitness and behavioral descriptor. Archive updates occur if a new individual either fills an empty cell or improves upon the incumbent.

3. Dynamic Emitter Selection: Bandit-Based Scheduling

Emitter allocation is handled dynamically via sliding-window, multiple-play UCB1 bandit algorithms. Over a window of $f:X\to\mathbb{R}$9 generations, the empirical mean reward for emitter $b:X\rightarrow\mathcal{B}\subseteq\mathbb{R}^k$0, $b:X\rightarrow\mathcal{B}\subseteq\mathbb{R}^k$1, combines with an exploration term, yielding index