---
title: Soft QD Using Approximated Diversity (SQUAD)
url: https://www.emergentmind.com/topics/soft-qd-using-approximated-diversity-squad
type: topic
---

# Soft QD Using Approximated Diversity (SQUAD)

Soft QD Using Approximated Diversity (SQUAD) is a differentiable, population-based optimization algorithm that reframes Quality-Diversity (QD) as continuous attraction-repulsion in behavior space. SQUAD circumvents the need for explicit discretization of the behavior space, scaling efficiently to high dimensions and large populations while preserving or outperforming state-of-the-art QD benchmarks. The approach formalizes QD objectives as maximization of aggregate "illumination" from a set of solutions over an abstract behavior space, using kernel-based interactions and a tractable differentiable approximation amenable to gradient-based optimization [2512.00810].

## 1. Soft QD Objective: Definition and Intuition

Let $\Theta = \{\theta_1, ..., \theta_N\}$ denote a population of parameter vectors, where $f(\theta) \in \mathbb{R}_+$ is a differentiable quality (objective) function and $\mathrm{desc}(\theta) \in \mathcal{B} \subseteq \mathbb{R}^d$ a differentiable behavior descriptor. With $f_n = f(\theta_n)$ and $b_n = \mathrm{desc}(\theta_n)$, each solution is treated as an isotropic Gaussian "light source" in behavior space, its "brightness" $f_n$ decaying by bandwidth $\sigma > 0$.

The induced behavior-value field is
$$
v_{\Theta}(b) = \max_{1 \leq n \leq N} f_n \exp\left(-\frac{\|b - b_n\|^2}{2 \sigma^2}\right),
$$
and the Soft QD Score is defined as the total illumination:
$$
S(\Theta) = \int_{b \in \mathbb{R}^d} v_{\Theta}(b) \, db.
$$

Direct optimization of $S(\Theta)$ is intractable, so SQUAD proceeds via a tractable lower bound. By applying inclusion-exclusion, truncating at pairwise terms, and bounding $\min(x, y)$ by $\sqrt{xy}$, one obtains
$$
\widetilde{S}(\Theta) = \sum_{n=1}^N f_n - \sum_{1 \leq i < j \leq N} \sqrt{f_i f_j} \exp \left( -\frac{ \| b_i - b_j \|^2 }{ \gamma^2 } \right),
$$
where $\gamma^2 = 8\sigma^2$.

- The sum of $f_n$ rewards high-quality solutions.
- The pairwise repulsion term, exponentially decaying with behavioral distance and weighted by $\sqrt{f_i f_j}$, enforces diversity.

## 2. Derivation, Differentiability, and Limit Properties

The lower bound $\widetilde{S}(\Theta)$ derives from the inclusion-exclusion form:
$$
\int \max_n g_n(b) db = \sum_i \int g_i db - \sum_{i<j} \int \min(g_i, g_j) db + \ldots
$$
For $g_n(b) = f_n \exp(-\|b - b_n\|^2 / (2\sigma^2))$, the $\min$ term is tightly upper bounded by the geometric mean, which admits Gaussian integral solutions.

The resulting objective, dropping constant factors and using $\gamma^2=8\sigma^2$:
$$
\widetilde{S}(\Theta) = \sum_{n=1}^N f_n - \sum_{1 \leq i < j \leq N} \sqrt{f_i f_j} \exp \left( -\frac{ \| b_i - b_j \|^2 }{ \gamma^2 } \right)
$$

If $f$ and $\mathrm{desc}$ are differentiable, so is $\widetilde{S}(\Theta)$. The per-solution gradient is:
\[
\frac{\partial \widetilde{S}}{\partial \theta_n}
= \frac{\partial f_n}{\partial \theta_n}
- \frac{1}{2} \sum_{j \neq n} \left[
  \sqrt{ \frac{f_j}{f_n} } 
  e^{- \frac{ \| b_n - b_j \|^2 }{ \gamma^2 } } \frac{ \partial f_n }{\partial \theta_n }
+ 
\sqrt{ f_n f_j } e^{- \frac{ \| b_n - b_j \|^2 }{ \gamma^2 } }
  \frac{ -2 }{ \gamma^2 } (b_n - b_j)^T \frac{ \partial b_n }{ \partial \theta_n }
\right]
\]

Appendix proofs confirm key theoretical properties:

- $S(\Theta)$ is nondecreasing under addition of new solutions or increase in $f_n$.
- $S(\Theta)$ is submodular; marginal gains diminish as the population grows.
- In the limit $\sigma \to 0$ ($\gamma^2 \to 0$), $S(\Theta)/(2\pi\sigma^2)^{d/2} \to \sum_n f_n$, i.e., soft QD recovers the canonical QD-Score on a fine grid.

## 3. SQUAD Algorithm and Optimization

The SQUAD algorithm optimizes the lower bound $\widetilde{S}(\Theta)$ using mini-batch stochastic gradient ascent. The procedure is as follows:

- **Inputs:** population size $N$, batch size $M$, neighbor count $K$, iterations $T$, diversity bandwidth $\gamma^2$, optimizer $O$ (e.g., Adam with learning rate $\eta$).
- **Initialization:** sample $\Theta = \{\theta_1, ..., \theta_N\}$; compute qualities $F = (f_i)$ and behaviors $B = (b_i)$; initialize optimizer state $S$.
- **Loop:** For $t = 1, ..., T$:
    - Choose batch $I \subset \{1,\ldots,N\}$ of size $M$.
    - For each $i \in I$:
        - Identify $K$ nearest neighbors $N_i$ in behavior space.
    - Compute
      $$
      \widetilde{S}_I = \sum_{i \in I} f_i - \frac{1}{2} \sum_{i \in I, j \in N_i} \sqrt{f_i f_j} \exp(-\|b_i-b_j\|^2/\gamma^2 )
      $$
    - Update $\{\theta_i: i\in I\}$ using $\nabla_{\theta_I} \widetilde{S}_I$ via $O$; re-evaluate $(F_I,B_I)$.
- **Termination:** Return $\Theta$ at final iteration.

Key hyperparameters: $N, M, K, \gamma^2, \eta$, and the Gaussian kernel width $\sigma$ if computing $S(\Theta)$ directly.

## 4. Theoretical Properties and Scalability

SQUAD inherits the following properties:

- **Monotonicity:** $S(\Theta)$ is nondecreasing as new solutions are added or $f_n$ is increased.
- **Submodularity:** Diminishing returns property enables approximate optimality under cardinality constraints.
- **Limiting Behavior:** For $\gamma \to 0$, SQUAD converges to standard QD-Score maximization over a grid.
- **Curse-of-Dimensionality Avoidance:** Does not require discretization or archives, and uses continuous, kernel-based repulsion, resulting in memory requirements invariant to behavior-space dimension $d$.
- **Approximation Error:** Contributions from neglected higher-order overlaps in the inclusion-exclusion expansion are bounded and decay as behavioral coverage increases.

These properties collectively underpin SQUAD’s ability to scale to high-dimensional behavior spaces and large solution populations.

## 5. Empirical Evaluation

SQUAD's performance was benchmarked on diverse tasks and compared to existing methods including CMA-MEGA, CMA-MAEGA, Sep-CMA-MAE, GA-ME, DNS, and DNS-G.

### Tasks and Metrics

- **LP:** Linear-Projection Rastrigin, solution dim $n=1024$, behavior dims $d=4,8,16$.
- **IC:** Image Composition (1024 circles, 5-d behavior).
- **LSI:** Latent-Space Illumination via StyleGAN2+CLIP, $d=2$ and $d=7$.

Metrics included QD-Score (sum of best-in-cell over CVT), coverage (number of occupied CVT cells), Vendi Score (VS, effective number of clusters), QVS (mean-quality × VS), mean and max objective.

### Outcome Summary

- **LP:** At $d=4$, CMA-MAEGA/CMA-MEGA slightly outperform SQUAD, but SQUAD surpasses all baselines in both QVS and QD-Score at $d=8$ and $d=16$ (for example, at $d=16$ QVS: SQUAD $\approx 6.6$, CMA-MAEGA $\approx 4.6$, CMA-MEGA $\approx 3.8$). Gradient-based methods, including SQUAD, outperform mutation-only approaches at higher dimensionalities.
- **IC:** SQUAD achieves highest mean objective ($83.4\pm0.02$), highest max objective ($93.6\pm0.10$), and best VS ($5.49\pm0.00$). Coverage is slightly below CMA-MAEGA (5.68 vs 5.85), but VS more accurately reflects true diversity.
- **Quality-Diversity Trade-Off:** Varying $\gamma^2$ in $[10^{-3}, 50]$ shows a tunable trade-off—higher diversity (VS) for larger $\gamma^2$ at the expense of mean objective.
- **LSI:** In base ($d=2$), SQUAD QD-Score $\approx 13.4 \times 10^3$, QVS $\approx177$, surpassing CMA-MEGA ($8.7\times10^3/140$), CMA-MAEGA ($6.8\times10^3/122$). In hard ($d=7$), SQUAD ($2.55\times10^3/151$) vs best baseline ($0.39\times10^3/99$). Other methods often experience failure (negative mean objectives).

## 6. Implementation and Practical Considerations

### Default Hyperparameters

- **IC/LP:** $N=1024$, $M=64$, $K=16$, $\eta=0.05$, $\gamma^2$ tuned to domain (IC: $\gamma^2=1$; LP: easy/medium/hard $\gamma^2=0.1/0.5/1.0$).
- **LSI:** $N=256$, $M=8$, $K=16$, $\eta=0.1$, $\gamma^2=0.01 \ldots 0.1$.

### Algorithmic Details

- If $\mathcal{B} = [0,1]^d$, behaviors should be mapped via $\logit(b)$ to $\mathbb{R}^d$; ablation indicates this is critical.
- Each batch update computes $O(MK)$ pairwise and $O(M)$ quality terms, with overall iteration cost $O(NK)$.
- Mini-batching and limited nearest neighbors optimize memory efficiency.

### Implementation Tips

- Automatic differentiation frameworks (JAX, PyTorch, etc.) are recommended for both objective and descriptor.
- Precompute and cache k-NN structures in behavior space.
- Annealing $\gamma^2$ or adapting $K$ to local density can improve performance.
- Monitor VS, coverage, and mean objective during optimization; early stopping is often effective (e.g., $\sim$200 iterations for IC/LSI).
- Ensure $f(\theta)$ is nonnegative for meaningful QVS evaluation.

### Computational Cost

- Simple tasks (LP) complete in under 1 minute.
- IC requires $\sim$190 minutes for 1000 iterations (RTX 4090), but high performance is typically attained in fewer than 200 iterations.
- LSI (base/hard): $\sim$730/1300 minutes, with convergence in substantially less than the full budget.

## 7. Significance in Quality-Diversity Optimization

SQUAD provides an alternative to archive-based QD: it offers a smooth, differentiable objective and adaptively balances quality and diversity through a tunable, analytically tractable surrogate. This formulation permits large-scale, high-dimensional QD optimization previously infeasible with grid-based methods. Empirical evidence demonstrates competitiveness and often superiority versus established QD algorithms on standard benchmarks, with additional robustness and scalability. These features make SQUAD a theoretically well-founded and practically effective approach for large-scale, high-diversity optimization tasks [2512.00810].

Source: https://www.emergentmind.com/topics/soft-qd-using-approximated-diversity-squad