---
title: Geometric Red-Teaming
url: https://www.emergentmind.com/topics/geometric-red-teaming-grt
type: topic
---

# Geometric Red-Teaming

Across the cited literature, Geometric Red-Teaming designates red-teaming formulations in which adversarial search is conducted over a structured space rather than over isolated test cases. In the LLM setting, the relevant geometry is over dialogue trajectories and strategy profiles, not isolated prompts; the attack surface is modeled as a dynamic game over multi-round interactions and policy populations [2310.00322]. In robotic manipulation, Geometric Red-Teaming (GRT) is a framework that probes robustness through object-centric geometric perturbations, automatically generating CrashShapes—structurally valid, user-constrained mesh deformations that trigger catastrophic failures in pre-trained manipulation policies [2509.12379]. This suggests a common emphasis on active robustness evaluation through structured exploration of failure-inducing directions.

## 1. Semantic strategy spaces in language-model red teaming

The LLM formulation explicitly replaces one-shot adversarial prompt hunting with a dynamic game over dialogue trajectories and policy populations. The paper defines a **Red Teaming Game (RTG)** as a bi-level game that combines a token-level **Markov Decision Process for Token Generation (MDPTG)**, $\mathcal{T}$, and a sentence-/dialogue-level **Extensive-form Team Game in Dialogue (ETGD)**, $\mathcal{D}$. It states that $\mathcal{G}$ is a tuple $(\mathcal{T}, \mathcal{D})$ with a token-level $\mathcal{T}$ and a sentence-level $\mathcal{D}$. This differs from standard single-round red-teaming in two stated ways: token-level optimization, in which each sentence is generated autoregressively and optimized as an MDP over tokens, and multi-round interaction, in which red and blue models alternate over several dialogue turns [2310.00322].

Within RTG, the dialogue is finite-horizon. The red team’s objective is to induce toxic output across multiple rounds, while the blue team tries to remain helpful and harmless. The sentence-level utility is written as
$$
U_\mathcal{L}(\pi_\mathcal{R}, \pi_\mathcal{B}) = \sum_{j=1}^{p} P^{j}_\mathcal{L}.
$$
The paper further assumes a zero-sum game. In this framing, later attacks can exploit weaknesses revealed by earlier turns, so the attack surface is dynamic rather than static. The paper’s own summary is that the “geometry” is over dialogue trajectories and strategy profiles, not isolated prompts.

The same trajectory-centered intuition appears in later automated LLM red-teaming work that formalizes the attacker’s search space as a path through conversation state space rather than as a single prompt. That alignment is explicit in the literature’s contrast between dynamic, context-aware attack trajectories and brittle prompt templates or single-turn attacks.

## 2. Population games, diversity, and equilibrium

The algorithmic engine for solving RTG is the **Gamified Red Teaming Solver (GRTS)**. It is based on **Double Oracle / PSRO** style iterative meta-game solving: maintain populations of policies for red and blue, compute a restricted Nash equilibrium on the current subgame, train a best response against the current equilibrium, add the new policy to the population, and repeat. The paper states that GRTS “works in expanding policy set” and uses sentence-level policies rather than token-level policies [2310.00322].

A central design choice is the explicit use of a **diversity measure of semantic space** when training the oracle policy. The paper states: “In the process of computing the best response in line 7, we introduced a measure of diversity in the semantic space,” and Algorithm 1 includes the line `Train oracle \(\pi^{'}_{i}\) over \(\rho\sim(\pi^{'}_{i},\pi_{-i})\) with diversity measure of semantic space`. This is introduced to mitigate **mode collapse**, the reduction in generation diversity caused by repeatedly exploiting one successful attack template. The intended effect is to make the red population explore different semantic regions, topics, and forms of attacks rather than converging to a narrow mode.

The theoretical claim is that GRTS converges to an **$\epsilon$-approximate Nash equilibrium** of RTG. The paper explains that $\pi^*$ is an $\epsilon$-approximate Nash equilibrium if no unilateral deviation can improve utility by more than additive $\epsilon$. Distance to equilibrium is measured using **exploitability**:
$$
\operatorname{Expl}(\sigma)=\sum_{i=1}^{m}\left(\max_{\pi_{i}^{\prime}} U_{i}\left(\pi_{i}^{\prime}, \pi_{-i}\right)-U_{i}\left(\pi_{i}, \pi_{-i}\right)\right).
$$
At the token level, the first-stage objective is
$$
J_1(\xi_\mathcal{L})=\mathbb{E}_{s \sim \rho}\left[V_{\xi_\mathcal{L}}(s)\right],
\qquad
\xi_\mathcal{L}^{*}=\arg \max _{\xi_\mathcal{L} \in \Xi_\mathcal{L}} J_1(\xi_\mathcal{L}).
$$
The paper reports exploitability falling from about 6.2 to around 0.8 after 15 iterations, which it interprets as approaching an approximate equilibrium.

The empirical consequences of this population-and-diversity view are presented as substantive. The reported visualizations cover **29 distinct attack topics**, and t-SNE clustering shows attacks distributed across broad semantic regions. GRTS uncovers both common and rare vulnerabilities, including profanity, violence, harmful activity, crime, substance abuse, animal abuse, and soliciting PII. It also highlights **implicit and concealed attacks**, including **Role Play Instruction**, **Goal Hijacking**, and **Reverse Exposure**. The paper connects this to a geometric interpretation via the **spinning top hypothesis**: the red team’s strategy population behaves like a spinning top that must keep moving to stay stable, and diversity prevents collapse into a single orientation. It therefore motivates a **diverse LLM population as a proxy for heterogeneous human expert red-teamers**, since manual annotation is described as expensive, slow, and limited in diversity.

## 3. Related automated LLM red-teaming paradigms

A closely related line of work reframes red teaming as a **sequential decision-making problem over conversational trajectories**. In “Automatic LLM Red Teaming,” the attacker is modeled as operating in an MDP
$$
\mathcal{M} = (\mathcal{S}, \mathcal{A}, T, R, \gamma),
$$
where $\mathcal{S}$ is the space of conversation histories and $\mathcal{A}$ is the space of possible utterances. The transition is defined by the target model’s autoregressive generation, with
$$
s_{t+1} = s_t || a_t || v_t.
$$
The method uses a **hierarchical RL** framework with a high-level policy $\pi_1$ that selects guides or attack styles and a low-level policy $\pi_2$ that generates tokens conditioned on the state and guide. Its high-level objective is described as maximizing the harmfulness of the target’s response while minimizing the harmfulness or detectability of the attacker’s own utterance, plus discounted future value. Token-level credit assignment is handled through masking-based marginal contribution, extended with pairwise masking to capture token interactions, and PPO is used at both levels [2508.04451].

This trajectory-based formulation is presented as a **dynamic, trajectory-based process** rather than **a one-step test**. It is also reported to be especially strong in **context-aware** settings, and the paper states that removing high-level guides causes performance drops of **60%–80%**. This supports the broader GRT intuition that effective red teaming may require forward-looking policies over paths through dialogue space rather than isolated adversarial strings.

By contrast, **Gradient-Based Red Teaming (GBRT)** is described as a fully differentiable, gradient-driven method that directly optimizes prompts through a frozen LM and a frozen safety classifier. A prompt is represented as a learnable distribution over tokens, Gumbel softmax is used to make both prompt sampling and autoregressive decoding differentiable, and the default objective minimizes the classifier’s predicted probability that the output is safe:
$$
L = p_{\text{safe}}(\widetilde{x}, \widetilde{y}).
$$
The paper also studies an output-only variant, a realism-loss regularizer, and a pretrained prompt-generator fine-tuning variant. GBRT is more effective than an RL-based red teaming baseline at finding prompts that trigger unsafe responses, including on a model fine-tuned to be safer, and the authors use **224 high-impact prompts** from GBRT and GBRT-ResponseOnly for subsequent DPO fine-tuning [2401.16656].

The distinction is methodological. Trajectory-based systems search over dialogue histories and adaptive policies; GBRT searches in a differentiable prompt space. This suggests that “geometric” in LLM red teaming is not restricted to one optimization formalism: in one formulation it names semantic strategy populations and dialogue trajectories, whereas in another nearby paradigm the search space is a soft, differentiable prompt distribution.

## 4. Object-centric geometric perturbations in robotic manipulation

In robotic manipulation, Geometric Red-Teaming is formalized as a black-box search over **3D object geometry**. Let $\pi: \mathcal{S} \rightarrow \mathcal{A}$ be a pretrained manipulation policy, $M = (V, F)$ a watertight triangle mesh, $D_\theta$ a deformation operator parameterized by $\theta \in \Theta$, and $\mathcal{J}(\pi, D_\theta(M))$ a task-specific success metric computed via simulation rollouts. The objective is to find a deformation that minimizes policy performance:
$$
\theta^* = \arg\min_{\theta \in \Theta,\; D_\theta(M) \in \mathcal{G}(M)} \mathcal{J}(\pi, D_\theta(M)),
$$
where $\mathcal{G}(M)$ is the set of physically plausible deformations of $M$. The discovered adversarial shapes are called **CrashShapes**, defined as structurally valid, user-constrained, physically plausible, failure-inducing deformed meshes [2509.12379].

The deformation model is **Jacobian field-based deformation** derived from APAP-style mesh editing. The method distinguishes **handle points** $H \subseteq V_0$, which are actively displaced, from **anchor points** $A \subseteq V_0$, which are kept fixed to avoid trivial global motion. For source mesh
$$
M_0 = (V_0, F_0),
$$
it defines a per-face local affine Jacobian field
$$
J_f \in \mathbb{R}^{3\times 3}, \quad \forall f \in F_0.
$$
The deformed mesh $V^*$ is obtained by solving
$$
V^{*} = \arg\min_V \|LV - \nabla^T A J\|^{2} + \lambda \|K_aV - T_a\|^2,
$$
where $L$ is the cotangent Laplacian, $\nabla$ the stacked per-face gradient operator, $A$ the mass matrix, $J$ the Jacobian field, $K_a$ the indicator matrix selecting anchor vertices, $T_a$ the anchor target positions, and $\lambda$ the anchor constraint strength. Soft handle constraints are then enforced by minimizing
$$
\mathcal{L}_h = \|K_h V^* - T_h\|^2.
$$

The implementation uses **only the first stage of APAP** and **omits the 2D diffusion prior**, because the latter adds substantial compute, provides minimal benefit for task-specific objects, and can even hurt specialized shapes like USB plugs. The reported resource difference is approximately **22 seconds** and **1 GB memory** for the Jacobian-only version, versus approximately **10 minutes** and **10 GB memory** for full APAP.

Because the simulator and success metric are non-differentiable, optimization is **gradient-free** and simulator-in-the-loop. The method uses a population-based black-box optimizer inspired by TOPDM with **selective perturbation**: only a random subset of handle displacements is mutated at each step. The stated loop initializes a population from Gaussian noise, perturbs $\lfloor \gamma M \rfloor$ handle points per iteration, evaluates candidates in simulation, selects the top $\lceil \rho N \rceil$ elites with lowest $\mathcal{J}_i$, replicates elites, and returns the best candidate found. The appendix reports a population size of **10 candidates**, **10** iterations, a selective perturbation fraction of **half** the parameters, and Gaussian initialization standard deviation **0.001** in normalized mesh units. The method also introduces a **Smoothness Score (SS)** budget,
$$
\mathrm{SS}(D) = \frac{1}{M}\sum_{i=1}^{M}\|d_i\|_2,
$$
with projection
$$
P_\tau(D) = sD,\qquad s=\min\left\{1,\frac{\tau}{\mathrm{SS}(D)}\right\},
$$
to constrain deformations to be small and plausible.

## 5. CrashShapes, blue-teaming, and hardware validation

The robotics paper evaluates GRT on three tasks. For **rigid-object grasping**, it attacks **Contact-GraspNet** on **22 YCB objects** with **64 grasp trials per object** in randomized poses; success requires the object to remain held after a **lift-and-shake** test. For **high-precision insertion**, it studies both a **state-based insertion policy** from **IndustReal** and a **point-cloud-initialized insertion policy** using **PointNet++**. For **articulated manipulation**, it attacks a state-based drawer-opening policy using **PartNet-Mobility** cabinet-style assets. The observed failures include bad grasp poses, grasp slippage, brittle contact patches, failure after shake, plug/socket misalignment, plug rotation in the gripper at socket contact, poor engagement caused by subtle connector deformation, and near-complete failure of drawer motion under small shape changes [2509.12379].

The main results are reported with **Final Drop**, **Iter @ 50%**, **AUC**, and **$\Delta$Complexity**. For grasping, Final Drop is **76.3%** for **VLM-guided**, **63.4%** for **Manual**, and **58.3%** for **VLM-guided + SS**. For articulated manipulation, Final Drop is **61.9%** for **VLM-guided**, **98.9%** for **Manual**, and **44.7%** for **Manual + SS**. For state-based insertion, Final Drop is **67.4%** for **VLM-guided**, **73.95%** for **Manual**, and **60.9%** for **Manual + SS**. For point-cloud-based insertion, Final Drop is **77.7%** for **VLM-guided**, **71.7%** for **Manual**, and **43.4%** for **Manual + SS**. The grasping ablation reports **63.3%** for Heuristic + Gaussian perturbation, **68.4%** for Heuristic + Optimization, **71.4%** for All Handles + Optimization, **65.1%** for VLM-guided + Gaussian, and **76.3%** for **VLM-guided + Optimization (Ours)**. The paper’s interpretation is that GRT surfaces catastrophic failures with small, plausible deformations, not just obviously distorted shapes.

The same paper uses **blue-teaming** to denote fine-tuning the policy on discovered CrashShapes. The fine-tuning procedure uses **PPO**, **Early stopping**, **No task augmentation**, and training on CrashShapes plus nominal object geometry. For the state-based insertion policy, performance on CrashShape CS-1 rises from **25.0%** to **87.8%**, and on CS-2 from **45.0%** to **93.8%**; nominal plug performance remains **87.5%** after fine-tuning for CS-1 and **96.0%** after fine-tuning for CS-2, against a nominal pretraining baseline of **96%**. For the point-cloud-initialized policy, CrashShape performance rises from **31.3%** to **81.3%**, while nominal performance after fine-tuning is **87.3%** against a nominal pretraining baseline of **86%**. The paper emphasizes **up to 60 percentage points improvement** on CrashShapes while preserving performance on the original object.

Real-robot validation is reported for insertion with an **xArm 6** and grasping with a **Franka Panda**. CrashShapes are 3D printed in **PLA** and processed with **CoACD convex decomposition**. In insertion, over **40 trials per shape**, nominal success is **90.0%**, CrashShape CS-1 yields **22.5%**, and CrashShape CS-2 yields **55.0%**; blue-teaming recovers performance to **90.0%** on CS-1 and **82.5%** on CS-2, while nominal performance remains **85.0%** or **95.0%** depending on the blue-teamed variant. In grasping, over **20 trials per object**, nominal success is **80.0%** for the mustard bottle and **90.0%** for the screwdriver, while CrashShapes reduce success to **30.0%** and **35.0%**, respectively. The paper states that these real-world outcomes **closely match simulation trends**, supporting the claim that CrashShapes are physically meaningful failure cases rather than simulator artifacts.

## 6. Scope and boundaries of the term

The surrounding literature makes clear that Geometric Red-Teaming is not a single fixed mathematical doctrine. In the LLM setting, the core geometric language refers to **semantic strategy space**, **dialogue trajectories**, and **policy populations**. In robotics, it refers to **object-centric geometric perturbations** of meshes and physically plausible deformation sets. This indicates that the commonality is structural search over a task-relevant space, while the underlying space itself is domain-specific.

That boundary is explicit in later infrastructure work. **OpenRT** is described as “primarily an engineering and benchmarking framework for automated red teaming of multimodal large language models, not a geometric red-teaming theory paper.” Its closest geometry-like components are standard adversarial constraints and embedding-based measures rather than a unified geometric attack surface. The framework defines multimodal adversarial inputs as perturbations of text and image modalities, constrains image attacks by an $\ell_\infty$ budget and text attacks by a perplexity threshold, and evaluates some attacks using embedding-based diversity. Its unified adversarial objective is
$$
(v', \widetilde{t}_{1:n}) = \arg\min_{\substack{\| v' - v \|_{\infty} \leq \epsilon \\ \text{PPL}(\widetilde{t}_{1:n}) \leq \beta}} -\log p(t_{n+1:n+m} \mid v', \widetilde{t}_{1:n}; \theta),
$$
and its evaluator uses ASR, efficiency, perplexity, and diversity. The paper integrates **37 diverse attack methodologies**, evaluates **20 advanced models**, reports an average ASR of **49.14%**, and states that reasoning models do **not inherently possess superior robustness against complex, multi-turn jailbreaks** [2601.01592].

A recurrent misconception is therefore that “geometric” must denote a latent-manifold or continuous-optimization theory. The cited literature does not support that restriction. In one case, the geometry is explicitly over dialogue trajectories and strategy profiles; in another, over mesh deformations and simulator-validated shape variation; and in adjacent infrastructure work, the geometry-like elements are perturbation budgets and embedding distances rather than a manifold model. The term, as used here, names structured adversarial search spaces whose organization matters for discovering robustness failures.

Source: https://www.emergentmind.com/topics/geometric-red-teaming-grt