Human-Robot Red Teaming

• Human-robot red teaming is a set of adversarial, exploratory, and evaluative practices designed to expose potential hazards in human-robot systems.
• It encompasses methods such as challenging world models, generating failure-inducing instructions, and assessing cyber-physical vulnerabilities to enhance robot safety.
• This framework drives model revision and risk mitigation by systematically testing robots under diverse environmental and operational perturbations.

Human-robot red teaming designates a set of adversarial, exploratory, and evaluative practices for exposing failures in robots and human-robot systems before those failures surface in deployment. In recent work, the term spans at least five technically distinct but related activities: collaborative challenge of robot world models for safety-aware reasoning; automated generation of difficult, feasible instructions for language-conditioned robot policies; predictive estimation of policy brittleness under off-nominal environmental factors without exhaustive hardware trials; simulator-in-the-loop search for failure-inducing object geometries; and cyber-physical security assessment of humanoids as embedded endpoints, surveillance nodes, and attack vectors (Sheetz et al., 2 Aug 2025, Karnik et al., 2024, Majumdar et al., 10 Feb 2025, Goel et al., 15 Sep 2025, Mayoral-Vilches, 17 Sep 2025). Across these strands, the unifying premise is that benchmark success or nominal task capability is insufficient: a robot must be challenged on the assumptions, perturbations, instructions, geometries, and trust relationships that later determine safety, robustness, and security.

1. Conceptual foundations

In the most explicit formulation, human-robot red teaming (HRRT) is a collaborative process in which humans and robots work together to challenge assumptions about an environment and explore the space of hazards that may arise. The red team is not primarily an adversary in the military or cyber sense; rather, it functions as a structured challenger to the human-robot team’s current model of the world. The robot’s role is to algorithmically surface possibilities, assumptions, and prompting questions that expose weaknesses in the current model, while the human contributes domain judgment, especially where ethical, moral, and safety-priority decisions are required, and helps direct model revision. This framing is distinct from ordinary task planning, which assumes a task model and searches for a plan within it, and distinct from standard human-robot interaction focused on usability or execution assistance, because the interaction is specifically organized around adversarial reflection on safety (Sheetz et al., 2 Aug 2025).

The broader literature extends this core idea in several directions. In one line of work, red teaming targets language grounding, asking what plausible human instructions, in a scene, cause a robot policy to fail or behave unsafely. In another, it targets environmental robustness, asking which lighting, distractors, viewpoints, or other off-nominal conditions most sharply degrade performance. A third line targets physical morphology, searching for structurally valid object deformations that induce catastrophic failures in manipulation policies. A fourth line treats the robot itself as a cyber-physical target and attack vector, so that red teaming includes reverse engineering, trust-boundary analysis, telemetry inspection, and cloud pivoting. A plausible implication is that “human-robot red teaming” is best understood not as a single algorithm, but as a family of methods unified by proactive failure discovery under deployment-relevant constraints (Karnik et al., 2024, Majumdar et al., 10 Feb 2025, Goel et al., 15 Sep 2025, Mayoral-Vilches et al., 17 Sep 2025).

2. Targets of evaluation and formal problem statements

The literature red teams different technical objects, but all formulations search beyond nominal evaluation. The following summary captures the principal targets already instantiated in published work.

Paradigm	Primary object under test	Core criterion
HRRT	symbolic model $M=(S,A)$	safer behavior through hazard discovery, model revision, and safety reporting
Predictive red teaming	policy $\pi$ under factor $f$	rank and predict degradation $R_f^\pi$
Embodied Red Teaming	feasible instruction set $\{c_i\}$	minimize success while maintaining diversity and feasibility
Geometric Red-Teaming	deformed mesh $D_\theta(M)$	minimize task performance under plausible geometry
Humanoid cybersecurity assessment	services, credentials, telemetry, cloud links	expose exploitation, exfiltration, and pivot potential

In HRRT, the safety problem is formalized through safety-aware reasoning, consisting of hazard identification, risk assessment, risk mitigation, and safety reporting. A hazardous condition is represented as

$$\psi = (\psi_t, s'_C, \psi_C, (s_1, \cdots, s_t), \psi_L)$$

and the manuscript defines

$$\begin{array}{c} \phi_R(\psi,s_t) = \psi_L(s'_C | s_1, \cdots, s_t) \cdot \psi_C(s_t, s'_C) \ \phi_S(\psi,s_t) = \frac{1}{\phi_R(\psi,s_t)} \end{array}$$

with state-level quantities

$$\begin{array}{c} \Phi_C(s_t) = \max_{\psi \in \Psi}{ \left( \psi_t(s_t) \cdot \psi_C(s_t,s'_C) \right) } \ \Phi_R(s_t) = \max_{\psi \in \Psi}{ \left( \psi_t(s_t) \cdot \phi_R(\psi,s_t) \right) } \end{array}$$

and task risk

$$\Lambda_R(a_{T,1}, \cdots, a_{T,t}) = \max_{s_i}{ \left( \Phi_R(s_1), \cdots, \Phi_R(s_t) \right) }.$$

A task is safely executable when

$\pi$0

The paper does not use this full formalism in the mainline experiments, but it explicitly presents these equations as the intended reasoning structure (Sheetz et al., 2 Aug 2025).

For instruction-level auditing, Embodied Red Teaming formulates red teaming as

$\pi$1

where instructions must remain feasible in the current environment. Predictive red teaming instead models robustness under environmental change through the proxy

$\pi$2

with $\pi$3 the anomaly rate of edited observations in the policy’s representation space. Geometric Red-Teaming seeks

$\pi$4

optionally under a deformation budget enforced through the Smoothness Score

$\pi$5

These formalisms differ in substrate—symbolic state models, language instructions, visual observations, or object meshes—but each turns red teaming into a search over structured, deployment-relevant failure modes rather than a fixed benchmark (Karnik et al., 2024, Majumdar et al., 10 Feb 2025, Goel et al., 15 Sep 2025).

3. Frameworks and mechanisms

The main architectural contribution of HRRT is the HRRT reasoning framework, organized into levels HRRT2, HRRT3, and HRRT4 over a model $\pi$6. At HRRT2, the team explores possible scenarios and outcomes supported by the current model: $\pi$7 At HRRT3, the team challenges assumptions implicit in actions: $\pi$8 where $\pi$9 with $f$0, and $f$1 with $f$2. At HRRT4, the team reflects on earlier outputs and updates the model: $f$3 where $f$4 is a dialogue tree implemented as a simple English-like interface. One full iteration is HRRT2 followed by HRRT3 followed by HRRT4, producing successive model hypotheses $f$5. In implementation, the red team includes a computational chatbot agent using $f$6, the blue team includes a human and a computational agent, the authors used ChatGPT as the blue computational agent, and the blue human agent typically selected 2–5 changes at each level (Sheetz et al., 2 Aug 2025).

Predictive red teaming operationalizes automated vulnerability discovery through RoboART, also expanded in the details as Robotics Auto Red Teaming (ART) and named edit in the method description. The pipeline first modifies nominal observations using Imagen 3 to instantiate environmental factors such as colored lighting, changed background colors, distractors, a nearby person, or table-height change, then predicts performance degradation using a policy-specific anomaly detector in the policy’s latent space. A Gemini Pro 1.5 vision-LLM acts as a critic, selecting the best edit or discarding a sample. The anomaly score is based on nearest-neighbor cosine distance, generalized in experiments to the mean of the $f$7-nearest-neighbor cosine distances, and the anomaly threshold is conformally calibrated so that nominal anomaly rate matches nominal hardware performance (Majumdar et al., 10 Feb 2025).

Embodied Red Teaming implements adversarial instruction search as an iterative in-context refinement loop using GPT-4o as the red-team VLM. The model receives an image of the scene, a task description, and optionally prior failure-inducing instructions; it then generates challenging instructions that remain grounded in the scene. To avoid repetition, ERT samples $f$8 instruction sets and selects the most diverse one by

$f$9

with $R_f^\pi$0 in experiments. Instructions that induce failure are appended back into the prompt, and the loop is repeated for $R_f^\pi$1 refinement rounds. The method therefore uses scene grounding, diversity selection, and failure-conditioned in-context learning rather than gradient-based adversarial optimization (Karnik et al., 2024).

Geometric Red-Teaming targets object-centric physical failure modes. It selects task-relevant handle points and anchor points on a watertight mesh, generates smooth deformations through a Jacobian field-based deformation model adapted from the first stage of APAP, evaluates each deformed mesh in simulation with a frozen policy, and searches the deformation space with a gradient-free selective perturbation optimizer. The deformation solve is written

$R_f^\pi$2

with soft handle loss

$R_f^\pi$3

The resulting CrashShapes are defined as deformed object meshes that are structurally valid, user-constrained, semantically coherent or plausible, and failure-inducing for the policy (Goel et al., 15 Sep 2025).

Cybersecurity-oriented red teaming of humanoids uses a different mechanism set. The Unitree G1 studies combine physical teardown and inspection, static analysis of filesystem and binaries, reverse engineering of master_service, runtime observation, traffic interception, cryptographic analysis, service architecture mapping, protocol analysis, and on-robot deployment of a Cybersecurity AI (CAI) agent for reconnaissance and exploitation preparation. In this framing, the robot is simultaneously a local endpoint, a ROS 2 / DDS node, a cloud-coupled appliance, a multimodal sensor platform, and a potential staging host for offensive preparation (Mayoral-Vilches et al., 17 Sep 2025, Mayoral-Vilches, 17 Sep 2025).

4. Empirical results

The strongest quantitative evidence for HRRT comes from symbolic planning. The planning experiments span eight domains: lunar habitat, Mars science team, household assembly/repairs, household cleaning, international travel, vehicle maintenance, nuclear warfare, and AI captain. For the main evaluation, the authors performed 5 HRRT iterations per domain, produced $R_f^\pi$4, converted these models to PDDL, and evaluated them with Pyperplan on 50 planning tasks per domain, for 400 total tasks. Aggregated over all model hypotheses and domains, the generated models achieved 350 successes out of 400 tasks, for an overall 0.875 success rate. Domain results were 49/50 = 0.98 for Lunar Habitat, 43/50 = 0.86 for Mars Science Team, 50/50 = 1.00 for Household Assembly/Repairs, 44/50 = 0.88 for Household Cleaning, 46/50 = 0.92 for International Travel, 47/50 = 0.94 for Vehicle Maintenance, 32/50 = 0.64 for Nuclear Warfare, and 39/50 = 0.78 for AI Captain. In the Mars Science Team ablation and saturation study, the authors ran 10 full HRRT iterations, saved 30 models total, and reported apparent model saturation around iteration 6 (Sheetz et al., 2 Aug 2025).

The execution-side HRRT results transfer red-team-derived safety knowledge into two embodiments and two environments. The paper uses NASA Johnson Space Center’s iMETRO platform for the lunar habitat and the Valkyrie humanoid robot for the household. Hazards are detected with color blob detection and YOLO object detection; mitigation selection is driven by environment-specific logistic regression risk mitigating action-utility models learned from HRRT-derived variables. The iMETRO robot performed 7 lunar-habitat trials with a 1.00 risk mitigation success rate; Valkyrie performed 5 household trials with a 0.60 risk mitigation success rate; and the combined result was 12 trials with a 0.83 cumulative success rate. The failures were reported as primarily due to false negatives in hazard detection, especially under degraded lighting, rather than failures of mitigation reasoning itself (Sheetz et al., 2 Aug 2025).

Predictive red teaming evaluates two policies with nominal success of about $R_f^\pi$5 on a real bimanual Kuka IIWA platform, across twelve off-nominal conditions and 500+ hardware trials. From Table 1, the hybrid policy achieves Spearman $R_f^\pi$6 and average prediction error $R_f^\pi$7, while the vanilla diffusion policy achieves Spearman $R_f^\pi$8 and average prediction error $R_f^\pi$9. The method therefore predicts performance degradation with average error below 0.19, and the paper demonstrates a closed-loop mitigation workflow in which fine-tuning on the three environmental factors predicted by edit to cause the highest degradation yields 2–7x improvement in the targeted off-nominal conditions and 2–5x cross-domain improvement in other lighting and background conditions (Majumdar et al., 10 Feb 2025).

Embodied Red Teaming shows that standard benchmarks overestimate robustness of language-conditioned robot models. On CALVIN, 3D-Diffuser drops from 92.9% on training instructions to 66.9% at ERT($\{c_i\}$0), 60.4% at ERT($\{c_i\}$1), and 58.4% at ERT($\{c_i\}$2); GR-1 drops from 94.1% to 61.8%, 54.7%, and 53.0% over the same refinement levels. On RLBench, 3D-Diffuser falls from about 90% on benchmark instructions to 56.1% on naive rephrasing and 0% at ERT($\{c_i\}$3). In SimplerEnv, OpenVLA drops from 76.0% on the original instruction to 30.8 \pm 3.80\% on ERT instructions. The paper also presents unsafe execution examples, including object-fall outcomes, and explicitly distinguishes robots following unsafe instructions from robots exhibiting unsafe behavior under neutral instructions (Karnik et al., 2024).

Geometric Red-Teaming produces severe policy collapse across grasping, insertion, and articulated manipulation. In the core simulation results, VLM-guided + Optimization (ours) on Contact-GraspNet reaches Final Drop 76.3%, while articulated manipulation with manual handle selection reaches Final Drop 98.9%. The method also validates on hardware: on an xArm 6, real-world insertion success falls from 90.0% on the nominal object to 22.5% on one CrashShape and 55.0% on another; on a Franka arm, grasping success drops from 80.0% to 30.0% for a mustard bottle CrashShape and from 90.0% to 35.0% for a screwdriver CrashShape. Subsequent “blue-teaming” recovers performance, for example from 25.0% $\{c_i\}$4 87.8% and 45.0% $\{c_i\}$5 93.8% in state-based insertion, while preserving nominal success (Goel et al., 15 Sep 2025).

The humanoid cybersecurity studies contribute a different empirical profile. They document persistent telemetry connections to 43.175.228.18:17883 and 43.175.229.18:17883, established within 5 seconds of system initialization, with a ReportInterval of 300 seconds, structured JSON payloads of 4.5–4.6 KB, and approximate throughput of 1.03 Mbps to the primary server and 0.39 Mbps to the secondary server during the reported capture. They also reconstruct the proprietary FMX protection scheme, identify a static Blowfish-ECB outer layer with a fleet-wide key, describe the inner LCG recurrence

$\{c_i\}$6

and operationalize a CAI agent on the robot for reconnaissance and exploitation preparation against manufacturer infrastructure (Mayoral-Vilches et al., 17 Sep 2025, Mayoral-Vilches, 17 Sep 2025).

5. Human oversight, trust, and automation

A central claim of HRRT is that trust in human-robot teams should be earned through explainable, safety-aware behavior, not assumed. The robot is not meant to blindly obey commands or rely on humans to carry all safety judgment; it is supposed to participate in hazard identification, risk assessment, risk mitigation, and safety reporting. The English-like dialogue at HRRT4 functions as a transparency mechanism, prompting questions such as whether there are independently verified external resources on failure cases, and the paper argues that inexplicable actions erode trust and may lead to disuse. Although no formal trust user study is reported, the framework is explicitly organized around calibrated trust in safety-critical collaboration (Sheetz et al., 2 Aug 2025).

The recent automation literature adds a complementary socio-technical claim: effective red teaming depends on a diverse human infrastructure, and automation should support rather than replace that infrastructure. The key framework terms are Proficiency, Agency, and Adaptability. Automation can expand scale, speed, consistency, and coverage, and can reduce worker exposure to harmful material, but over-prioritizing efficiency can reduce worker agency, degrade context-specific adaptability, and deskill the workforce. In robotics, these concerns are amplified because safety evaluation depends on embodied context, local norms, deployment constraints, and practical judgment about when robot behavior is merely unsuccessful and when it is unsafe (Zhang et al., 28 Mar 2025).

The robotics red-teaming papers instantiate several hybrid allocations of labor. In predictive red teaming, a human operator or safety analyst chooses which environmental factors to stress, then uses the automated system to estimate which conditions are most damaging and to prioritize scarce robot time for the most informative physical tests. In Embodied Red Teaming, GPT-4o generates instruction candidates, but the entire pipeline remains grounded in scene feasibility and robot rollout feedback. In HRRT, humans judge which suggested states, actions, and failure cases are meaningful, provide the evaluative perspective needed to prioritize risks, and direct model revision where computational agents do not understand ethical or operational nuance. A common misconception is that red teaming in robotics is equivalent either to autonomous attack generation or to purely human expert review. The evidence instead supports a hybrid pattern in which automation broadens search and humans retain evaluative authority over high-stakes judgments (Majumdar et al., 10 Feb 2025, Karnik et al., 2024).

6. Limitations, controversies, and research directions

The current literature is explicit about methodological limits. In HRRT, the blue human agent was one of the authors rather than an independent participant pool; the physical experiments involved only 12 total trials across two embodiments; perception false negatives were the main execution bottleneck; and several details of the risk-assessment mathematics and dataset-generation procedures remained in commented-out sections rather than the visible experimental pipeline. The planning evaluation also lacks a strong non-HRRT external baseline or a purely human-authored model pipeline. Most importantly, the paper states that iterating HRRT does not solve the “unknown unknowns” problem in any absolute sense; it only creates more opportunities to uncover them (Sheetz et al., 2 Aug 2025).

Predictive red teaming is limited to failure-inducing changes that appear in robot observations. It does not detect hidden factors such as object mass, friction, compliance, actuator degradation, or latent sensor calibration errors unless they manifest visually. The paper also identifies an edit-to-real gap, an anomaly-to-failure gap, lack of multi-view geometric consistency, sensitivity of calibration in high-dimensional embeddings, and dependence on first-timestep observations rather than longer temporal windows. Embodied Red Teaming, for its part, is mostly simulation-based, limited to manipulation tasks, and operationalizes unsafe behavior mainly through object-fall examples rather than a broad standardized taxonomy. It reports no human annotation study or inter-annotator analysis for feasibility judgments (Majumdar et al., 10 Feb 2025, Karnik et al., 2024).

Geometric Red-Teaming assumes watertight, manifold triangle meshes, discovers failures through Isaac Gym rollouts, and does not automatically explain why a discovered CrashShape fails. The method encourages plausibility through priors and preprocessing rather than a formal topology guarantee, and its efficient black-box search is not exhaustive. The humanoid cybersecurity studies likewise have sharp scope boundaries: the initial access method is intentionally withheld, offensive cloud work remained at reconnaissance and exploitation preparation rather than destructive compromise, and no full malicious locomotion or manipulation takeover is demonstrated. These limits matter because they mark the current boundary between red-team discovery and fully operational attack demonstration (Goel et al., 15 Sep 2025, Mayoral-Vilches, 17 Sep 2025).

Several future directions recur across the literature. HRRT identifies recruitment of independent expert humans as a next step. Predictive red teaming proposes 3D scene editing / Gaussian splatting, better anomaly detectors, use of multiple timesteps, and multi-round predictive red teaming. Embodied Red Teaming points toward broader embodied domains, especially navigation robots, better surrogate evaluators for failure and safety, and joint training of language representations with robot policies to capture embodied similarity. Geometric Red-Teaming suggests extensions to cooperative manipulation, handoff settings, richer physical adversaries such as wear and compliance, causal failure diagnosis, and online oversight. The humanoid security work argues for adaptive CAI-powered defenses and, more broadly, for security standards tailored to physical-cyber convergence. Taken together, these directions suggest that human-robot red teaming is moving toward integrated programs that combine symbolic safety reasoning, adversarial instruction generation, robustness prediction, physical-geometry stress testing, and cyber-physical threat analysis within a single deployment-oriented safety process (Zhang et al., 28 Mar 2025, Mayoral-Vilches et al., 17 Sep 2025).