Separate harmful intent from general sentence content

Determine how to separate harmful intent from general sentence content in activation-probe signals for vision-language-action models, beyond the lexical information captured by a bag-of-words baseline.

Background

The paper evaluates linear probes that distinguish harmful instructions from harmless controls using internal activations of LLMs, vision-LLMs, and vision-language-action models. Although the probes perform well, a bag-of-words baseline also achieves high AUROC, indicating that some of the apparent harmful-intent signal may instead reflect general lexical or sentence-level differences.

The authors therefore leave unresolved the methodological problem of disentangling representations that specifically encode harmful intent from representations that merely encode general sentence content. Resolving this issue is important for determining whether activation-based safety monitors genuinely detect harmful intent rather than surface linguistic features.

References

A bag-of-words baseline also scores highly (0.97 to 1.00), so part of this signal may be lexical; we leave separating harm from general sentence content to future work.

— What the Guard Misses, the Robot Executes: Implied Harm in VLA Instructions  (2610.05818 - Karne et al., 5 Oct 2026) in Section 4, paragraph beginning “Robot training erodes the harm signal.”