Safe Completion in VLMs

• Safe completion in VLMs is defined as generating correct, context-appropriate outputs by mitigating input–output misalignment and sensitivity to minor perturbations.
• The research highlights that even trivial changes in prompts or images can lead to significant drops in performance, jeopardizing tasks like object localization and manipulation.
• Proposed strategies include robust benchmarking, real-time detection with human-in-loop support, and explainability tools to enhance safety in critical environments such as robotics and healthcare.

Safe completion in vision-LLMs (VLMs) refers to the reliable generation of correct, safe, and context-appropriate outputs in response to multimodal stimuli—typically images and text—especially within safety- or risk-critical domains. This concept encompasses not only the avoidance of harmful, biased, or unsafe content, but also robust resilience to benign variations in input and adversarial manipulations. The need for safe completion is acute in applications such as robotics, autonomous systems, healthcare, and broad public deployment. Recent research has rigorously examined the vulnerability of VLM-controlled agents to minor perturbations in input modalities, exposing significant safety and alignment challenges even in highly performant models.

1. Input Modality Sensitivities in VLM-Controlled Systems

VLM-controlled systems interpret and act upon both natural language instructions and visual (RGB, segmentation) observations. Models often assume rigid prompt structures—typically $$\text{Action} + \text{Base Object} + \text{Target Object}$$, where “Base Object” and “Target Object” are adjective-noun pairs such as “red swirl block” or “purple container.” Visual inputs are expected to be of high quality and structured predictably. Minor deviations in these modalities, such as prompt rephrasings or slight visual translation, cropping, or rotation, can precipitate substantial interpretive errors.

In empirical studies, robotic platforms leveraging state-of-the-art VLMs (e.g., KnowNo, VIMA, Instruct2Act) were shown to fail catastrophically with small, semantically neutral modifications to input (e.g., altering adjective from “red” to “crimson”, or applying a $<10^\circ$ image rotation) (Wu et al., 15 Feb 2024). Such input sensitivity has direct safety consequences: misidentification of objects or mislocalization in a manipulation task can cause execution errors, equipment damage, or safety hazards in real deployments.

2. Misalignment-Induced Vulnerabilities: Formalization and Impact

The primary mechanism driving unsafe completions is input–output misalignment. Misalignment occurs when the incoming prompt or image departs from the patterns or templates seen during model training, leading the model’s keyword detection or perception pipeline astray. This is formalized by the paper as:

• Prompt structure: $$\text{Action} + \text{Base Object} + \text{Target Object}$$
• Disruption types: Prompt misalignment (e.g., stealth or noun rephrasings break expected parsing); perception misalignment (e.g., geometric transformations distort object recognition).

Even trivial misalignments—such as swapping a noun with a common synonym or adding an artifact to the segmentation mask—can interfere with both language and perception modules. The downstream effect is misclassification or erroneous task execution, phenomena that are especially unacceptable in safety-critical contexts.

3. Empirical Perturbation Strategies and Adversarial Evaluations

To quantify robustness and safe completion, systematic perturbation strategies have been explored—both in the textual and perceptual domains. Representative attacks include:

• Prompt Attacks: Five principal types are considered:
  1. Simple Rephrasing
  2. Stealth Rephrasing (subtle semantic drift)
  3. Extension Rephrasing (redundancy injection)
  4. Adjective Rephrasing (long-form synonym substitution)
  5. Noun Rephrasing (object label substitution)

• Perception Attacks:

  • Image quality attacks (blurring, noise)
  • Transformation attacks (rotation, translation, cropping)
  • Object addition (insertion of artifacts in RGB/seg maps)
• Mixture Attacks: Composite of prompt and perception perturbations

Benchmarks consist of manipulation and scene understanding tasks (e.g., pick-and-place with generalization levels; object localization; rearrangement). The core metric is task execution success rate, measured across hundreds of task attempts.

4. Quantitative Results and Systemic Implications

Safe completion performance in VLM-robotics is dramatically reduced under even lightweight attack scenarios. The measured performance drops include:

• Prompt attacks: For KnowNo and similar models, success rates drop to 15.3–18.7%. On VIMA, prompt attacks reduce visual manipulation accuracy by 15.5%.
• Perception attacks: Induce up to 40% drops in some task settings.
• Mixture attacks: Compound failures; overall, 21.2% reduction for prompt attacks and 30.2% for perception attacks observed in manipulation benchmarks.

Importantly, minor perturbations (e.g., $<10^\circ$ rotation) suffice for failure, demonstrating the non-trivial impact of small deviations. The implication is that present-day VLM-controlled robotics lack adequate safety margins when facing realistic sensor noise or linguistic variation. This makes them unsuitable for unsupervised deployment in safety-critical environments.

5. Strategies for Enhancing Safe Completion

Robust and safe completion necessitates multi-pronged approaches:

• Robustness Benchmarks: Systematic adversarial datasets encompassing input modality variations for both training and evaluation.
• Safeguard Mechanisms: Incorporate “ask for help” or uncertainty deferral—where the system automatically triggers human-in-the-loop review upon detecting anomalous input.
• Explainability and Interpretability Tools: Identify linguistic or perceptual pipeline vulnerabilities to implement targeted defenses.
• Real-Time Detection and Feedback: Introduce runtime metrics and watchdog mechanisms to intercept unsafe or anomalous completions before execution.
• Multi-Modal Vulnerability Auditing: Expand robustness inspection beyond text and image to include audio, sensor fusion, or complex contextual scenes.

6. Conclusions and Outlook

The evidence from manipulation and perception perturbation studies shows that safe completion in VLM-based robotics remains an unsolved problem: minor deviations in either prompt or perception channels can precipitate outsized failures, many with significant real-world safety implications. Adversarial benchmarking quantifies these vulnerabilities and sets the stage for more rigorous standards for robustness in VLM deployments.

Effective solutions require deeper integration of safety checks, adversarially resilient architectures, and systematic human-in-the-loop mechanisms. There is a strong call for the community to develop benchmark-driven and formal methods for safeguarding VLM deployments—particularly in high-stakes applications where the cost of unsafe completion is intolerably high (Wu et al., 15 Feb 2024).