Relationship-Aware Safety Unlearning for Multimodal LLMs

Abstract: Generative multimodal models can exhibit safety failures that are inherently relational: two benign concepts can become unsafe when linked by a specific action or relation (e.g., child-drinking-wine). Existing unlearning and concept-erasure approaches often target isolated concepts or image-text pairs, which can cause collateral damage to benign uses of the same objects and relations. We propose relationship-aware safety unlearning: a framework that explicitly represents unsafe object-relation-object (O-R-O) tuples and applies targeted parameter-efficient edits (LoRA) to suppress unsafe tuples while preserving object marginals and safe neighboring relations. We include CLIP-based experiments and robustness evaluation under paraphrase, contextual, and out-of-distribution image attacks.

• The paper introduces object–relation–object graphs and a six-term LoRA objective that selectively suppresses unsafe multimodal associations while preserving related safe concepts.
• Experiments on CLIP reduced unsafe text–image similarity by up to 0.70 across paraphrase, contextual, and out-of-distribution image attacks, while preservation drift remained at or below 0.061.
• The paper finds that consistency, preservation, and adversarial losses are essential because push-only unlearning achieves forgetting but causes greater parameter drift and utility degradation.

This paper introduces a framework for relationship-aware safety unlearning in multimodal models, targeting unsafe object–relation–object (O–R–O) compositions—such as a child drinking wine—rather than isolated concepts or image–text pairs (2603.14185). The central observation is that many safety failures are compositional: both participating objects and the relation are individually benign, so concept-level erasure either fails to address the hazard or causes collateral damage to legitimate uses of the same entities.

Motivation and problem formulation

The authors formalize three desiderata for an unlearning procedure: selectively suppress unsafe O–R–O tuples, preserve safe realizations of the same objects and relations (e.g., adult–drinking–wine), and remain robust to prompt obfuscation and compositional paraphrase. They frame this against a taxonomy of five objectives: relational safety schema design, relationship-aware parameter editing, context preservation via counterfactual losses, red-team robustness evaluation, and auditable safety criteria. The positioning relative to prior work is explicit: generator-side combination erasure in diffusion models and encoder-side pair-level unlearning (MultiDelete, CLIPErase) operate on samples, pairs, or global concepts; none represent relational structure explicitly at the encoder level, which the paper identifies as the gap it fills.

Method

The method has two components.

Relational graph construction. For a target tuple $O_1\!-\!R_1\!-\!O_2$, the authors build a graph whose nodes are objects and edges are relations. To preserve the relation elsewhere, additional nodes $O_3$, $O_4$ are attached via safe instances of $R_1$; to preserve object co-occurrence under benign relations, an edge $R_2$ connects $O_1$ and $O_2$. The graph thus provides an explicit specification of what to forget versus what must be retained—a separation absent from pair-level unlearning baselines.

Multi-objective LoRA editing. Unlearning is performed by training low-rank adapters on attention and projection layers of the target model, with a composite loss over cosine similarities in the shared embedding space: $\mathcal{L}_3$ pushes apart embeddings of the unsafe tuple; $\mathcal{L}_2$ pulls node (object) representations; $\mathcal{L}_1$ pulls safe-edge representations; $O_3$0 anchors performance on unrelated concepts; $O_3$1 is a consistency term constraining the edited model to stay near the base model; and $O_3$2 pushes adversarial paraphrases of the unsafe relation. The total objective is

$O_3$3

The LoRA choice is motivated on two grounds stated in the paper: memory efficiency given that both encoders are active during training, and reversibility/isolation of edits from the base weights.

Experimental setup and results

The evaluation removes a deliberately safe relation—"kid eating a hamburger"—from CLIP, chosen so that forgetting can be attributed to the method rather than pretraining behavior. Training data (images via gemini-2.0-flash image generation, captions via gemini-2.5-flash) is fully synthetic, enabling dataset construction for arbitrary relations without curated corpora. Training uses AdamW at learning rate $O_3$4, batch size 32, three epochs, on a single A100 40GB GPU.

Forgetting is measured as the drop in text–image cosine similarity under three attack types:

Attack type	Base cos	Post-unlearning cos	Δcos
Paraphrase	0.3010	−0.3868	0.6878
Contextual	0.2522	−0.2359	0.4881
OOD image	0.2716	−0.4296	0.7012

These are strong suppression magnitudes: post-edit similarities go negative across all three attack families, indicating the association is not merely weakened but inverted. Preservation drift on held-out safe cases is small:

| Preservation case | Base cos | Post-unlearning cos | |Δcos| | |---|---|---|---| | Single node preservation | 0.2820 | 0.3428 | 0.0608 | | New safe edge | 0.2158 | 0.2607 | 0.0450 | | New safe node | 0.2866 | 0.2981 | 0.0115 | | New neutral edge | 0.3224 | 0.2754 | 0.0470 |

The ablation compares the full objective against a baseline using only $O_3$5. The baseline achieves comparable forgetting but exhibits the largest parameter drift and substantial utility degradation; removing $O_3$6 increases drift on core concepts, and removing $O_3$7 weakens generalization of forgetting to paraphrased prompts. This supports the paper's principal empirical claim—that selective relational unlearning requires a balanced multi-objective loss rather than a single push term—and implies that push-only unlearning objectives reported elsewhere may overstate practical safety gains if utility collapse is not measured jointly.

Limitations and open questions

Several limitations are acknowledged or evident. First, evaluation is confined to CLIP as a static contrastive encoder and to a single unlearned relation ("kid eating a hamburger"); scaling to generative multimodal LLMs or diffusion models is left open, as is whether results transfer when multiple tuples are removed simultaneously or repeatedly—an interaction MUSE-style protocols show to be nontrivial. Second, the relationship graph is flat: hierarchical propagation (e.g., suppressing child–drinking–alcohol subclasses such as toddler–drinking–beer without per-tuple labeling) is proposed but not implemented. Third, adversarial robustness relies on simple text paraphrases rather than learned red-team generation, and the evaluation metric remains cosine similarity; downstream effects on zero-shot classification and retrieval accuracy are asserted as necessary but not yet measured. Fourth, the hyperparameters ($O_3$8) required careful manual tuning per the authors' own account, raising a question about sensitivity of the forget/retain trade-off across relations and models. Finally, synthetic data generation depends on external vision–LLMs, so coverage of the unsafe relation's visual variability is bounded by the generator's distribution.

Conclusion

The paper contributes an explicit relational formulation of multimodal safety unlearning—O–R–O graphs plus a six-term LoRA-based objective—and demonstrates on CLIP that large-magnitude forgetting (Δcos up to 0.70 under paraphrase, contextual, and OOD image attacks) can coexist with minimal preservation drift (≤0.061). Its main empirical lesson is that auxiliary pull, consistency, and adversarial terms are each necessary; push-only unlearning degrades utility unacceptably. Whether the approach extends to generative decoders, hierarchical relation taxonomies, and auditable metrics such as the proposed Relational Safety Score remains unresolved by the current experiments.