The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models

Abstract: Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics (e.g., FID, CLIPScore) that are insensitive to fine-grained semantic correctness, creating an illusion of high utility. We show that when utility is measured with structured evaluation, this illusion breaks: on TIFA (Text-to-Image Faithfulness evaluation with Question Answering), safety-aligned models suffer substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships. To diagnose the source of this gap, we analyze the text-encoder prompt embedding space and uncover semantic collapse, a contraction of embedding spread coupled with distortion of inter-prompt similarity structure, which strongly correlates with structured utility loss. Guided by this insight, we propose StructureAware Geometric Regularization (SAGE), a safety alignment objective that explicitly preserves embedding spread and inter-prompt relational structure during adaptation. Our method restores structured utility (TIFA +5.0% over prior state-of-the-art) while maintaining strong safety performance and competitive coarse-grained utility scores. Our source code and trained models are available at https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/.

• The paper demonstrates that standard global metrics like FID and CLIPScore conceal significant losses in compositional and semantic fidelity, creating a 'utility mirage'.
• It reveals that safety alignment induces embedding spread contraction and local neighborhood distortion, leading to a semantic collapse in text-to-image models.
• The proposed SAGE framework integrates embedding spread preservation and local structure alignment to restore fine-grained compositional integrity while maintaining robust safety.

The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models

Introduction and Motivation

Text-to-image (T2I) diffusion models have quickly reached widespread usage due to their high fidelity and diverse outputs, but their training on web-scale datasets exposes them to unsafe semantic concepts, leading to NSFW content generation. The field has therefore devoted significant effort toward safety alignment: steering models away from generating harmful content while nominally preserving their usefulness on benign prompts.

A prevailing narrative in recent literature claims that advanced safety alignment techniques largely eliminate the safety–utility trade-off, supposedly delivering strong unsafe content suppression with minimal utility loss. This judgment, however, is grounded predominantly in coarse global metrics such as FID and CLIPScore. The paper "The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models" (2607.00402) rigorously interrogates these claims, providing structured evaluation, geometric embedding analyses, and proposing an improved alignment framework that reveals and corrects hidden degradations.

Exposing the Metric-Utility Gap

The central thesis is that the standard deployment of global utility metrics (FID, CLIPScore) introduces a "utility mirage": aligned models appear to maintain high utility but are, in fact, systematically failing to satisfy compositional and semantic constraints embedded in complex prompts. Structured benchmarks such as TIFA, which assess not only perceptual similarity but elementwise prompt adherence (attributes, counts, relational constraints), reveal substantial losses in semantic fidelity across mainstream safety alignment techniques.

Empirical analysis demonstrates significant drops in TIFA and GenEval scores across widely-used methods, even when FID and CLIPScore appear competitive with the base model. Importantly, TIFA exposes severe category-level collapses (e.g., a 13% drop in food-related prompts for DES), indicating that certain semantic spaces are highly vulnerable to alignment collapse.

Figure 1: Category-level TIFA utility drop and CLIPScore for images generated by DES; CLIPScore fails to reflect category-level compositional degradation captured by structured metrics.

Diagnosis: Semantic Collapse in Embedding Geometry

The investigation attributes these degradations to what is termed semantic collapse in the prompt embedding space. Existing safety alignment techniques, operating primarily via text encoder fine-tuning, induce two characteristic distortions:

• Embedding Spread Contraction: Reduction in the overall spread of benign prompt embeddings, compressing the semantic space, impeding discriminability required for compositional rendering.
• Local Neighborhood Distortion: Disruption of the relative structure; semantically distinct prompts are drawn together or reshuffled in the adapted embedding space, obliterating the base model's relational geometry.

Pairwise cosine distance analyses confirm that methods like DES induce widespread off-manifold reshaping, while SAGE preserves base-model geometry.

Figure 2: Pairwise semantic distance distortion for 400 benign prompts; DES severely distorts relationships, while SAGE preserves base-model geometry (higher Spearman correlation).

Quantitatively, reduction in embedding spread ratio correlates strongly with TIFA degradation across both methods and semantic categories. Large relative drops in local neighborhood overlap (Jaccard score) are also tightly linked with compositional performance loss.

Figure 3: Relationship between spread ratio and structured utility: methods with high spread contraction have larger TIFA drops.

The SAGE Framework: Structure-Aware Geometric Regularization

To directly counteract semantic collapse, the authors propose SAGE (Structure-Aware Geometric Regularization), which supplements standard alignment losses with explicit regularizers targeting embedding geometry:

• Embedding Spread Preservation (ESP): Penalizes contraction in the benign prompt embedding space spread, enforcing a lower bound relative to the base model and guarding against universal collapse.
• Local Structure Alignment (LSA): Matches the pairwise similarity structure among top-K neighborhood pairs, maintaining semantic partitioning crucial for compositional generation. To avoid restoring unsafe directions, LSA is applied along concept-perturbed axes (e.g., after nudity direction perturbation).

The overall loss combines safety constraints, point-wise utility alignment, ESP, and LSA, providing a principled route to maintain fine-grained semantics.

Experimental Results

Structured Utility

On TIFA, SAGE nearly closes the gap to the base model (75.4 vs. 76.3 TIFA), outperforming DES (+5.0%) and STEREO (+7.3%) and, crucially, recovers category-level performance, e.g., food-related prompts. This restores compositional faithfulness missed by coarse metrics.

Robustness & Safety

SAGE achieves average attack success rates (ASR) competitive with the strongest baselines (~1.2%), even under contemporary jailbreak and adaptive attack scenarios, demonstrating that embedding geometry preservation does not weaken safety. Notably, SAGE outperforms prior methods under adaptive white-box attacks (e.g., U3-Attack), establishing strong empirical robustness.

Distributional Quality

Improved FID (15.93, lower than base 17.23) and maintained CLIPScore (26.4 vs 26.5) confirm that regularization does not compromise global generative quality.

Compositional Generalization

On GenEval and T2I-CompBench++, SAGE achieves minimal drops (<2%) from the base in synthetic compositional tasks, strongly outperforming prior aligned models.

Qualitative Assessment

Qualitative samples show SAGE preserving detailed compositional semantics (object attributes, relationships) that other safety-aligned methods consistently erode.

Figure 4: Qualitative comparison on compositional prompts; SAGE preserves compositionality lost by prior safety-aligned methods.

Figure 5: Qualitative comparison (Base vs. SAGE) for benign prompts; SAGE maintains semantic details.

Theoretical and Practical Implications

These results invalidate narratives relying exclusively on FID or CLIPScore for safety utility claims. They establish that text-encoder based safety alignment must account for higher-order geometric properties of semantic embeddings, not simply pointwise similarity. Embedding geometry preservation is thus essential for retaining prompt adherence and instruction fidelity under alignment constraints.

For practitioners, this research mandates the inclusion of structured utility benchmarks and embedding geometric analyses in the evaluation of safety-aligned T2I models. Future directions entail extending geometry-aware regularization to diffusion backbones and developing unsupervised methods for discovering unsafe concept axes and managing their entanglements.

Conclusion

This work rigorously demonstrates that standard safety alignment evaluations for T2I models are insufficient—coarse metrics obscure critical compositional failures. Semantic collapse of the embedding space explains widespread fidelity loss under state-of-the-art alignment methods. The proposed SAGE framework, by directly regularizing both spread and local relational structure of embeddings, significantly mitigates these effects, restoring structured utility while maintaining robust safety. This paradigm represents a substantial methodological advancement in safe and faithful generative modeling, and its principles are likely foundational for subsequent alignment research in multimodal and instruction-following AI systems.

• "The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models" (2607.00402)