SynOOD: Boundary-Aware OOD Detection

• SynOOD is a boundary-aware method that synthesizes near-boundary OOD samples using diffusion-based inpainting and MLLM-guided prompts.
• It iteratively refines synthetic images with noise gradients and an energy objective to sharpen decision boundaries in CLIP embeddings.
• Empirical results show SynOOD improves AUROC and reduces FPR95 with minimal overhead, demonstrating its practical impact on OOD detection.

SynOOD is a boundary-aware approach for out-of-distribution (OOD) detection in pre-trained vision-LLMs, particularly CLIP. The central objective is to synthesize near-boundary OOD samples—images that are difficult for vanilla OOD detectors because they closely resemble in-distribution (InD) samples in the embedding space—then use these to fine-tune the model for sharper boundary discrimination. SynOOD integrates iterative diffusion-based inpainting with multimodal LLMs (MLLMs) for prompt selection, producing OOD examples that are both realistic and semantically controlled. Empirical results demonstrate that SynOOD yields state-of-the-art OOD detection performance with minimal additional computational overhead (Li et al., 14 Jul 2025).

1. Motivation and Problem Statement

In CLIP-style vision-language embedding spaces, some OOD images—despite being truly out-of-sample—lie close enough to InD class clusters to produce higher matching scores with InD text labels than with any negative label. As a result, even powerful pre-trained models may misclassify these “near-boundary” OOD examples, limiting practical deployment of foundation models for OOD detection tasks. SynOOD is designed to directly address this limitation by synthetically generating OOD data precisely at the challenging decision boundary, then using these data to fine-tune CLIP such that its image-text matching becomes maximally discriminative at the InD/OOD boundary.

2. Foundation Models: Diffusion and Multimodal LLMs

SynOOD leverages two categories of foundation models:

• In-painting diffusion models: Models such as Stable Diffusion provide a principled mechanism to gradually transform an InD image through controlled corruption and denoising steps. This process enables fine-grained semantic manipulation needed for near-boundary synthesis.
• Multimodal LLMs (MLLMs): Architectures like LLaVA are used to extract “contextual prompts”—secondary scene labels (e.g., “bamboo”, “leaf”, “tourist” for a “panda” image). These prompts guide the semantic direction of inpainting, ensuring that the OOD sample diverges subtly from the InD class while retaining high structural fidelity.

The integration of diffusion models for sample generation and MLLMs for contextual guidance permits controlled, photorealistic, and semantically meaningful sampling at the InD/OOD interface.

3. Iterative In-Painting and Noise Refinement Pipeline

The SynOOD pipeline for synthesizing near-boundary OOD samples is as follows:

1. Contextual prompt extraction: For each InD image $x^{\rm in}$ and prompt $p^{\rm in}$, an MLLM $\phi$ yields secondary labels $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$. CLIP’s text encoder $\psi$ then provides a prompt feature $P = \psi(p^{\rm con})$.
2. Latent variable preparation: The InD image is encoded via a VAE $f$, producing $z_{\text{in}} = f(x^{\rm in})$. Initial noise $\epsilon \sim \mathcal{N}(0,I)$ is introduced.
3. Iterative refinement:
   • Perform forward diffusion to timestep $T$:

   $p^{\rm in}$0

- Denoise iteratively back to $p^{\rm in}$1 using a U-Net noise predictor $p^{\rm in}$2 and contextual prompts. - Reconstruct candidate OOD image $p^{\rm in}$3 from $p^{\rm in}$4. - Evaluate OOD “energy” loss $p^{\rm in}$5 (see next section). - Backpropagate an approximated gradient to update the noise variable $p^{\rm in}$6, steering generation toward the InD/OOD boundary while maximizing the OOD objective. - Iterate a few steps.

1. Result: A synthesized OOD image $p^{\rm in}$7 residing near the InD/OOD boundary.

4. OOD Energy Objective and Skip-Gradient Mechanism

The formulation for the OOD objective is derived from the “energy score” used in prior OOD detection frameworks: $p^{\rm in}$8 where $p^{\rm in}$9 are output logits and $\phi$0 is a temperature parameter. The practical loss for the SynOOD generation step is: $\phi$1 Here, $\phi$2 and $\phi$3 modulate the loss on a frozen classifier $\phi$4.

Gradients of $\phi$5 with respect to the initial noise $\phi$6 are computed by differentiating through the chain of VAE, U-Net, and the classifier. To reduce computational overhead, a “Skip Gradient” approximation is employed: $\phi$7 The noise is then updated via: $\phi$8 where $\phi$9 is a learning rate. This process enables controlled movement of the synthetic data toward regions critical for fine-grained OOD detection.

5. CLIP Fine-Tuning via Projection and Negative-Label Loss

Using the synthetic OOD images, SynOOD performs a twofold fine-tuning:

1. Image-encoder (projection) loss: CLIP’s backbone $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$0 is frozen, and only a lightweight projection head $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$1 is learned. For each sample pair $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$2, with $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$3 and text encoding $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$4:

   $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$5

   This cross-modal contrastive objective is defined over a balanced set of real InD and synthetic OOD instances.

2. Negative-label text-feature loss: A subset of negative text features $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$6 is made learnable. For synthetic OOD embeddings $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$7, a parallel loss sharpens discrimination from negative class features:

   $p^{\rm con} = \phi(x^{\rm in},\,p^{\rm in})$8

Ablation studies indicate that step-by-step fine-tuning combining both losses yields optimal results (Li et al., 14 Jul 2025).

6. Empirical Evaluation and Performance

SynOOD demonstrates substantial empirical gains on canonical OOD benchmarks:

Setting	AUROC (%)	FPR95 (%)
NegLabel baseline	94.21	25.40
SynOOD	97.01	14.27

This corresponds to a 2.80% improvement in AUROC and an 11.13% reduction in FPR95 on ImageNet-1k versus four common OOD sets. Further, SynOOD attains strong performance on the OpenOOD benchmark under both near-OOD and far-OOD regimes using only 50,000 InD images for fine-tuning.

Ablations reveal that the iterative inpainting (“Grad Image”) pipeline substantially outperforms one-shot text-to-image approaches, and that ~50,000 synthetic samples constitute an optimal trade-off between accuracy and resource usage. Overhead is minimal, with parameter growth under 1% and inference cost increasing by approximately 2 ms per image.

7. Significance, Limitations, and Directions for Extension

SynOOD empirically establishes the value of boundary-aware synthetic data generation for sharpening OOD detection in vision-LLMs. The tightly integrated diffusion-MLLM pipeline enables direct targeting of hard OOD cases without requiring architectural changes, yielding near-zero deployment burden for existing CLIP-based workflows.

Open questions include:

• Generalization to architectures beyond CLIP.
• Performance of alternate generative priors (e.g., text-guided GANs).
• Adaptive calibration of negative-label sets for various domains.

A plausible implication is that extensions of this paradigm—iterative, context-driven sample synthesis plus fine-tuned negative-feature learning—could inform future frameworks for robust decision boundary refinement in diverse foundation model settings (Li et al., 14 Jul 2025).