---
title: Forced Prompt Learning (FA) for OOD Detection
url: https://www.emergentmind.com/topics/forced-prompt-learning-fa
type: topic
---

# Forced Prompt Learning (FA) for OOD Detection

Forced Prompt Learning (FA) is a CLIP-based prompt-learning framework for few-shot out-of-distribution (OOD) detection that improves OOD separability by enriching **in-distribution (ID)** semantics rather than by learning explicit OOD-related knowledge. In the formulation introduced in "FA: Forced Prompt Learning of Vision-Language Models for Out-of-Distribution Detection" [2507.04511], the method operates in the realistic regime where only a small number of labeled ID samples are available for downstream tuning and no true OOD training data are assumed. Its central idea is to learn a **forced prompt** that must become more semantically aligned with ID images than a frozen reference prompt initialized from the same manual template. The resulting prompt is intended to encode more diversified, richer, and more nuanced descriptions of ID classes than class labels alone provide, thereby improving ID/OOD separability without external auxiliary datasets, synthetic OOD samples, or negative labels [2507.04511].

## 1. Definition and problem setting

FA is defined for **few-shot OOD detection with CLIP** under the constraint that only a small labeled ID training set is available for adaptation. The method is motivated by the observation that many recent CLIP-based OOD detectors improve performance by injecting or approximating **OOD-related knowledge**, for example through auxiliary OOD datasets, exposed outlier regions, synthetic “ID-like” negatives, or negative prompts. FA is proposed as a contrasting design principle: instead of approximating the open-ended OOD world, it seeks better OOD detection by learning **richer ID semantics** from the few-shot ID set alone [2507.04511].

In this formulation, CLIP supplies a frozen image encoder \(f(\cdot)\) and text encoder \(g(\cdot)\). For \(C\) ID classes, a handcrafted prompt \(\hat{\mathbf{u}}_c\) such as “a photo of a [class-c]” yields text features
\[
\hat{\mathbf{t}}_c = g(\hat{\mathbf{u}}_c)\in \mathbb{R}^{d\times 1}, \qquad c=1,\dots,C.
\]
Given an image \(\mathbf{x}\), the image feature is
\[
\mathbf{z}=f(\mathbf{x})\in \mathbb{R}^{d\times 1}.
\]
The baseline prompt-tuning reference is CoOp-style learning of context vectors
\[
\mathbf{u}_c = [\mathbf{v}_1,\cdots,\mathbf{v}_L,\mathbf{w}_c], \qquad c\in\{1,\cdots,C\},
\]
with class posterior
\[
p(y=c\,|\,\mathbf{x}) = \dfrac{e^{\cos(\mathbf{z},\mathbf{t}_c)/\tau}}{\sum_{j=1}^C e^{\cos(\mathbf{z},\mathbf{t}_j)/\tau}}.
\]
FA retains the frozen CLIP backbone but changes the prompt-learning objective: the trainable prompt is not merely optimized for closed-set classification, but is explicitly trained to be **more salient than a frozen original prompt for ID samples** [2507.04511].

A practical implication is that FA is an **ID-centric** OOD detector. It does not require external auxiliary OOD datasets, exposed outlier regions, synthetic OOD generation, negative prompts, or OOD labels. The method also keeps the **same number of trainable parameters as CoOp**, because the added original-prompt branch is frozen rather than trainable [2507.04511].

## 2. Prompt architecture and forced objective

FA introduces two prompt branches per class concept: an **original prompt** and a **forced prompt**, both initialized from the same manual template. The original prompt remains frozen and serves as a semantic reference; the forced prompt is trainable. The image encoder, text encoder, and class-name tokens in both prompts are frozen, and only the context vectors of the forced prompt are updated. As in low-shot CoOp, the forced prompt uses a **shared learnable vector across classes** rather than class-specific independent context vectors [2507.04511].

Let the text encoder produce class-wise text features for the two branches:
- original prompt features: \(\{\mathbf{t}_1^o,\dots,\mathbf{t}_C^o\}\),
- forced prompt features: \(\{\mathbf{t}_1^f,\dots,\mathbf{t}_C^f\}\).

For image feature \(\mathbf{z}\), FA computes
\[
s_j^f = \cos(\mathbf{z},\mathbf{t}_j^f), \qquad s_j^o = \cos(\mathbf{z},\mathbf{t}_j^o).
\]

The key innovation is the **forced cross-entropy loss**
\[
\mathcal{L}_{FCE} = \mathbb{E}_{(\mathbf{x}, y_c) \sim \mathcal{D}_{train}^{ID}} \left[ -\log \frac{e^{s_c^f/\tau}}{\sum_{j=1}^C e^{s_j^f/\tau} + \sum_{j=1}^C e^{s_j^o/\tau}} \right].
\]
This loss expands the softmax denominator to include both the trainable forced-prompt similarities and the frozen original-prompt similarities. Because the original prompt already carries meaningful CLIP semantics, the forced prompt can dominate only by learning a better class description for ID samples than the baseline prompt already provides [2507.04511].

FA then introduces the **forced coefficient** \(K\), a nonnegative integer hyperparameter:
\[
K \ge 0,\qquad K\in\mathbb{N}.
\]
The refined loss is
\[
\mathcal{L}_{FCE-K} = \mathbb{E}_{(\mathbf{x}, y_c) \sim \mathcal{D}_{train}^{ID}} \left[ -\log \frac{e^{s_c^f/\tau}}{\sum_{j=1}^C e^{s_j^f/\tau} + K\sum_{j=1}^C e^{s_j^o/\tau}} \right].
\]
When \(K=0\), the original-prompt term disappears and the method degenerates to ordinary CoOp-like training. As \(K\) increases, the original prompt acts as a stronger semantic competitor. In the paper’s interpretation, this increases the “intensity” with which the model is compelled to learn from ID data, encouraging more comprehensive class semantics [2507.04511].

The conceptual claim is precise: class names and their default CLIP text embeddings are too coarse to fully describe a class, so a prompt trained to surpass the manual prompt on ID images should encode richer semantic content. This suggests that FA improves OOD detection not by modeling the OOD universe, but by making the representation of ID classes more complete [2507.04511].

## 3. Training pipeline and OOD scoring

The FA training pipeline is deliberately simple. It begins with pre-trained CLIP, using **ViT-B/16 in experiments**. Two prompt branches are built from the same manual initialization “a photo of a [class-c].” CLIP’s image encoder, text encoder, the original prompt, and the class-name token portion are frozen. Only the shared context tokens of the forced prompt are learnable. For each ID training pair \((\mathbf{x},y_c)\), FA computes \(\mathbf{z}=f(\mathbf{x})\), obtains \(\mathbf{t}_j^o\) and \(\mathbf{t}_j^f\), computes \(s_j^o\) and \(s_j^f\), and optimizes the forced prompt with \(\mathcal{L}_{FCE-K}\). There is **no extra regularization term beyond this loss in the main method** [2507.04511].

At inference, FA uses only the **forced prompt** branch for ID classification. For OOD detection, it plugs into standard CLIP scoring rules, especially **MCM** and **GL-MCM**, modified to account for both forced and original prompt similarities. For MCM, with global image feature \(\mathbf{z}^g=\mathbf{z}\) and combined candidate text feature set \(\mathbf{t}_c^a\), the paper gives
\[
S_{\mathrm{MCM}}(\mathbf{x}) = \max_c \frac{e^{\cos(\mathbf{z}^g,\mathbf{t}_c^a)/\tau_0}}{\sum_{j=1}^C e^{\cos(\mathbf{z}^g,\mathbf{t}_j^f)/\tau_0} + K e^{\cos(\mathbf{z}^g,\mathbf{t}_j^o)/\tau_0}},
\]
with \(\tau_0=1\) at inference. For GL-MCM,
\[
S_{\text{GL-MCM}}(\mathbf{x}) = S_{\text{MCM}}(\mathbf{x}) + S_{\text{L-MCM}}(\mathbf{x}),
\]
where
\[
S_{\text{L-MCM}}(\mathbf{x}) = \max_{i,c} \frac{e^{\cos(\mathbf{z}_i^l,\mathbf{t}_c^a)/\tau_0}}{\sum_{j=1}^C e^{\cos(\mathbf{z}_i^l,\mathbf{t}_j^f)/\tau_0} + K e^{\cos(\mathbf{z}_i^l,\mathbf{t}_j^o)/\tau_0}},
\]
and \(\mathbf{z}_i^l\) is the \(i\)-th local feature extracted from the CLIP image encoder [2507.04511].

OOD detection then follows standard thresholding:
\[
D(\mathbf{x})=
\begin{cases}
1, & \text{if } S(\mathbf{x})\ge \mu \\
0, & \text{if } S(\mathbf{x})< \mu
\end{cases},
\]
where \(1\) denotes ID and \(0\) denotes OOD. Relative to prior prompt-learning OOD methods, FA is architecturally closest to CoOp, but differs in the addition of a **frozen reference prompt** and the **forced loss** [2507.04511].

## 4. Empirical performance and benchmark behavior

FA is evaluated on standard **ImageNet-1k OOD benchmarks** using iNaturalist, SUN, Places, and Textures as OOD datasets, and also on more challenging cleaner OOD datasets **OpenImage-O, NINCO, and ImageNet-O**. The experiments cover **1-shot, 4-shot, and 16-shot** settings and also test transfer to UCF101, EuroSAT, Food101, StanfordCars, Caltech101, FGVCAircraft, Flowers102, and OxfordPets. The reported metrics are **FPR95**, **AUROC**, and **ID Top-1 accuracy** [2507.04511].

On ImageNet-1k conventional OOD benchmarks, FA achieves the strongest average OOD performance among methods that do not use real outliers. In the **1-shot** setting, **FA\(_{GL}\)** reaches **27.81 FPR95 / 93.26 AUROC**, improving over **SCT\(_{GL}\)** at **31.62 / 92.01**. In the **16-shot** setting, **FA\(_{GL}\)** obtains **25.68 / 93.82**, compared with **27.27 / 93.31** for **SCT\(_{GL}\)**. The paper also emphasizes that even **FA\(_{MCM}\)** in 1-shot slightly exceeds **SCT\(_{GL}\)** on average AUROC/FPR95, which is presented as evidence that the gain stems from better prompt learning rather than only from the scoring rule [2507.04511].

On the more challenging cleaner OOD datasets in the **16-shot** setting, **FA\(_{GL}\)** achieves **53.61 FPR95 / 83.93 AUROC**, improving over **SCT\(_{GL}\)** at **58.25 / 82.24**. **FA\(_{MCM}\)** also improves strongly there, reaching **57.40 / 84.08** [2507.04511].

A notable secondary result is that FA improves OOD detection **without severely harming ID classification**. On ImageNet-1k top-1 ID accuracy, FA is close to the strongest prompt-learning baselines: in **1-shot**, FA reports **68.67%** versus **68.63%** for SCT; in **4-shot**, **69.96%** versus **69.93%**; and in **16-shot**, SCT remains slightly higher at **71.78%** versus **71.02%** for FA, while FA’s OOD performance is better [2507.04511].

Cross-dataset transfer reinforces the same interpretation. On **UCF101** as ID in 16-shot, FA is reported as dramatically stronger than prior methods, with average MCM performance around **2.63 FPR95 / 99.42 AUROC** and GL around **4.56 / 99.03**. On **EuroSAT**, the paper highlights particularly large gains: **FA\(_{MCM}\)** exceeds **LoCoOp\(_{GL}\)** by **16.09 AUROC points** and **38.7 FPR95 points** on average. This suggests that richer ID semantics can be especially useful when the downstream domain departs from ImageNet-like natural images [2507.04511].

## 5. Ablations, robustness, and design choices

The ablation studies show that FA’s improvement is not obtained merely by adding another prompt branch. Replacing \(\mathcal{L}_{FCE-K}\) with standard cross-entropy produces a much weaker model. On ImageNet-1k 16-shot, **FA\(_{CE}\)** yields **41.43 / 91.01** with MCM and **34.29 / 90.99** with GL-MCM, whereas **FA\(_{FCE-K}\)** gives **29.07 / 93.77** and **25.68 / 93.82**. The paper’s conclusion is explicit: the forced competition objective is essential [2507.04511].

Initialization is also critical. If both prompts are randomly initialized, performance drops substantially. The best setup is **manual/manual** initialization for both the forced and original prompt. In the 16-shot ImageNet-1k benchmark, this configuration gives **29.07 / 93.77** with MCM and **25.68 / 93.82** with GL, outperforming mixed or random alternatives. This supports the argument that the original prompt should serve as a semantically meaningful CLIP prior rather than an arbitrary baseline [2507.04511].

The choice of a **shared learnable vector** rather than class-specific independent vectors also matters in low-shot settings. The shared+manual configuration gives the best result, consistent with CoOp’s earlier low-data observation that class-specific prompts are harder to train with limited supervision [2507.04511].

The **forced coefficient \(K\)** is reported as effective but not fragile. The paper fixes **\(K=3\)** in all experiments. Sensitivity analysis over \(K=1\) to \(6\) shows fairly stable performance above the best baseline, with AUROC improving and then plateauing as \(K\) increases. The authors interpret the plateau as a bottleneck in how much more comprehensive semantics the forced prompt can extract. The ablation also confirms that **\(K>0\)** is beneficial and that **\(K=0\)** degenerates to CoOp [2507.04511].

Computationally, FA remains lightweight: all experiments were run on a **single Nvidia A30 GPU**. This is relevant because several competing prompt-learning OOD methods introduce additional trainable negative prompts or other OOD-oriented mechanisms, whereas FA preserves CoOp-level trainable parameter count [2507.04511].

## 6. Relation to neighboring prompt-learning methods and terminological scope

FA belongs to the CLIP prompt-learning literature, but its design principle differs from several adjacent prompt-learning directions. Relative to **CoOp**, FA retains frozen CLIP encoders and learns prompt context vectors, yet adds a frozen reference prompt and a forced loss that turns prompt learning into a competition against a manual semantic prior [2507.04511]. Relative to **LoCoOp** and **SCT**, the difference is more philosophical: those methods improve OOD detection by calibrating against ID-irrelevant local regions or OOD-like signals, whereas FA improves OOD detection by making ID class semantics more complete [2507.04511].

The term “forced prompt learning” should also be distinguished from other prompt-learning formulations that are unrelated to CLIP OOD detection. **Prompt Diffusion** learns a diffusion model in prompt space to produce sample-conditional prompts for robustness under distribution shift, but it is a plug-in adaptation mechanism for textual, visual, or multimodal prompts rather than an OOD detector built around forced competition with a frozen reference prompt [2410.20164]. **Fed-CPrompt** studies task-specific prompts in rehearsal-free federated continual learning and relies on asynchronous prompt learning plus a contrastive continual loss; it is not an FA method in the sense of [2507.04511] [2307.04869]. Likewise, **PromptFL** and related federated prompt-learning methods move adaptation into a small prompt parameter manifold for distributed optimization, but address federated classification rather than OOD detection [2208.11625].

A second disambiguation concerns the abbreviation **FA** itself. In speech research, FA often means **forced alignment**, as in comparisons of Montreal Forced Aligner with WhisperX and MMS [2406.19363], end-to-end neural alignment with bidirectional attention [2203.16838], or slot-filling speech-LLM alignment for multilingual long-form audio [2601.18220]. That usage is terminologically unrelated to **Forced Prompt Learning** in CLIP-based OOD detection [2507.04511].

## 7. Limitations, benchmark interpretation, and significance

The paper identifies one practical limitation in benchmark interpretation: FA performs less impressively on **SUN** in the conventional ImageNet benchmark. The authors attribute this partly to contamination issues discussed in recent OOD work, since SUN contains many images overlapping with ImageNet semantics and is therefore not a perfectly clean OOD set [2507.04511].

A broader limitation is conceptual rather than numerical. FA assumes that richer ID descriptions are sufficient to improve OOD separation. The empirical evidence is strong, but the method does **not explicitly model adversarially close OOD categories**. The paper does not report severe failure cases beyond dataset cleanliness and the likely saturation effect at large \(K\), where the ability of the forced prompt to extract more comprehensive semantics appears to plateau [2507.04511].

Even with that limitation, FA is significant because it introduces a clean alternative principle for CLIP-based OOD detection: rather than approximating the unknowable OOD universe, one can improve OOD robustness by enriching the model’s understanding of ID classes. Technically, this is achieved with a minimal modification to CoOp-style prompt tuning: add a frozen reference prompt, train a copy to beat it through forced cross-entropy, and control the competition through a forced coefficient \(K\). The method uses no auxiliary OOD data, keeps the trainable parameter count equal to CoOp, and achieves state-of-the-art few-shot OOD detection across standard and challenging benchmarks [2507.04511].

Source: https://www.emergentmind.com/topics/forced-prompt-learning-fa