Conditional Domain Prompt Learning

• CoDoL is a prompt-learning framework designed to improve out-of-distribution generalization by integrating explicit domain information into vision-language models.
• The approach uses a Domain Meta Network to generate instance-conditioned domain tokens that, when combined with class tokens, yield better text-image alignment.
• Empirical results on benchmarks like PACS and VLCS demonstrate state-of-the-art performance improvements, confirming CoDoL’s effectiveness in managing domain shift.

Conditional Domain Prompt Learning (CoDoL) is a prompt-learning framework for out-of-distribution (OOD) generalization in vision-LLMs (VLMs), especially CLIP-style architectures, that uses domain information during prompt formation and augments it with instance-conditioned domain tokens generated from image features. In its canonical form, CoDoL addresses two difficulties emphasized in prompt-based CLIP adaptation: inaccurate text descriptions and limited vision-language embedding alignment under domain shift. Relative to static prompt learning, it belongs to the broader move toward conditional prompting inaugurated by methods such as Conditional Context Optimization (CoCoOp), which showed that image-conditional prompts generalize better to unseen classes and domains than fixed learned contexts (Zhou et al., 2022), while CoDoL specializes conditional prompting to explicit domain-aware OOD generalization (zhang et al., 18 Sep 2025).

1. Historical placement within prompt learning

Prompt learning for VLMs emerged as a parameter-efficient alternative to full fine-tuning, but early methods were largely class-centric or static. CoCoOp identified that static learned context in CoOp is not generalizable to wider unseen classes within the same dataset, and introduced a lightweight neural network that generates an input-conditional token for each image, yielding stronger domain generalization performance as well (Zhou et al., 2022). This established the basic conditional-prompting template: retain frozen encoders, learn a small prompt-related module, and condition prompts on visual evidence.

Domain-aware variants then diverged along several lines. Domain Prompt Learning (DPL) for domain generalization learns a domain prompt through a lightweight three-layer MLP from batches of unlabeled images, raising the accuracy of zero-shot CLIP from 73.7% to 79.3% on PACS, VLCS, OfficeHome, and TerraIncognita while freezing the backbone (Zhang et al., 2021). Domain Adaptation via Prompt Learning (DAPL) embeds domain information directly into prompts through domain-agnostic and domain-specific context tokens, reframing unsupervised domain adaptation as prompt learning rather than explicit feature alignment (Ge et al., 2022). In federated multi-domain settings, ADAPT learns intra-domain textual prompts and shared visual prompts, then uses attention over visual prompts to infer domain correspondence before mixing textual prompts for prediction (Wei et al., 2023). For specialized domains such as remote sensing and medical imaging, Domain-Controlled Prompt Learning (DCPL) injects domain biases from large-scale specific domain models into both the visual and language branches (Cao et al., 2023).

CoDoL occupies a specific point in this design space. It does not rely solely on implicit instance conditioning, and it does not treat domain information as a fixed label-conditioned template. Instead, it uses readily-available domain information to form prompts and introduces a lightweight Domain Meta Network (DMN) to generate input-conditional tokens for images in each domain, thereby capturing both domain-specific and instance-specific information (zhang et al., 18 Sep 2025).

2. Formal problem setup and prompt parameterization

CoDoL is formulated on samples $(x, y, k)$, where $x \in \mathcal{X}$ is an image, $y \in \mathcal{Y}$ is a label, and $k \in \{1, \ldots, K\}$ is a domain index. The training set is

$\mathcal{D}_{tr} = \{(x_i, y_i, k_i)\}_{i=1}^n,$

and the objective is to maximize accuracy on a novel test distribution $p_{ts} \ne p_{tr}$ (zhang et al., 18 Sep 2025).

The prompt sent to the text encoder for class $y$ and domain $k$ is constructed as

$$t_{y,j}(x)= \underbrace{[V]_1^c, \ldots, [V]_{M_c}^c}_{\text{Class tokens}}, \underbrace{[V]_1^k(x), \ldots, [V]_{M_k}^k(x)}_{\text{Instance-specific domain tokens}}, y, k_j.$$

Here, $[V]_m^c$ are learnable class context tokens, while $x \in \mathcal{X}$0 are instance-conditioned domain tokens. The visual embedding is first computed as $x \in \mathcal{X}$1, and the DMN $x \in \mathcal{X}$2 maps $x \in \mathcal{X}$3 to a vector in $x \in \mathcal{X}$4. The instance-conditioned domain tokens are then defined by

$x \in \mathcal{X}$5

Given the resulting text embedding $x \in \mathcal{X}$6, class probability is computed by aggregating over domain-conditioned prompts:

$x \in \mathcal{X}$7

Training uses the cross-entropy objective

$x \in \mathcal{X}$8

Inference predicts the label by maximizing posterior probability over domain-conditioned prompts. As in other prompt-learning approaches in this lineage, CoDoL keeps the backbone encoders frozen and trains only prompt- and DMN-related parameters (zhang et al., 18 Sep 2025).

3. Domain information and vision-language alignment

The central claim of CoDoL is that prompt-based CLIP adaptation suffers from inaccurate text prompts and limited vision-language embedding alignment under domain shift. Its remedy is explicit domain conditioning in the text branch together with instance-conditioned domain tokens from the DMN. By adding domain tokens to the textual prompt, CoDoL aligns the domain’s visual characteristics to the expected language representation, increasing cross-modal similarity even when the test distribution differs from training (zhang et al., 18 Sep 2025).

This emphasis on alignment distinguishes CoDoL from prompt methods whose primary concern is class-shift robustness. CoCoOp improves generalization by generating an image-specific conditional token, but its formulation does not explicitly encode a domain index (Zhou et al., 2022). DPL also conditions prompts on domain evidence, but does so by generating a domain prompt from a batch of unlabeled images from the current domain and concatenating it to class prompts (Zhang et al., 2021). DAPL, by contrast, embeds domain information into prompts through domain-agnostic and domain-specific context tokens and uses pseudo-labeling for unlabeled target data in unsupervised domain adaptation (Ge et al., 2022). CoDoL combines explicit domain-aware prompting with instance conditioning, making domain awareness part of the prompt semantics rather than only a by-product of image-conditioned modulation.

Empirically, the paper reports visualizations showing higher cosine similarity between image and text embeddings when domain information is included, which is presented as direct evidence of improved multimodal alignment. It also evaluates OOD generalization in two settings: multi-domain training with leave-one-domain-out evaluation, and the more challenging single-domain training regime in which the model is trained on one domain and tested on all others (zhang et al., 18 Sep 2025).

4. Benchmarks and empirical profile

CoDoL is evaluated on four OOD benchmarks: PACS, VLCS, OfficeHome, and DigitDG. The dataset descriptions reported with the method are: PACS with 4 domains and 7 classes; VLCS with 4 domains and 5 classes; OfficeHome with 4 domains and 65 classes; and DigitDG with 4 domains and 10 classes (zhang et al., 18 Sep 2025).

Under multiple training domains, the reported ViT-B/16 results are state of the art among compared methods: 98.77% on PACS, 88.55% on VLCS, 85.63% on OfficeHome, and 82.56% on DigitDG. The summary further states average accuracy improvements of 1–3% over prior SOTA on most benchmarks and backbones, including RN50 and ViT-B/16 (zhang et al., 18 Sep 2025).

Benchmark	Structure	CoDoL (ViT-B/16)
PACS	4 domains, 7 classes	98.77%
VLCS	4 domains, 5 classes	88.55%
OfficeHome	4 domains, 65 classes	85.63%
DigitDG	4 domains, 10 classes	82.56%

The single-training-domain setting is described as particularly challenging, and CoDoL is reported to outperform alternatives by approximately 3–4% on challenging splits. Ablation studies indicate that removing DMN or replacing it with a class meta network degrades performance, supporting the claim that input-conditional domain prompt tokens are essential. The work also reports that longer domain prompt token sequences often yield better results, and that even when only 20% of training samples have domain labels, CoDoL still outperforms prior prompt-tuning methods; using more domain information leads to more stable and higher performance (zhang et al., 18 Sep 2025).

5. Relation to adjacent conditional-prompt paradigms

CoDoL can be situated among several neighboring families of conditional prompt learning that differ in conditioning signal, architectural locus, and target transfer setting.

Method	Conditioning signal	Characteristic mechanism
CoCoOp (Zhou et al., 2022)	Image feature	Meta-Net generates an input-conditional token
DPL (Zhang et al., 2021)	Unlabeled domain batch	Three-layer MLP generates a domain prompt
ADAPT (Wei et al., 2023)	Domain correspondence from visual prompt attention	Weighted combination of domain-specific text prompts
DCPL (Cao et al., 2023)	Domain embedding from an LSDM	Domain biases control both visual and language branches
MuGCP (Yang et al., 11 Jul 2025)	MLLM-derived semantic and visual conditionals	AMG and MPF integrate SCP, VCP, and contextual prompts

This comparison clarifies what is distinctive in CoDoL. First, it assumes readily-available domain information rather than recovering domain structure only implicitly. Second, it combines explicit domain tokens with an input-conditional generator, whereas CoCoOp uses image-conditioned prompting without a domain index and DPL conditions on aggregated domain evidence from unlabeled batches (Zhou et al., 2022). Third, it remains parameter-efficient: like several prompt-learning relatives, it freezes the VLM encoders and trains only a small prompt-related subsystem, but unlike DCPL it does not require a large-scale specific domain model to provide domain biases (Cao et al., 2023). Fourth, unlike MuGCP, which leverages Multi-modal LLMs as conditional prompt learners and introduces Attention Mutual-Guidance and Multi-Prompt Fusion, CoDoL is centered specifically on domain-aware OOD classification rather than the broader problem of multimodal conditional prompt generation (Yang et al., 11 Jul 2025).

A plausible implication is that CoDoL occupies an intermediate position between explicit domain-conditioned prompting and purely instance-conditioned prompting. It is more domain-aware than CoCoOp, but less dependent on auxiliary semantic machinery than MuGCP or on external domain-specific foundation models than DCPL.

6. Misconceptions, design trade-offs, and later extensions

A common misconception is that CoDoL is simply a static domain-specific text template. In fact, the reported formulation contains both learnable class context tokens and instance-specific domain tokens generated by DMN from the image embedding, so the prompt is not fixed even within a single domain (zhang et al., 18 Sep 2025). Another misconception is that domain-aware prompting necessarily requires full backbone adaptation. CoDoL, DPL, CoCoOp, and several related methods instead preserve frozen encoders and update only lightweight prompt modules, indicating that domain-aware transfer can be parameter-efficient (Zhang et al., 2021).

The principal trade-off is the role of explicit domain information. CoDoL is designed to exploit readily-available domain information, and its robustness study shows gains even when only 20% of samples have domain labels. By contrast, Prompt Customization (PC) performs input-driven prompt generation and modulation without requiring a domain/task index at inference, and is described as a generalization of conditional domain prompt learning to instance-level adaptation (Dai et al., 2024). In federated learning, ADAPT softens explicit domain assignment by learning multiple domain prompts and using attention weights to determine how much each prompt should influence a given image (Wei et al., 2023). These alternatives suggest that the choice between explicit domain labels and implicit instance-level adaptation is not binary, but a continuum of conditioning strategies.

Several later developments extend the conditional-prompt idea in directions relevant to CoDoL. ProMIM integrates masked image modeling into conditional prompt learning and reports that masking 50%–75% of image patches improves generalization by reducing overfitting in image-conditioned prompt generation (Bui et al., 7 Aug 2025). MuGCP uses MLLMs to generate Semantic Conditional Prompts and Visual Conditional Prompts, with cross-modal interaction pushed beyond the final output layer (Yang et al., 11 Jul 2025). In continual learning, probabilistic prompt modeling represents each prompt as a Gaussian distribution and uses a query-conditioned mixture to sample diverse prompts, explicitly targeting prompt collapse (Park et al., 6 Jul 2026). A plausible implication is that future CoDoL-like systems may combine explicit domain tokens with richer semantic conditionals, probabilistic prompt distributions, or prompt-pool composition mechanisms, while retaining the central CoDoL objective of improving vision-language embedding alignment under distribution shift.