Dynamic Domain Prompting

• Dynamic Domain Prompting is a method that conditions prompts on domain cues, instance signals, or online domain shifts to tailor performance for specialized image and text tasks.
• It leverages recurrent design patterns such as external domain-knowledge control, instance-conditioned mutual prompting, and online domain memory for adaptive feature modulation.
• Empirical results on datasets like DomainNet and ImageNet-C demonstrate improved accuracy and efficiency compared to static prompting techniques.

Searching arXiv for the cited and closely related papers on dynamic/domain-aware prompting. Searching "(Cao et al., 2023) Domain-Controlled Prompt Learning" Dynamic Domain Prompting denotes a family of prompt-learning strategies in which the prompt is not a fixed, global template, but is conditioned on domain cues, instance cues, or online estimates of domain shift. In the literature summarized here, this conditioning is used to adapt frozen or mostly frozen backbones—including CLIP, ResNet, ViT, DLA detectors, rerankers, and dynamic graph models—by injecting domain-sensitive signals into visual features, language features, or both. The resulting systems target specialized image domains such as remote sensing and medical imaging, unsupervised and source-free domain adaptation, continual test-time adaptation, document layout analysis, open-domain retrieval, dynamic graphs, and domain-specific LLM applications (Cao et al., 2023, Du et al., 2024, Zhang et al., 2024, Zhang et al., 10 Mar 2026, Abdallah et al., 2024, Tang et al., 25 Jun 2025).

1. Conceptual scope and nomenclature

Prompt learning originally emerged as a parameter-efficient transfer paradigm for large pre-trained vision-LLMs such as CLIP. In its static form, a small set of learnable context tokens is optimized while the backbone remains frozen. Dynamic Domain Prompting extends that idea by making the prompt depend on domain information rather than treating the prompt as universally valid across all inputs. In specific-domain recognition, this addresses the claim that existing prompt learning methods often lack domain-awareness or domain-transfer mechanisms, leading to suboptimal performance because specialized images are misinterpreted in natural-image patterns (Cao et al., 2023).

Within federated CLIP adaptation, a related formulation appears in ADAPT, whose basic idea is that the prompted CLIP should detect the input image’s domain correspondence before making the prediction of its category. The abstract reports that, by learning and sharing only $0.08$M parameters, ADAPT attains a $68.4\%$ average accuracy over six domains in DomainNet and improves the original CLIP by $14.8\%$ (Wei et al., 2023). At the same time, the supplied details explicitly distinguish ADAPT from Fed-DPT and state that the searched document “Dual Prompt Tuning for Domain-Aware Federated Learning” does not contain a module or algorithm named ADAPT. This clarifies a nomenclature issue: “dynamic” or “domain-aware” prompting is not a single standardized method name, but a broader design space with multiple, only partially overlapping implementations.

A recurring misconception is that dynamic prompting is equivalent to learning better text templates. The surveyed works contradict that reduction. Some methods inject domain-conditioned biases into both vision and language streams, some perform instance-conditioned mutual prompting across modalities, some maintain online prompt memories for recurring domains, and some generate prompts from descriptive domain knowledge or retrieval signals rather than from labels alone (Cao et al., 2023, Du et al., 2024, Zhang et al., 2024, Zhang et al., 10 Mar 2026).

2. Recurrent design patterns

Across the literature, Dynamic Domain Prompting can be organized into a small number of recurrent mechanisms.

System	Setting	Dynamic signal
DCPL	Specific-domain image recognition	LSDM-derived language and vision biases
DAMP	Unsupervised domain adaptation	Instance-conditioned mutual visual/text prompting
DPCore	Continual test-time adaptation	Prompt coreset with ID/OOD update
PromptDLA	Document layout analysis	Descriptive domain cue $d$ or inferred domain
DynRank	Passage reranking	Question-type-conditioned prompt template
DyGFM	Multi-domain dynamic graphs	Divergence-conditioned semantic and temporal prompts

The first pattern is external domain-knowledge control. DCPL uses a frozen large-scale specific domain model, such as a ViT-MAE trained on remote sensing or MedSAM for medical images, and maps its encoder output into domain biases that control both the visual and language branches of CLIP (Cao et al., 2023). PromptDLA uses explicit natural-language domain descriptions such as “a document page of financial report” or “Persian scientific article page,” then injects the resulting prompt embedding into a DLA backbone (Zhang et al., 10 Mar 2026).

The second pattern is instance-conditioned multimodal prompting. DAMP learns a single domain-agnostic prompt context, then uses image contextual information to prompt the language branch and, symmetrically, uses the textual prompt to guide visual prompting. The prompt is therefore domain-agnostic in parameterization but instance-conditioned in effect (Du et al., 2024).

The third pattern is online domain memory and routing. DPCore keeps a prompt coreset in which each stored prompt summarizes a previously seen target domain. Incoming batches are matched against that memory, and the method decides whether to refine a weighted combination of known prompts or to create a new prompt for an out-of-domain batch (Zhang et al., 2024). DyGFM generalizes this idea to multi-domain dynamic graphs by computing semantic and temporal divergence to route among source-domain experts and then generating divergence-conditioned graph prompts for downstream adaptation (Yuan et al., 13 May 2026).

The fourth pattern is dynamic prompt generation by retrieval or classification. DynRank classifies a question into fine-grained types and maps the predicted type to a prompt template used for reranking passages (Abdallah et al., 2024). In domain-specific AI applications, dynamic prompt recommendation combines contextual query analysis, retrieval-augmented grounding, hierarchical skill organization, adaptive skill ranking, and telemetry to synthesize prompt suggestions (Tang et al., 25 Jun 2025). A broader LLM-oriented variant clusters tasks by semantic similarity and selects prompting techniques adaptively from a knowledge base (Ikenoue et al., 20 Oct 2025).

3. Vision-language mechanisms

In vision-language adaptation, Dynamic Domain Prompting often takes the form of explicit feature modulation. DCPL extends a frozen CLIP (ViT-B/16) by inserting two lightweight MLP control nets that consume the frozen LSDM representation $R_b \in \mathbb{R}^{d_r}$ and output a language bias $D_b^l$ and a vision bias $D_b^v$, both in the CLIP embedding space. The MLPs follow a “Linear $\rightarrow$ ReLU $\rightarrow$ Linear” pattern,

$$f_{LC}(R) = W_2^l(\mathrm{ReLU}(W_1^l R + b_1^l)) + b_2^l,\quad f_{VC}(R) = W_2^v(\mathrm{ReLU}(W_1^v R + b_1^v)) + b_2^v.$$

The text-side learnable context vectors are shifted by $68.4\%$0, the CLIP image feature $68.4\%$1 is shifted by $68.4\%$2, and Gaussian noise is added through

$68.4\%$3

before softmax classification. The authors describe this noise as a way to escape suboptimal solutions in a global domain oscillation manner (Cao et al., 2023).

DAMP moves from domain-controlled biasing to mutual prompting. With CLIP’s image encoder $68.4\%$4 and text encoder $68.4\%$5 frozen, it learns a single prompt context $68.4\%$6 and a prompting module $68.4\%$7. Language-guided visual prompting updates the image embedding through a decoder stack,

$68.4\%$8

while vision-guided language prompting symmetrically updates each class embedding,

$68.4\%$9

Final prediction is made from $14.8\%$0, and the training objective combines supervised classification, semantic-consistency, information-maximization, and an instance-discrimination contrastive loss (Du et al., 2024). The paper’s central claim is that prompting only the language branch lacks flexibility, and the proposed bidirectional prompting addresses that limitation.

A related but distinct line appears in prompt-driven dynamic object-centric learning for single-domain generalization. Here, hand-crafted scene-description prompts such as “an image taken in a foggy scene” are encoded by frozen CLIP, and the resulting prompt embedding matrix $14.8\%$1 interacts with ResNet feature maps through cross-modal attention,

$14.8\%$2

followed by Slot-Attention-style processing that yields binary spatial and channel gating masks. The gated feature map is

$14.8\%$3

Although this work is framed as scene-prompted object-centric perception rather than domain-labeled prompting, it exemplifies the same general principle: prompt-conditioned dynamic allocation of representational capacity (Li et al., 2024).

4. Dynamic prompting under domain shift and continual adaptation

In source-free domain generalization, DPStyler treats domain variation as style variation in CLIP text space. At the start of each epoch, it refreshes $14.8\%$4 style tokens using either random generation from one of five initialization distributions or StyleMix, where coefficients are drawn from $14.8\%$5 and mixed over a small adjective dictionary. This removes the two-stage “fix styles then train” limitation described for earlier approaches (Tang et al., 2024). The style-removal module then applies channel-wise attention,

$14.8\%$6

and maximizes entropy over a style-domain classifier to enforce domain uncertainty. The full loss is

$14.8\%$7

The model also uses an ensemble of $14.8\%$8 templates to mitigate prompt sensitivity (Tang et al., 2024).

Continual Test-Time Adaptation introduces a different challenge: domains recur with varying frequencies and durations, so prompt parameters must be updated online without catastrophic forgetting. DPCore formalizes this by maintaining a coreset

$14.8\%$9

where each element stores a prompt and domain statistics. For a new batch $d$0, similarity weights are computed as

$d$1

The method then compares $d$2 to $d$3. If the weighted prompt suffices, the batch is treated as in-domain and the existing core elements are updated by weighted interpolation; otherwise, the batch is treated as out-of-domain and a new prompt is learned from scratch and appended to the coreset (Zhang et al., 2024). The abstract reports that DPCore reduces trainable parameters by $d$4 and computation time by $d$5 compared to previous approaches.

DyGFM extends dynamic prompting to multi-domain dynamic graph foundation models. It decouples transferable semantics from domain-specific temporal dynamics during pre-training, computes semantic and temporal divergence between source and target domains, and routes among source experts with

$d$6

Target-domain prompts are then generated by two small MLPs,

$d$7

and injected into frozen semantic and temporal encoders (Yuan et al., 13 May 2026). This suggests that dynamic prompting has expanded from token-level conditioning in VLMs to routing-conditioned adapters in graph foundation models.

5. Descriptive knowledge, retrieval, and domain-specific prompt generation

PromptDLA shows how dynamic domain prompting can operate when the domain prior is itself a natural-language description. The framework interleaves a domain-aware prompter with standard DLA backbones such as ViT+FPN+Cascade R-CNN or DiT+DETR. Given image $d$8 and domain cue $d$9, the prompter generates

$R_b \in \mathbb{R}^{d_r}$0

where $R_b \in \mathbb{R}^{d_r}$1 may be human-curated templates, LVLM generation, or a hybrid, and $R_b \in \mathbb{R}^{d_r}$2 is a frozen text encoder such as CLIP or LLaMA (Zhang et al., 10 Mar 2026). The prompt may be prepended in ViTs, tiled and concatenated in CNNs, or fused per window in Swin. A notable factual point is that, at training or inference, the domain cue may be known or inferred by a lightweight classifier, with ViT zero-shot CLIP reported to achieve approximately $R_b \in \mathbb{R}^{d_r}$3 on DocLayNet.

In open-domain QA retrieval, DynRank applies dynamic zero-shot prompting through question classification rather than visual domain descriptors. A RoBERTa-base question classifier predicts one of 5 coarse-grained and 50 fine-grained question types, then a mapping $R_b \in \mathbb{R}^{d_r}$4 selects a template tailored to that type. Passage reranking uses the prompt-conditioned likelihood

$R_b \in \mathbb{R}^{d_r}$5

so that different prompts steer the PLM toward pseudo-questions focused on dates, persons, entities, or quantities as appropriate (Abdallah et al., 2024).

For domain-specific AI applications, dynamic prompting becomes a system-level orchestration problem. The reported architecture includes a contextual query processor, a retrieval engine using FAISS and ElasticSearch, a two-level plugin/skill taxonomy, a learned ranking function with telemetry features, and prompt synthesis from predefined and adaptive templates with few-shot examples. The ranking feature vector is

$R_b \in \mathbb{R}^{d_r}$6

and the ranking loss is a pairwise hinge objective (Tang et al., 25 Jun 2025). In a broader LLM setting, adaptive prompt generation via task clustering uses semantic embeddings, cosine assignment to cluster centroids, and cluster-to-technique mappings to synthesize prompts without relying on pre-existing templates or frameworks (Ikenoue et al., 20 Oct 2025).

These systems broaden the meaning of Dynamic Domain Prompting beyond domain adaptation in the narrow statistical sense. Here, “domain” can denote application context, document type, question type, or task cluster. A plausible implication is that the same architectural principle—dynamic selection of prompt structure from structured side information—has become portable across both perception and generation pipelines.

6. Empirical behavior, limitations, and open issues

A consistent empirical pattern is that domain-aware or dynamically conditioned prompts outperform static baselines. On eight remote-sensing datasets, DCPL reports an average harmonic mean of approximately $R_b \in \mathbb{R}^{d_r}$7, versus $R_b \in \mathbb{R}^{d_r}$8 for MaPLe, and in cross-dataset transfer from MLRSNet to the other seven datasets it reports $R_b \in \mathbb{R}^{d_r}$9 average accuracy versus $D_b^l$0 for MaPLe (Cao et al., 2023). DAMP reports $D_b^l$1 average accuracy on Office-Home with a ResNet-50 backbone, compared with $D_b^l$2 for PADCLIP, and $D_b^l$3 on Mini-DomainNet with ViT-B/16, compared with $D_b^l$4 for PADCLIP (Du et al., 2024). DPCore reports $D_b^l$5 average accuracy on ImageNet-C under dynamically-continual adaptation, versus $D_b^l$6 for EATA (Zhang et al., 2024). PromptDLA reports $D_b^l$7 on DocLayNet, $D_b^l$8 on M6Doc, and $D_b^l$9 on D$D_b^v$0LA with a DiT backbone (Zhang et al., 10 Mar 2026). DynRank reports an average $D_b^v$1 nDCG@10 on a BEIR subset, exceeding $D_b^v$2 for RankGPT and $D_b^v$3 for UPR (Abdallah et al., 2024).

The ablations also identify what makes the prompting dynamic rather than merely learnable. In DCPL, visual-only bias yields HM $D_b^v$4, language-only bias yields HM $D_b^v$5, and combining both yields $D_b^v$6; the proposed Gaussian noise gives an HM boost of approximately $D_b^v$7, exceeding naive dropout or small-mutation jitter at no more than approximately $D_b^v$8 (Cao et al., 2023). In prompt-driven object-centric learning on PACS, moving from a static ResNet-18 baseline at $D_b^v$9 to dynamic gating without prompts yields $\rightarrow$0, adding prompts with normal attention yields $\rightarrow$1, and the full prompts plus Slot-Attention model yields $\rightarrow$2 (Li et al., 2024). In DyGFM, removing SS-IB, routing, or conditioned prompts lowers node and link performance, and the reported fine-tuning converges in approximately $\rightarrow$3 epochs versus $\rightarrow$4 for TIGPrompt and $\rightarrow$5 for DyGPrompt (Yuan et al., 13 May 2026).

Several limitations recur. PromptDLA notes the computational overhead of LVLM-based prompt generation and dependence on domain-label availability or accurate domain classification in fully zero-shot scenarios (Zhang et al., 10 Mar 2026). DynRank notes that reranking $\rightarrow$6 passages with an LLM is expensive and that its 50 templates are manually written (Abdallah et al., 2024). DPStyler reports sensitivity to the initial text template, which motivates model ensembling (Tang et al., 2024). DyGFM identifies scalability to a very large number of domains as an open question because expert count grows with the number of source domains (Yuan et al., 13 May 2026). In dynamic prompt recommendation, removing telemetry features causes a $\rightarrow$7–$\rightarrow$8 drop in usefulness, and collapsing the two-stage hierarchy into flat retrieval increases latency by $\rightarrow$9 and reduces precision@1 by approximately $\rightarrow$0 (Tang et al., 25 Jun 2025).

A final point of debate concerns whether simply merging domains or learning a single prompt should suffice. The surveyed evidence argues otherwise. PromptDLA reports that jointly training DocLayNet and PubLayNet without prompts decreases DocLayNet mAP from $\rightarrow$1 to $\rightarrow$2 because of annotation conflicts, while adding domain prompts recovers and exceeds single-domain performance (Zhang et al., 10 Mar 2026). DAMP explicitly argues that separate domain-specific textual prompts limit cross-domain knowledge transfer and that language-only prompting lacks flexibility (Du et al., 2024). DPCore shows that adapting one shared set of parameters across dynamically recurring domains leads to convergence problems, forgetting, or misapplication of knowledge in real-world CTTA settings (Zhang et al., 2024). Taken together, these results position Dynamic Domain Prompting not as a single algorithm, but as a general response to the inadequacy of static prompting under heterogeneous, recurring, or specialized domain shifts.