Dual-Domain Prompter

• Dual-domain prompters explicitly separate domain-shared and domain-specific cues, ensuring effective adaptation across diverse data sources.
• They employ methods like control networks, optimal transport, and prototype projections to enhance cross-modal alignment and parameter efficiency.
• This approach overcomes monolithic prompt tuning limitations by improving transfer accuracy and minimizing domain misalignment in varied applications.

A dual-domain prompter is an architectural and algorithmic construct in prompt learning that leverages explicit representations for at least two distinct “domains” (which may be task types, data sources, modalities, or subpopulations), integrating domain-awareness directly into learned prompts or context vectors for large pre-trained models. This paradigm has emerged from limitations of monolithic prompt learning, especially in applications where base model representations and domain-specific features diverge substantially (e.g., medical imaging vs. natural photography, multi-domain sequential recommendation, task-incremental continual learning, and fusion-based vision-LLMs). The dual-domain prompter approach strategically balances shared, global knowledge with domain-conditioned cues, supporting improved adaptation, robustness, and parameter efficiency.

1. Conceptual Foundations of Dual-Domain Prompting

Dual-domain prompting seeks to overcome the domain misalignment inherent in generic prompt tuning (e.g., CoOp, standard prompt learning for CLIP) by parameterizing prompts or context tokens into two respective branches:

• Domain-shared/invariant context: Tokens or biases capturing general semantic information, invariant across all domains.
• Domain-specific context: Tokens, biases, or prompt templates suited for each concrete domain.

This separation can be explicit (as in two context banks) or implicit (e.g., using control networks to produce additive domain-conditioned biases). Dual-domain prompters exploit this scheme in both vision and language modalities, enabling joint adaptation of both the input encoding and the text/image representation space.

Representative frameworks include:

• Domain-Controlled Prompt Learning (DCPL) (Cao et al., 2023): Explicit domain-aware bias injected into both vision and language branches via control networks fed by a specific-domain encoder.
• Dual Distribution-Aware Context Prompt Learning (Dude) (Nguyen et al., 5 Jul 2024): Fusion of domain-shared and class-specific context, with alignment via unbalanced optimal transport.
• PLCR (Guo et al., 2023): Continuous prompts combining domain-invariant and domain-specific tokens for sequential recommender models.
• ADAPT (Wei et al., 2023): Intra- and inter-domain prompt injection in text encoder for federated learning.
• SDPT (Zhou et al., 16 Jul 2024): Synchronous tuning of shared prototype tokens for both image and text, leveraging linear inverse projections.
• ChordPrompt (Wang et al., 24 Jun 2025): Cross-modal prompt synergy, projecting visual and textual prompts between modalities for continual multi-domain learning.

2. Mathematical Structures and Prompt Construction

Formal formulations underpin dual-domain prompt learning. Let $x$ denote the input (image, text, or sequence), $d$ the domain index, and $c$ the class index.

DCPL (Cao et al., 2023)

• Domain embedding: $R_b = \text{Encoder}_{\text{LSDM}}(I)$.
• Control nets: $b_\ell = f_{LC}(R_b)$ (language), $b_v = f_{VC}(R_b)$ (vision).
• Prompt construction:
  • Language: $p_\ell = [v_1^{ct}, ..., v_M^{ct}] + b_\ell$.
  • Vision: $p_v = x + b_v$.

Dude (Nguyen et al., 5 Jul 2024)

• Visual tokens: $V = [v_1, ..., v_M]$.
• Shared context: $P_{ds}$, class-specific context: $P_{cs}^i$ (from LLM).
• Classification via UOT:

$$Pr(c=i|x) = \frac{\exp\left((1 - d^i)/\tau\right)}{\sum_{j=1}^K \exp\left((1 - d^j)/\tau\right)}$$

with $$d^i = \gamma_{ds} UOT(P_{ds}, V) + \gamma_{cs} UOT(P_{cs}^i, V)$$.

SDPT (Zhou et al., 16 Jul 2024)

• Shared prototype tokens $P \in \mathbb{R}^{L \times d}$.
• Modality mapping via inverse projections:

$$\Phi_{\text{text}}^{-1}(P), \;\Phi_{\text{image}}^{-1}(P),$$

using frozen fusion-layer weights, enabling synchronous representation across modalities.

PLCR (Guo et al., 2023)

• Prompt template for each domain:

$$t_k^A = [v_1, ..., v_{M_1}, d^A_1, ..., d^A_{M_2}, \text{Item}^A_k]$$

• Losses: Dual-target cross-entropy with domain separation constraint:

$L = L_A + L_B + \lambda L_{\text{sep}}$

3. Training Objectives, Regularization, and Optimization

Dual-domain prompters optimize both domain-shared and domain-specific parameters, often incorporating domain-detection or routing mechanisms for inference. Common objectives include:

• Contrastive/softmax similarity loss (as in CLIP):

$$L = -\log\,\frac{\exp(\langle f_{\text{vis}}(x), f_{\text{text}}(x, c)\rangle / \tau)}{\sum_{c'}\exp(\langle f_{\text{vis}}(x), f_{\text{text}}(x, c')\rangle / \tau)}$$

• Orthogonality/separation constraints (PLCR, Dude):

$$L_{\text{sep}} = \| V^T D^A \|_F^2 + \|V^T D^B \|_F^2$$

• Unbalanced OT distance (Dude): Minimizes transport cost between visual and prompt embeddings, subject to relaxed mass-matching penalties:

$$UOT_\lambda(\alpha, \beta) = \text{min}_{T \ge 0} \langle T, C \rangle - \lambda H(T) + \rho_1 \widetilde{\text{KL}}(T 1_N \| m) + \rho_2 \widetilde{\text{KL}}(T^T 1_M \| n)$$

• Domain-classification auxiliary loss (ADAPT):

$$L_{\text{dom}} = -\mathbb{E}_x \log p_{\text{dom}}(d | x)$$

Optimizers are typically Adam with restrained learning rate for prompt parameters. Training involves freezing the backbone encoders (vision, language, sequence) and updating only the prompt parameters and/or small control networks.

4. Inference Workflows and Routing Strategies

At inference, dual-domain prompters require either a domain assignment or a learned domain weight to select, weight, or fuse prompt branches.

• Weighted domain fusion (ADAPT): Softmax over attention distribution yields a convex combination of all domain-specific prompts per input.
• Explicit routing (PromptMono, ChordPrompt): The prompt pool for the detected domain is selected via metadata, learned prototype, or auxiliary classifier.
• Synchronous prototype projection (SDPT): Unified tokens mapped synchronously into both modalities, obviating explicit routing, and enabling joint semantic alignment.
• Online user-dependent adaptation (P^3, PLCR): Query-dependent prompt expansion, either via nearest-neighbor retrieval or few-shot fine-tuning.

5. Performance Characteristics Across Benchmarks

The dual-domain prompter strategy yields measurable improvements in transfer/generalization tasks, few-shot learning, and federated scenarios.

Framework	Main Modality	Dual-domain Mechanism	Key Performance Gains
DCPL (Cao et al., 2023)	Vision-Language	LSDM-driven domain bias in vision/text	+2.94% HM, +1.04% transfer, +4.07% medical
Dude (Nguyen et al., 5 Jul 2024)	Vision-Language	Shared + class-specific prompt + UOT	76.84% (few-shot, 4-shot), 1–2% over prior
SDPT (Zhou et al., 16 Jul 2024)	Fusion-based VL	Shared prototype, inverse projections	0.04% param tuned, 57.6 mAP (COCO), SOTA
PLCR (Guo et al., 2023)	Sequential Rec.	Domain-invariant and domain-specific	HR@10=8.06% vs. 5.17% baseline
ADAPT (Wei et al., 2023)	Federated CLIP	Inter/intra-domain prompt + detector	68.4% vs. 53.6% zero-shot
ChordPrompt (Wang et al., 24 Jun 2025)	Vision-Language	Cross-modal prompt exchange + routing	87.0% Last, +4.8 pts Transfer

Parameter efficiency is typical: SDPT tunes only 0.04% of GLIP-L parameters to outperform full fine-tuning and recent prompt/adaptor methods.

6. Comparative Merits: Dual-Domain vs. Single-Domain Prompting

Single-domain prompt frameworks are susceptible to domain drift, catastrophic forgetting, and poor cross-domain generalization. Dual-domain prompters are demonstrably superior when:

• The domains differ greatly in task structure or underlying data distribution.
• The model must operate in regimes with few available samples for novel domains.
• Parameter efficiency is required (as in federated/continual learning).
• Fine-grained, class-specific discrimination is critical.

Ablation analyses show that removing either the domain-shared prompt or the domain-specific prompt substantially degrades performance. Replacing unbalanced optimal transport with balanced OT yields noisier alignments and lower accuracy.

7. Extensions, Limitations, and Future Perspectives

Dual-domain prompters are extendable to:

• Any new domain for which a robust domain encoder exists, with dimensional compatibility.
• Multi-domain continual learning (ChordPrompt) via prompt pools and prototype-based routing.
• Fusion-models (SDPT) via inverse mapping of shared prototype tokens.

Limitations include increased inference cost if the domain encoder or prompt pool is very large, the need for careful prototype selection/routing, and potential overfitting if noisy domain-specific context is not regularized (e.g., via UOT or prompt-augmentation).

A plausible implication is that further research may pursue learned noise schedules (DCPL), multi-modal prompt gating, and hierarchical or dynamic context banks for more expressive domain adaptation without sacrificing efficiency or alignment.

8. Conclusion

Dual-domain prompters systematically enhance the adaptability, robustness, and efficiency of prompt-based parameter-efficient transfer in large pre-trained models. By architecting domain-shared and domain-specific representations and leveraging regularized alignment (e.g., via optimal transport, cross-modal fusion, or control networks), these frameworks outperform naive prompt tuning across numerous benchmarks and modalities (Cao et al., 2023, Nguyen et al., 5 Jul 2024, Zhou et al., 16 Jul 2024, Guo et al., 2023, Wei et al., 2023, Wang et al., 24 Jun 2025). The dual-domain paradigm underpins current state-of-the-art results for both generalization and specialized domain adaptation, and it is a primary trajectory for future scalable prompt learning research.