Dual-Prompt Framework Overview

• Dual-Prompt Framework is a set of methods that leverages two distinct prompt configurations to decouple and contrast components, enhancing robustness, interpretability, and adaptability.
• Methodologically, dual prompts are created by applying orthogonal perturbations to separate components, enabling controlled evaluation and joint optimization across various tasks.
• Empirical studies demonstrate significant performance gains, efficient continual learning, and robust multimodal fusion when using dual-prompt approaches in AI systems.

A dual-prompt framework is a class of methods that leverage two distinct, complementary prompt structures or prompt pools to control learned representations or evaluation settings in LLMs, vision-LLMs, or multimodal architectures. Dual-prompt frameworks systematically decouple or contrast prompt components, yielding increased robustness, interpretability, and adaptability compared to single-prompt designs. These frameworks manifest across prompt evaluation, representation learning, continual learning, federated adaptation, and multimodal fusion, among others.

1. Principles and Formalism of Dual-Prompt Frameworks

Dual-prompt frameworks are founded on the principle of introducing two independent prompt configurations, typically denoted $p^{(A)}$ and $p^{(B)}$, to stress the model in orthogonal or complementary ways. This dualism can be achieved by perturbing different prompt components, learning two physically separate prompt vectors, or maintaining independent pools for task-general and task-specific information.

Formally, in a modular prompt system such as PromptSuite, a prompt is represented as a tuple of $n$ components: $p = (c_1, c_2, \ldots, c_n)$ A perturbation function $\delta_i$ applied to $c_i$ gives a new prompt: $$p' = (\delta_1(c_1), \delta_2(c_2), \ldots, \delta_n(c_n))$$ Dual-prompt evaluation consists of constructing two prompt variants by applying different perturbation types $\tau_A, \tau_B$ to separate components, producing $p^{(A)}$ and $p^{(B)}$: $p^{(B)}$0 where only one component per variant is perturbed and the rest are held constant (Habba et al., 20 Jul 2025).

In collaborative learning settings, dual prompt pools $p^{(B)}$1 (shared) and $p^{(B)}$2 (private) are introduced, as in continual or federated learning, and their interaction is subject to explicit decoupling constraints for stability and adaptivity (Zhang et al., 2 Mar 2026, Zheng et al., 21 Oct 2025).

2. Dual-Prompt Designs Across Modalities

Dual-prompt frameworks are instantiated in a range of modalities, each adapted to domain-specific constraints:

• Textual Prompt Evaluation (PromptSuite): Orthogonal perturbations are applied to instruction and format components to generate prompt pairs for robust LLM assessment (Habba et al., 20 Jul 2025).
• Prompt Optimization (P3): System and user prompts are jointly optimized, yielding coupled improvements over optimizing either alone (Zhang et al., 21 Jul 2025).
• Vision-Language Representation: Approaches such as DPC (Li et al., 17 Mar 2025), DUDE (Nguyen et al., 2024), and DCAR (Wang et al., 6 Aug 2025) define two prompts with domain-shared/class-specific or attribute/category semantics, fusing them via adapters, self-attention, or prompt weighting.
• Multimodal and Federated Adaptation: In multimodal segmentation and federated learning, dual prompts are used for modality-specific and domain-specific fusion (DPLNet (Dong et al., 2023), FedDEAP (Zheng et al., 21 Oct 2025), DP2FL (Chang et al., 23 Apr 2025)).
• Incremental and Continual Learning: Dual-pool architectures (LDEPrompt (Li et al., 13 Apr 2026), PDP (Zhang et al., 2 Mar 2026)) maintain separate pools for old-task and new-task prompts, managing freezing, expansion, and instance-level retrieval.

A non-exhaustive summary table is provided below:

Subdomain	Dual-Prompt Mechanism	Representative Work
LLM evaluation	Perturb instruction/format	PromptSuite (Habba et al., 20 Jul 2025)
Prompt optimization	System & user prompt joint	P3 (Zhang et al., 21 Jul 2025)
Continual learning	Shared/private prompt pools	PDP (Zhang et al., 2 Mar 2026), LDEPrompt (Li et al., 13 Apr 2026)
Vision-language	Base/new, attr/cat, dom/cls	DPC (Li et al., 17 Mar 2025), DUDE (Nguyen et al., 2024), DCAR (Wang et al., 6 Aug 2025)
Multimodal/federated	Global/task, dom/pers	DP2FL (Chang et al., 23 Apr 2025), FedDEAP (Zheng et al., 21 Oct 2025)

3. Methodological Workflows

Controlled Dual-Prompt Evaluation

In multi-prompt evaluation, as in PromptSuite (Habba et al., 20 Jul 2025), researchers:

1. Define modular prompts: Select clear decomposition into instruction, format, demos, instance content.
2. Specify perturbation types: For dual-prompt, pick two non-overlapping perturbations (e.g., LLM-based instruction paraphrasing, rule-based format restructuring).
3. Generate paired prompts: For each instance, construct the two variants $p^{(B)}$3, $p^{(B)}$4 following orthogonality and semantic-preservation constraints.
4. Evaluate model robustness: Compare consistency of outputs, sensitivity (score variance per input), and aggregate accuracy or F1 (e.g., substantial swings of 10–30% accuracy from instruction perturbations on challenging tasks).

Joint Dual-Prompt Optimization

In frameworks such as P3 (Zhang et al., 21 Jul 2025):

• Offline, both system and user prompts are iteratively refined; user-prompt complements are amassed via LLM search/ranking and system prompts are co-optimized to match the evolving user-prompt space.
• Online, an amortized function (a small retriever or fine-tuned mapping) is used to produce query-specific user-prompt supplements, maintaining synergy with the jointly evolved system prompt.
• Metrics used include task accuracy, consistency, and LLM-judge scores; optimization yields improvements far beyond unilateral tuning (e.g., +7.6 points over state-of-the-art on Arena-hard QA).

Pool Decoupling, Expansion, and Regularization

Incremental learning methods (e.g., PDP (Zhang et al., 2 Mar 2026), LDEPrompt (Li et al., 13 Apr 2026)) adopt:

• Dual prompt pools: One for task-general (shared, forward-transfer) knowledge (kept trainable across tasks) and one for task-specific, which is frozen after each task.
• Directional decoupling loss: Maintains orthogonality between shared and private prompt features to eliminate prompt coupling and drift.
• Prompt retrieval and dynamic expansion: For a new instance, keys are matched across pools; after each task, new prompts are absorbed into the global pool, supporting scalability.

4. Empirical Impact and Evaluation

Dual-prompt frameworks have demonstrated quantifiable gains across multiple benchmarks and modalities:

• Robustness: Dual-prompt evaluation reveals the substantial variance due to prompt design, making it indispensable for reliable model assessment (e.g., 10–30% swings on GPQA-Diamond, (Habba et al., 20 Jul 2025)).
• Performance: Joint dual-prompt optimization (e.g., P3) achieves state-of-the-art accuracy across QA and mathematical reasoning (e.g., +3.4% over PAS on GSM8K), while dual-pool decoupling in incremental detection yields up to +9.2% mAP over prior best (Zhang et al., 2 Mar 2026).
• Generalization and Adaptivity: Dual-prompt in federated and domain generalization settings allows strong zero/few-shot performance for new clients and domains with negligible loss relative to full retraining (Chang et al., 23 Apr 2025, Zheng et al., 21 Oct 2025).
• Parameter Efficiency: Dual-prompt learning enables adaptation with minimal overhead compared to end-to-end finetuning (e.g., <10M trainable params in DPLNet (Dong et al., 2023) vs 120M+ in full dual-branch).

5. Implementation Guidelines and Best Practices

Across studies, several best practices and constraints are identified for successful deployment of dual-prompt frameworks:

• Component orthogonality: Select prompt perturbations or pools that target non-overlapping components to ensure interpretability and reduce confounding.
• Semantic preservation: Manual or automated checks (e.g., 95% meaning preservation in LLM-based paraphrasing) safeguard against invalid prompt variants.
• Prompt pool expansion and freezing: Maintain a frozen shared/global pool for stability, expand only with task-specific contributions after training.
• Human validation: For nuanced changes (especially LLM-generated or paraphrased prompts), human verification is used to confirm qualitative fidelity.
• Efficiency: Lean prompt-specific modules (e.g., lightweight MLPs, adapters, or a few vectors) reduce resource requirements versus full backbone adaptation.

6. Theoretical and Practical Significance

The dual-prompt paradigm addresses two key failure modes pervasive in prompt-learning systems: lack of robustness to prompt changes and prompt drift/coupling in continual learning. By separating or contrasting prompt functions, it allows for:

• Direct measurement of model sensitivity and agreement under distinct prompt regimes.
• Stability and plasticity in lifelong learning, enforcing clear boundaries between shared memory and new-task adaptation.
• Operationally tractable strategies for multi-domain, federated, and data-scarce scenarios, leveraging the modularity of prompt representations.

Limitations include increased engineering complexity (in maintaining separate pools or validation sets), potential privacy leakage from prompt sharing in federated contexts, and modest increases in memory for pool storage (Habba et al., 20 Jul 2025, Chang et al., 23 Apr 2025, Zhang et al., 2 Mar 2026).

7. Notable Dual-Prompt Frameworks

A non-exhaustive survey of influential frameworks:

• PromptSuite: Task-agnostic multi-prompt evaluation toolkit, dual-prompt via controlled, orthogonal perturbations (Habba et al., 20 Jul 2025).
• P3 (Prompts Promote Prompting): Joint system/user prompt optimization for LLM instruction-following (Zhang et al., 21 Jul 2025).
• PDP (Prototype-guided Dual-pool Prompting): Decoupled shared/private prompt pools in incremental object detection (Zhang et al., 2 Mar 2026).
• DP2FL and FedDEAP: Federated dual-prompt learning for generalization and seamless client integration in large foundation models (Chang et al., 23 Apr 2025, Zheng et al., 21 Oct 2025).
• DPC, DUDE, DCAR: Base/New, domain/class, or attribute/category dual-prompt schemes for fine-grained vision-language tasks (Li et al., 17 Mar 2025, Nguyen et al., 2024, Wang et al., 6 Aug 2025).
• DPLNet, DPSeg, DP-GPT4MTS: Dual-prompt for multimodal segmentation, cost volume learning, and time series forecasting, respectively (Dong et al., 2023, Zhao et al., 16 May 2025, Liu et al., 6 Aug 2025).

---

Research on dual-prompt frameworks has established their value across evaluation, optimization, and generalization tasks in LLMs and vision-LLMs. Their principled decomposition of prompt space yields substantially more robust, interpretable, and controllable systems than single-prompt baselines, and ongoing work continues to explore their application to increasingly complex, federated, and multi-domain settings (Habba et al., 20 Jul 2025, Zhang et al., 21 Jul 2025, Zhang et al., 2 Mar 2026, Chang et al., 23 Apr 2025, Liu et al., 6 Aug 2025).