Mixture-of-Prompts Router Explained

• Mixture-of-Prompts Router is a prompt-space mechanism that replaces a single global prompt with multiple specialized prompt experts based on input characteristics.
• It employs both soft and hard routing strategies—such as softmax gating, nearest-centroid matching, and top-K selection—to dynamically allocate prompts.
• Empirical studies demonstrate that using a mixture approach improves metrics like accuracy, F1 scores, and robustness across diverse applications.

A Mixture-of-Prompts Router is a routing mechanism that replaces a single global prompt with a set of prompt experts and an input-conditioned selection or mixing rule. In the literature, the “experts” may be soft prompts, hard prompt templates, principle-based prompts, prompt tensors in diffusion models, or even prompt-conditioned quality estimators over model pools; the router may be a softmax gate, a nearest-centroid matcher, a top-$K$ selector, or a regression-based decision rule. Across these instantiations, the common objective is to match heterogeneous inputs to prompt structures that better fit their semantic region, attack pattern, timestep, domain, or downstream cost–quality trade-off (Zhao et al., 23 May 2025, Petridis et al., 2024, Ham et al., 2024, Zhang et al., 19 Mar 2026).

1. Formal abstraction

The central abstraction is a prompt-space analogue of mixture-of-experts. Let $\{P_1,\dots,P_K\}$ denote prompt experts and let a router compute weights or assignments from an input representation $\phi(x)$. A canonical soft form appearing in adversarial VLM tuning is

$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$

followed by feature-level aggregation

$$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$

where prompt-specific text features are mixed for class $j$ and sample $i$ (Zhao et al., 23 May 2025). A timestep-conditioned diffusion variant uses

$$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$

so that routing varies across denoising steps and block depth (Ham et al., 2024).

Hard routing also appears in several forms. In constitutional prompt routing, a test input embedding $v_{\text{test}}$ is assigned to the nearest cluster centroid: $$i^* = \arg\max_{i \in \{1,\dots,k\}} \cos(v_i, v_{\text{test}}),$$ with the corresponding expert prompt used for inference (Petridis et al., 2024). In automated prompt construction, semantic clustering yields centroids $\{P_1,\dots,P_K\}$0, and routing is performed by

$\{P_1,\dots,P_K\}$1

where each region has its own instruction and demonstrations (Wang et al., 2024). In model-selection settings, the prompt itself is routed to an expert model through a quality estimator,

$\{P_1,\dots,P_K\}$2

or, with constraints,

$\{P_1,\dots,P_K\}$3

which turns prompt routing into decision-theoretic expert selection (Zhang et al., 19 Mar 2026, Hari et al., 2023).

These forms differ in mechanics, but they share three invariants: a bank of prompt-conditioned alternatives, a prompt- or input-dependent routing policy, and a downstream aggregation or choice rule. This suggests that “Mixture-of-Prompts Router” is best understood as a family of routing schemes rather than a single architecture.

2. Prompt experts and their parameterizations

Prompt experts vary substantially across domains. In adversarial vision–language adaptation, AMPT defines a bank

$\{P_1,\dots,P_K\}$4

where each $\{P_1,\dots,P_K\}$5 is a learnable adversarial text prompt, and each class prompt is

$\{P_1,\dots,P_K\}$6

The prompt bank may use unified context (UC) or class-specific context (CSC), while the CLIP backbone remains frozen (Zhao et al., 23 May 2025).

In prompt learning for VLM classification, MoCoOp represents each expert as a soft context $\{P_1,\dots,P_K\}$7 tied to a semantic group of hard templates. For class $\{P_1,\dots,P_K\}$8,

$\{P_1,\dots,P_K\}$9

The grouping is explicit: “flowers,” “satellite,” “a person doing {},” and related manually organized template families determine expert identity and initialization (Du et al., 2024).

In MoPD, the prompt bank is asymmetric: the student is a single learnable soft prompt

$\phi(x)$0

while the teachers are multiple hand-crafted hard prompts

$\phi(x)$1

Here the router does not mix student prompts; it selects hard prompts used for distillation, making the prompt bank a teacher ensemble rather than a direct inference-time expert set (Chen et al., 2024).

In ConstitutionalExperts, each expert is not a vector prompt but a set of explicit natural-language principles

$\phi(x)$2

inserted into a classification template. Each cluster-specific expert evolves through add, delete, and revise operations on individual principles, preserving interpretability (Petridis et al., 2024). In “One Prompt is not Enough,” each expert is a pair

$\phi(x)$3

combining an instruction $\phi(x)$4 and a region-specific demonstration set $\phi(x)$5 (Wang et al., 2024).

In diffusion model patching, the experts are prompt tensors added to latent token inputs: $\phi(x)$6 and the mixture is performed across sub-prompts for each block and timestep (Ham et al., 2024). In pathology restoration, the prompt set is heterogeneous rather than homogeneous: defocus prompt $\phi(x)$7, pathology prompt $\phi(x)$8, and edge prompt $\phi(x)$9, derived from distinct sources and routed into different stages of a restoration pipeline (Cai et al., 16 Mar 2025).

The literature therefore does not constrain a prompt expert to be textual or continuous. A prompt expert may be a learnable context sequence, a discrete instruction-plus-demo package, a constitutional rule set, or a learned tensor in latent space.

3. Router mechanisms and optimization

Routers differ in both conditioning signal and training objective. In AMPT, the router receives adversarial image features

$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$0

and outputs

$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$1

where $$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$2 is a two-layer MLP and $$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$3 controls sharpness. The same image-dependent but class-agnostic mixture weights are used for all class prompts of a sample (Zhao et al., 23 May 2025).

MoCoOp also routes from image features, but constrains routing with a hard-prompt-derived target distribution. Given group text features $$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$4, it forms

$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$5

and trains the router by

$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$6

This makes routing a distillation problem from hand-crafted template groups to a learned gate (Du et al., 2024).

MoPD uses a related but distinct design. Its gating network takes image feature $$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$7, applies a linear map $$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$8, then a sparse top-$$\mathbf{w}(x)=\mathrm{softmax}(g(\phi(x))) \in \mathbb{R}^K,$$9 mask and softmax: $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$0 The router is trained jointly with a student soft prompt by a weighted distillation loss

$$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$1

and a prompt-selection loss

$$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$2

which explicitly encourages the gate to favor useful teacher prompts (Chen et al., 2024).

Diffusion Model Patching routes on timestep and block depth, optionally with conditioning $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$3: $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$4 and combines this with importance and load-balancing losses to prevent prompt collapse (Ham et al., 2024). By contrast, ConstitutionalExperts uses no learned router at all: clustering precedes expert training, and the nearest-centroid rule defines routing with no explicit routing loss (Petridis et al., 2024). FineRouter sits between these extremes. It first assigns prompts to latent tasks via a classifier, then uses task-specific prediction heads to estimate model quality, and finally combines prompt-level and task-level scores through

$$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$5

with $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$6 in experiments (Zhang et al., 19 Mar 2026).

The training signal is therefore not uniform across the field. Some routers are optimized by downstream classification or reconstruction loss alone, some by distillation from hard prompts, some by bandit-style selection on validation F1, and some by direct quality regression over expert models.

4. Representative instantiations across domains

The concept appears in at least four distinct research programs.

First, in vision–language robustness and adaptation, the router mixes text prompts or prompt-derived features conditioned on images. AMPT uses a bank of adversarial text prompts and sample-specific mixture weights to align text features with adversarial image features (Zhao et al., 23 May 2025). MoCoOp uses multiple soft prompts with top-$$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$7 selection and semantically grouped supervision from hard templates (Du et al., 2024). MoPD uses a gating network over hard prompts only during training, distilling a selected hard-prompt mixture into a single student prompt (Chen et al., 2024).

Second, in language-model prompt optimization, the router partitions semantic space and dispatches inputs to region-specific discrete prompts. ConstitutionalExperts clusters training data in embedding space, evolves one principle-based prompt per cluster, and routes test inputs by nearest centroid (Petridis et al., 2024). “One Prompt is not Enough” similarly partitions the problem space into semantic regions using K-means in embedding space and assigns each query to the closest prompt expert combining instruction and demonstrations (Wang et al., 2024). “Sweeping Heterogeneity with Smart MoPs” instead uses a small MLP gate on an internal LLM representation to produce a softmax vector $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$8 over prompt tokens or prompt groups, thereby reweighting attention to prompt experts in later layers (Dun et al., 2023).

Third, in diffusion and image restoration, prompts become latent-space or modality-specific conditioning modules. Diffusion Model Patching adds prompt tensors to DiT block inputs and routes among them at each timestep, effectively making denoising stage-specific (Ham et al., 2024). Pathology restoration via Mixture of Prompts constructs $$z_t^{j,i} = \sum_{k=1}^{K} \tilde{w}_k^i \, z_{t,k}^{j},$$9, $j$0, and $j$1, then uses prompt-conditioned MoE gating in a Restormer stage and cross-attention or concatenative conditioning in a diffusion stage (Cai et al., 16 Mar 2025).

Fourth, in model selection and scalable routing, the prompt itself is the routing object. Tryage uses a small LM router to predict per-model loss for a prompt and chooses the model minimizing predicted loss plus user-weighted constraints (Hari et al., 2023). FineRouter discovers 332 latent task types, uses a task classifier as the first-stage gate, and activates task-specific quality estimators in a second-stage MoE over frontier models (Zhang et al., 19 Mar 2026).

These instantiations differ in whether prompts are the experts, the conditioning signals, or the objects being routed. A plausible implication is that “Mixture-of-Prompts Router” spans both prompt-conditioned computation and prompt-conditioned expert selection.

5. Empirical behavior and performance

The principal empirical claim recurring across the literature is that a single prompt is often too rigid for heterogeneous data. In AMPT, for equal total parameters, using multiple shorter prompts outperformed one longer prompt across five datasets: $j$2 vs $j$3 yielded adversarial robustness $j$4 on average and clean accuracy $j$5, while $j$6 vs $j$7 yielded adversarial robustness $j$8 and clean accuracy $j$9 (Zhao et al., 23 May 2025). On Caltech101 16-shot CSC, the ablation isolating mixture and router showed PGD $i$0 for baseline, mixture without router, and mixture plus router, respectively (Zhao et al., 23 May 2025).

In constitutional prompt routing, CE + MoE improved F1 over single-prompt CE on every reported average, with $i$1 F1 for ConstitutionalExperts, $i$2 for ProTeGi, and $i$3 for PromptBreeder when MoE routing was added (Petridis et al., 2024). The same paper reports that ConstitutionalExperts outperforms best prompt optimization baselines by $i$4 F1 on average $i$5 (Petridis et al., 2024). In “One Prompt is not Enough,” the resulting MoP obtains an average performance of $i$6 versus $i$7 for APE+Demos and $i$8 for InstructZero+Demos, with an average pairwise win rate of $i$9 (Wang et al., 2024).

Prompt-space routing also shows gains in more resource-constrained settings. In federated and compressed LLM adaptation, Smart MoPs reduces final perplexity by $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$0 up to $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$1 in the federated scenario and from $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$2 up to $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$3 in the centralized scenario, depending on pruning and task heterogeneity (Dun et al., 2023). In model routing, FineRouter reports macro average $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$4 and quality $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$5, surpassing the strongest individual model, Claude-Sonnet-4.5, at macro average $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$6 and quality $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$7, while achieving better performance than Claude-Sonnet-4.5 at less than half its API cost (Zhang et al., 19 Mar 2026). Tryage, in a masked-language-model setting over heterogeneous domains, identifies the optimal model with an accuracy of $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$8, compared to $$\alpha^{(i)}(t)=\mathrm{softmax}(W^{(i)} z_t^{(i)}), \qquad \tilde{P}^{(i)}_t = \sum_{k=1}^{K} \alpha^{(i)}_k(t) P^{(i)}_k,$$9 by GPT 3.5 Turbo and $v_{\text{test}}$0 by Gorilla (Hari et al., 2023).

In generative image modeling, the router appears to matter as much as the prompt bank itself. Diffusion Model Patching improves the FID of converged DiT-L/2 on FFHQ from $v_{\text{test}}$1 to $v_{\text{test}}$2, a $v_{\text{test}}$3 improvement, with only a $v_{\text{test}}$4 parameter increase and $v_{\text{test}}$5 additional training iterations (Ham et al., 2024). In VLM prompt learning, MoPD raises average base-to-new harmonic mean from $v_{\text{test}}$6 for CoOp to $v_{\text{test}}$7, with New accuracy $v_{\text{test}}$8 (Chen et al., 2024), while MoCoOp reports average harmonic mean $v_{\text{test}}$9 in base-to-new evaluation across 11 datasets (Du et al., 2024).

These findings suggest that prompt multiplicity alone is often beneficial, but input-conditioned routing supplies the additional gains when the input distribution is heterogeneous, adversarially perturbed, stage-dependent, or cost-constrained.

6. Mechanistic analysis, limitations, and controversies

Several papers argue that routing quality cannot be inferred solely from aggregate task metrics. In sparse MoE analysis, “When Are Experts Misrouted?” shows that the standard router is well-aligned with route utility on confident tokens but uninformative on fragile tokens, where lower-loss equal-compute routes consistently exist inside the frozen model but are not selected; a minimal router-only update to the final-layer router shifts pass@K on AIME 2024+2025 and HMMT 2025 for both Qwen3-30B-A3B and GPT-OSS-20B (Yoon et al., 8 May 2026). Although this paper studies token-level MoE rather than prompt-space routing, it suggests that prompt routers may also suffer from a counterfactual blind spot if training only scores the executed route.

RIDE tests routing-style meta prompts as a textual proxy for routing and rejects a simple sparsity-based interpretation. Across Qwen3-8B, Llama-3.1-8B-Instruct, and Mistral-7B-Instruct-v0.2, meta prompts consistently densify early/middle-layer representations rather than increasing sparsity; natural-language expert instructions are often stronger than structured tags; and the densification–stability link is weak, appearing only in Qwen with near-zero correlations in Llama and Mistral (Zhang et al., 31 Mar 2026). This directly challenges the assumption that “expert routing” necessarily yields sparser or more certain internal computation.

Safety and interpretability work points to similar complexity. In Mixtral 8x7B-Instruct, benign and harmful prompt groups remain close under both activation-based and gradient-based routing signals, most experts are shared across prompt types, and safety-relevant routing is subtle, depth-dependent, and distributed rather than dominated by a fixed set of experts (Siddiky, 22 May 2026). A plausible implication is that prompt routing for safety should not rely on a single “safe prompt” or a single expert, but on distributed late-layer or multi-expert patterns.

The same caution applies to optimization heuristics. “Routers Learn the Geometry of Their Experts” proves that, for a token routed to expert $$i^* = \arg\max_{i \in \{1,\dots,k\}} \cos(v_i, v_{\text{test}}),$$0, the router row $$i^* = \arg\max_{i \in \{1,\dots,k\}} \cos(v_i, v_{\text{test}}),$$1 and expert input-side weights receive gradients along the same input direction $$i^* = \arg\max_{i \in \{1,\dots,k\}} \cos(v_i, v_{\text{test}}),$$2, differing only in scalar coefficients. The paper further shows that auxiliary load balancing spreads input-directed gradients across router weights, making distinct router directions nearly three times more similar to each other (Ahrac et al., 12 May 2026). This suggests that load balancing in a Mixture-of-Prompts Router may trade off prompt specialization against balanced utilization unless the balancing signal is designed carefully.

Security is an additional concern. “Stealing User Prompts from Mixture of Experts” demonstrates that Expert-Choice-Routing can disclose a victim’s prompt through a batching side channel, extracting the entire prompt with $$i^* = \arg\max_{i \in \{1,\dots,k\}} \cos(v_i, v_{\text{test}}),$$3 queries or 100 queries on average per token in the setting considered (Yona et al., 2024). The paper concerns token-level routing rather than prompt-level routers, but it establishes that routing coupled across users can itself become an attack surface. This suggests that prompt-level routers should avoid cross-user capacity competition and deterministic tie handling when prompts from different users share a routing pool.

Open directions already appear explicitly in the surveyed work: richer but still stable routers for prompt banks (Ran et al., 31 Aug 2025), automatic discovery of fine-grained prompt/task regions rather than manual taxonomies (Zhang et al., 19 Mar 2026), domain-robust router features and invariant training objectives (Jiang et al., 12 Feb 2026), and counterfactual or intervention-based evaluation of routing decisions rather than reliance on end-task averages alone (Yoon et al., 8 May 2026). Across these directions, the topic is moving from prompt engineering toward a more general theory of routed prompt computation.