Multi-Criteria Prompting

• Multi-Criteria Prompting is a method that designs prompt strategies to balance multiple, often conflicting, objectives in neural and language models.
• Architectural innovations like Switch-LSTM and latent representations via VAEs enable dynamic, criterion-specific processing for tasks such as recommendation and reasoning.
• Optimization and curriculum design techniques enhance prompt construction and evaluation, driving improved robustness, transferability, and interpretability across diverse applications.

Multi-criteria prompting refers to the design and application of prompt strategies—particularly in neural and LLM systems—that explicitly account for multiple, often conflicting, criteria or objectives. These criteria can include segmentation standards, compositional attributes, evaluation dimensions, or even task-specific difficulty signals. The central goal is to ensure that models can adaptively integrate and balance multiple sources of supervision or guidance, thereby improving robustness, transferability, and interpretability across diverse settings, including word segmentation, recommendation systems, complex reasoning, and information retrieval.

1. Architectural Foundations: Multi-Route and Sub-Criteria Models

Early solutions to multi-criteria prompting focus on decomposing complex tasks into modular sub-criteria and providing dynamic routing mechanisms for controlling model behavior. The Switch-LSTM architecture (Gong et al., 2018) exemplifies this, replacing monolithic LSTM cells with a group of K independent LSTM units, each representing a latent “sub-criterion.” A switcher module leverages task embeddings to compute routing probabilities $$a_{t,k} = \text{softmax}(W[\mathbf{e}(x_t), \mathbf{s}_{t,k}, \mathbf{e}_m])$$ for each time step. The final hidden state is a weighted sum $h_t = \sum_k a_{t,k} s_{t,k}$, flexibly blending shared and criterion-specific knowledge.

This mechanism generalizes to various tasks beyond segmentation, facilitating knowledge transfer (by adapting only task embeddings for new criteria) and enabling dynamic multi-criteria supervision in prompting, classification, and generation scenarios.

2. Signal Extraction: Latent and Explicit Multi-Criteria Representations

The effectiveness of multi-criteria prompting depends on extracting and modeling rich, multi-dimensional signals. Latent multi-criteria ratings (Li et al., 2019) employ variational autoencoders (VAEs) to encode user reviews into high-dimensional continuous embeddings, which are then compressed (via Gumbel-Softmax reparameterization) into discrete, low-dimensional vectors $D_w$ corresponding to criteria. Such embeddings capture nuanced semantic relations that go beyond explicit ratings, enabling better recommendation performance and generalizing to multi-criteria decision tasks.

In recommender system design, criteria-specific representations can be formalized as

$$\hat{r}_{u,i}^{(k)} = \mu + b_u^{(k)} + b_i^{(k)} + \mathbf{p}_u^\top \mathbf{q}_i^{(k)},$$

allowing models to aggregate user feedback across multiple criteria (Zheng, 22 Nov 2024).

Graph-based approaches such as CPA-LGC (Park et al., 2023) expand the user–item graph to encode each item-criterion pair, generating user-specific criteria-preference and item-specific criterion embeddings, and propagating these via light graph convolutions.

3. Aggregation and Decision Functions: Prioritization, Ranking, and Outranking

Effective multi-criteria prompting further demands principled aggregation, prioritization, and ranking strategies. In pathfinding, prioritized multi-criteria reduction (Dinitz et al., 2021) encodes each criterion into a distinct bit segment within composite weights, ensuring lexicographic ordering. Shortest paths are computed over the ensemble weight, automatically optimizing for higher-priority criteria.

Multi-criteria sorting problems are elegantly addressed by Choquet integral-based formulations (Pelissari et al., 2019), which aggregate preferences in a non-additive fashion to handle synergy and redundancy:

$$CI(a) = \sum_{T \subseteq N} m(T) \cdot \min_{j \in T} p_j(a),$$

where $m(T)$ encodes interaction and $p_j(a)$ is the normalized preference.

Hybrid multi-criteria ranking (Zheng et al., 2023) uses a relaxed Pareto (k-dominance) integer score as major sorting, refined by normalized subsorting (Average Ranking, Maximum Ranking, Global Detriment, or Profit Gain). The final score is $HybridScore = Score_{major} + Score_{sub}$, yielding fine-grained, stable rankings in top-N recommendation scenarios.

4. Automated Prompt Construction and Modular Evaluation

Recent advances push for systematic prompt construction, modular evaluation, and query-dependent adaptation. PromptSuite (Habba et al., 20 Jul 2025) is a task-agnostic framework that decomposes prompts into independent components (instruction, format, demonstration, instance) and applies controlled perturbations to each, enabling robust multi-prompt evaluation. Experiments across diverse tasks reveal substantial performance fluctuations with prompt variation, underscoring the necessity of multi-criteria evaluation.

In information retrieval, criteria-based judgment frameworks (Farzi et al., 13 Jul 2025) decompose relevance into exactness, topicality, coverage, and contextual fit, each judged independently via LLM prompts. Scores are aggregated using mapping rules (e.g., sumdecompose via LaTeX piecewise function) for robust, interpretable evaluation, outperforming direct grading on system ranking/leaderboard correlation.

5. Optimization Strategies: Holistic and Dynamic Multi-Component Prompting

Optimization techniques for multi-criteria prompting increasingly employ iterative, holistic, and query-dependent strategies. P3 (Zhang et al., 21 Jul 2025) jointly refines both system and user prompts in an offline-onlinedual optimization process, leveraging hard samples to update system prompts and query-dependent adaptation for user prompts. By optimizing all components together, models achieve superior performance across reasoning and general question-answering tasks; improvements are realized both in direct LLM output quality and in reduced resource usage with in-context learning variants.

AMuLaP (Wang et al., 2022) automates selection of multi-label mappings using statistics-driven “voting” over vocabulary distributions, with deduplication mechanisms to enforce robustness.

6. Reasoning and Curriculum Design: Multi-Agent, Multi-Path, and Balanced Difficulty

Multi-step reasoning and curriculum design approaches reflect another dimension of multi-criteria prompting. Complexity-based prompting (Fu et al., 2022) prioritizes chains with higher reasoning complexity (longer reasoning step count), consistently improving multi-step reasoning accuracy over handcrafted or random prompts.

CoMM (Chen et al., 26 Apr 2024) generalizes this to collaborative multi-agent setup, where agents are prompted to adopt distinct expert roles and reasoning paths (e.g., physicist vs. mathematician), cross-validating outputs before aggregation. Empirical studies confirm that independent role-specific few-shot prompting ensures better performance and error robustness relative to single-model, single-path approaches.

Curriculum design rooted in “tailored teaching with balanced difficulty” (Yang et al., 26 Aug 2025) employs dual difficulty signals—model-perceived (prediction disagreement) and intrinsic sample complexity—to sample prompt examples for training. Sampling probabilities follow

$$P(d_m, d_s) = \frac{1}{Z} \exp(-\alpha d_m - \beta d_s)$$

ensuring a balance in exposure across complexity dimensions. This methodology substantiates improved performance and stability in multimodal chain-of-thought reasoning.

7. Practical Applications and Implications Across Domains

Multi-criteria prompting techniques find utility in recommendation systems (through latent or explicit multi-criteria ratings and graph convolutional aggregation), flexible job shop scheduling (integrating MCDM methods such as TOPSIS, EDAS, CP, PROMETHEE within simulation models (Thenarasu et al., 2023)), retrieval-augmented generation (with frameworks like REBEL fusing multi-criteria reranking and chain-of-thought (LeVine et al., 14 Mar 2025)), and vision-LLMs (e.g., PMPO integrating multiple prompt branches at partitioned encoder depths (Tian et al., 2023)).

The incorporation of multi-criteria at inference time, dynamic adaptation based on query intent, explicit modularity in prompt engineering, and curriculum-aware training design collectively strengthen robustness, accuracy, and interpretability across diverse machine learning domains.

8. Prospects and Open Challenges

The surveyed literature underlines several open problems for multi-criteria prompting: determining optimal decomposition and aggregation strategies, disentangling overlapping criteria, scaling to large heterogeneous problem spaces, quantifying curriculum signals, and reconciling the tradeoff between robustness and computational efficiency. Future research avenues include more sophisticated integration of latent criteria, dynamic graph representations for criteria dependencies, query-conditioned reranking in retrieval systems, and end-to-end joint optimization of prompt components.

In sum, multi-criteria prompting represents a paradigm shift in guiding model behavior under heterogeneous supervision, with architectural, algorithmic, optimization, and evaluation innovations driving advancements across NLP, IR, recommender systems, reasoning, scheduling, and multimodal learning.