---
title: Max–Min Similarity Prompt Sampling
url: https://www.emergentmind.com/topics/max-min-similarity-prompt-sampling
type: topic
---

# Max–Min Similarity Prompt Sampling

Searching arXiv for recent papers related to max–min similarity prompt sampling and its adjacent formulations.
Max–Min Similarity Prompt Sampling is a prompt construction and selection paradigm in which similarity and diversity are jointly managed when forming prompts or prompt sets for large language models. In the most direct few-shot formulation, it can be understood as a query-aware extension of In-Context Sampling (ICS): rather than using a single in-context learning prompt, one constructs multiple prompts for the same query, seeks examples that are similar to the query, avoids over-concentrating highly similar examples within the same prompt, and aggregates predictions across prompts by majority vote [2311.09782]. Across adjacent literatures, the same design intuition appears under several technical forms: rank-based diversity and hybrid sampling in ICS [2311.09782], worst-group or reweighted active sampling under min-max objectives [2006.06879], bilevel similarity minimization and maximization [2307.08779], graph-based prompt selection via Maximum Independent Set on similarity graphs [2606.01400], metric-fair joint prompting of similar items [2512.07608], discriminative min–max similarity/dissimilarity assignment [1901.10760], and distribution-level textual regularization over prompt sets [2502.14376].

## 1. Conceptual definition and problem setting

In standard In-Context Learning (ICL), a single prompt is formed as
\[
\{I + (x^{\text{icl}_1}, y^{\text{icl}_1}) + \dots + (x^{\text{icl}_\ell}, y^{\text{icl}_\ell}) + x\},
\]
where \(I\) is a natural language task instruction, \((x^{\text{icl}_i}, y^{\text{icl}_i})\) are labeled in-context examples, and \(x\) is the query instance [2311.09782]. In-Context Sampling generalizes this by sampling a candidate pool \(C = \{(x_i, y_i)\}_{i=1}^n\), constructing \(k\) distinct prompts for the same query from subsets \(S_j \subset C\) of size \(\ell\), querying the model once per prompt, and aggregating predictions using majority vote:
\[
\hat{y}_{\text{final}} = \operatorname{mode}(\hat{y}_1, \dots, \hat{y}_k).
\]
This committee-based design treats prompts as committee members and uses voting to increase reliability and confidence [2311.09782].

Within that setting, a Max–Min Similarity Prompt Sampling view introduces two simultaneous desiderata. First, selected demonstrations should be similar to the query or representative of the data distribution. Second, selected demonstrations should not collapse into a narrow cluster. The data block states this explicitly as the desire to “maximize similarity between query \(x\) and selected in-context examples” and to “maximize diversity among selected examples (or equivalently, maximize the minimum pairwise distance/similarity margin)” [2311.09782]. The same two-term structure also appears in graph-based benchmark reduction, where the goal is to select prompts that are “maximally non-redundant / diverse” while preserving ranking behavior [2606.01400].

A plausible implication is that “Max–Min Similarity Prompt Sampling” is less a single algorithm than a family of prompt-selection procedures in which one term enforces relevance or representativeness and the other enforces separation, coverage, or robustness. The data block repeatedly frames this trade-off as central rather than incidental [2311.09782] [2606.01400].

## 2. In-Context Sampling as the immediate prompting formulation

The clearest prompting instantiation comes from ICS. For a fixed query \(x\), ICS samples a candidate pool, constructs multiple prompts with \(\ell = 3\) demonstrations in the paper, obtains \(k\) predictions, and returns the majority label [2311.09782]. The paper evaluates three data similarity-based strategies for the candidate-sampling stage and, optionally, the prompt-construction stage: Diversity, Similarity, and Hybrid [2311.09782].

Similarity is computed in embedding space with a sentence-transformer encoder (Sentence-BERT; Reimers & Gurevych, 2019) using cosine similarity:
\[
\cos(u,v)=\frac{\langle u,v\rangle}{\|u\|\|v\|}.
\]
For each datum \(x_i\) in a pool \(D\), average similarity to the dataset is defined as
\[
s(x_i,D)=\cos\left(\text{embed}(x_i), \sum_{j=1}^{|D|}\text{embed}(x_j)\right).
\]
This is described as similarity to the mean embedding of the dataset [2311.09782].

The Diversity strategy computes these scores, sorts them in descending order, sets \(t=\left\lfloor\frac{|D|}{n}\right\rfloor\), and selects evenly spaced points along the ranking:
\[
R_{\text{div}}=\{x_{S_i}\mid i=1, 1+t, 1+2t, \dots\}.
\]
The paper states that this approximates sampling across the whole similarity spectrum and gives a “Conceptual connection to max–min diversity,” while noting that it is not an explicit solution of
\[
\max_{S:|S|=n}\min_{i\neq j\in S} d(x_i,x_j).
\]
Instead, it is described as a cheap heuristic to increase pairwise diversity by spreading selected points out in embedding space [2311.09782].

The Similarity strategy takes the top \(n\) points with highest average similarity to the dataset mean:
\[
R_{\text{sim}}=\{x_{S_1},\dots,x_{S_n}\}.
\]
The paper says this is closer to a max-mean similarity objective than to max–min, but that it can be combined with diversity-based selection [2311.09782]. The Hybrid strategy selects \(n/2\) points via Diversity, removes them from the ranked list, then selects \(n/2\) highest-ranked remaining examples, yielding
\[
R_{\text{hyb}} = R_{\text{div}} \cup R_{\text{sim}}.
\]
This is said to “implicitly balance” high similarity to the dataset mean and spread across the embedding spectrum [2311.09782].

These strategies are global rather than query-specific. The data block therefore proposes a query-aware extension by replacing the global score \(s(x_i,D)\) with
\[
s_q(x_i)=\cos(\text{embed}(x_i), \text{embed}(x)).
\]
This suggests a direct Max–Min Similarity Prompt Sampling formulation in which examples are first filtered or ranked by query similarity and then diversified within the prompt [2311.09782].

## 3. Similarity, diversity, and explicit max–min formulations

The data block distinguishes carefully between heuristic diversity and explicit max–min optimization. For query relevance, one can rank examples by \(s_q(x_i)\). For diversity, the stated explicit objective is
\[
\max_{S \subset D, |S|=\ell}\; \min_{\substack{i,j\in S \\ i\neq j}} d(x_i,x_j),
\]
where \(d\) can be cosine distance \(d(u,v)=1-\cos(u,v)\) [2311.09782]. A common greedy max–min algorithm is then described: pick an initial point, for example the most similar to the query, and iteratively add the point that maximizes its minimum distance to the already selected set,
\[
x_{t+1} = \arg\max_{x \in D \setminus S_t} \min_{x' \in S_t} d(x,x').
\]
The paper does not implement this explicit formula, but states that its Diversity and Hybrid strategies are cheap approximations to the same intuition and that “too much similarity inside the same prompt is harmful” [2311.09782].

This trade-off is also visible in the paper’s guidance for a two-level design. First, candidate pool selection should find globally representative examples or examples similar to the query. Second, prompt construction should ensure that all chosen examples are reasonably similar to the query while maximizing pairwise distance among them [2311.09782]. The data block gives a concrete greedy scheme: filter examples by a similarity threshold \(\tau\), pick the most query-similar example first, then repeatedly select the point that maximizes the minimum cosine distance to the current prompt set [2311.09782].

A related but not identical structure appears in “Similarity Min-Max: Zero-Shot Day-Night Domain Adaptation,” where a transformation \(D\) minimizes feature similarity while a feature extractor \(F\) maximizes it:
\[
\mathop{\max}_{\theta_F}\mathop{\min}_{\theta_D} \operatorname{Sim}(F(I),F(D(I))).
\]
There, cosine similarity is minimized at the transformation stage and maximized via BYOL-style loss at the model stage [2307.08779]. This is not prompt sampling, but it provides a concrete “similarity min–max” paradigm with a min step that enlarges a gap and a max step that learns invariance to that gap [2307.08779]. A plausible implication is that prompt-level variants can separate a hard-example generation stage from a robustness-alignment stage, rather than solving both within a single combinatorial sampler.

## 4. Algorithmic realizations across prompt selection and grouping

Several distinct algorithmic realizations of max–min-style prompt selection are represented in the data block.

The first is ICS-style few-shot prompt construction. The paper evaluates candidate pool sizes \(n \in \{50,100,250,500\}\), numbers of prompts \(k \in \{3,5,10,20\}\), and fixed \(\ell=3\) examples per prompt [2311.09782]. For a given query, the implementation samples a candidate set using Random, Diversity, Similarity, or Hybrid; for each of \(k\) prompts it samples 3 demonstrations from the remaining pool; and used demonstrations are removed so prompts are disjoint in that implementation [2311.09782].

The second is graph-based prompt selection for benchmark reduction. “Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs” models a benchmark as a similarity graph and selects a subset via Maximum Independent Set (MIS) [2606.01400]. Each prompt \(p_i\) is embedded as \(e_i \in \mathbb{R}^d\); distances considered are cosine distance, Pearson correlation distance, and standardized Euclidean distance on L2-normalized embeddings [2606.01400]. The graph is built from percentile thresholds \(p_{10}, p_{20}, p_{50}, p_{80}, p_{90}, p_{95}\) derived from 10,000 random prompt pairs [2606.01400]. The paper notes a wording mismatch in one description of edge construction, but states that conceptually, for max-min diversity, edges should encode prompt conflict under similarity, so that an independent set forbids selecting two prompts that are too similar [2606.01400]. In that conceptualization, MIS becomes a hard-threshold max–min sampler: enforce a minimum separation and maximize the number of selected prompts.

The third is metric-fair grouping of similar items. “Metric-Fair Prompting: Treating Similar Samples Similarly” embeds question stems with the Qwen3-4B embedding model, computes cosine similarity, and pairs each question with its nearest neighbor, then solves both jointly under instructions that impose a “Lipschitz-like constraint” so that similar inputs yield similar scores [2512.07608]. The prompt instructs the model to think in terms of a margin-based classifier \(f:\mathcal{X}\to\mathbb{R}\), with prediction \(y=\mathbf{1}\{f(x)>0\}\), and to prefer cross-item consistency unless a clear clinical conflict exists [2512.07608]. This is a max-similarity grouping strategy rather than a max–min diversity strategy, but the same paper explicitly suggests future directions such as “cluster-then-cover” and “active pairing,” which the data block interprets as natural contexts for max–min sampling [2512.07608].

The fourth is assignment by discriminative min–max score in a learned transform space. “Clustering with Jointly Learned Nonlinear Transforms Over Discriminating Min-Max Similarity/Dissimilarity Assignment” defines a parametric discriminative prior
\[
f_c({\bf y}_i,\boldsymbol{\theta}) =
\min_{c_1\in\mathcal{C}_d}
\left[
\frac{\varrho({\bf y}_i,\boldsymbol{\tau}_{c_1})}
{\max_{c_2\in\mathcal{C}_s}\varrho({\bf y}_i,\boldsymbol{\nu}_{c_2})}
+
\varsigma({\bf y}_i,\boldsymbol{\tau}_{c_1})
\right],
\]
where \(\varrho\) is a sign-aware similarity score and \(\varsigma\) is a support-intersection energy [1901.10760]. Although this work addresses clustering rather than prompting, it provides a fully explicit min–max similarity/dissimilarity assignment rule. The data block then interprets this as transferable to prompt selection by comparing prompt-conditioned embeddings against good and bad prototypes and choosing the prompt with minimum discriminative score [1901.10760].

## 5. Empirical behavior and operating regimes

The central empirical claim from ICS is that multiple prompts plus voting outperform single-prompt ICL, and that similarity-based candidate sampling improves over fully random ICS [2311.09782]. On Mistral-7B with \(n=100\) and \(k=10\), the baseline single-prompt ICL accuracy is reported as 64.74 on e-SNLI, 56.90 on Multi-NLI, 40.42 on ANLI, and 78.83 on Contract-NLI [2311.09782]. Random ICS increases these to 72.57, 61.17, 42.22, and 86.69, respectively [2311.09782]. Best similarity-based ICS configurations are reported as approximately 74.0–74.1 on e-SNLI for Similarity/Random or Hybrid/Random, approximately 62.41 on Multi-NLI for Hybrid/Random, about 42.6 on ANLI for Diversity-based or Hybrid-based strategies, and approximately 90.48 on Contract-NLI for Similarity/Random [2311.09782]. The stated pattern is that any non-random similarity-based strategy in at least one ICS step beats fully random ICS, and that candidate-sampling strategy matters more than per-prompt sampling strategy [2311.09782].

The same paper also reports clear operating heuristics. Increasing \(n\) beyond 100 gives diminishing returns, accuracy gains plateau, increasing \(k\) from 3 to 10 helps, and 10 to 20 yields marginal improvements with extra cost; thus a moderate candidate set \(n \approx 100\) and moderate number of prompts \(k \approx 10\) form a good balance [2311.09782]. The paper caps generation to 10 tokens and does not deeply analyze context length [2311.09782].

The graph-based MIS work reports a different but complementary operating regime. Across 2563 completed configurations, mean Spearman \(\rho\) is approximately 0.967–0.968 depending on algorithm, about 84% of configurations have \(\rho \ge 0.95\), and stochastic solvers yield mean Kendall’s \(W \approx 0.997\), with \(W \ge 0.90\) in 99.2% of configurations [2606.01400]. Threshold percentile controls the trade-off. At \(p_{10}\), mean \(\rho = 0.905 \pm 0.10\), mean \(W = 0.990\), mean subset size is about 70 prompts, and reduction is 88–93%; at \(p_{80}\), mean \(\rho = 0.992 \pm 0.010\), mean \(W = 0.999\), mean subset size is about 429 prompts, and reduction is 37–45%; at \(p_{95}\), mean \(\rho = 0.997\), mean \(W = 0.9996\), mean subset size is about 593 prompts, and reduction is 17–23% [2606.01400]. The primary failure mode is overly dense graphs at low thresholds, especially on GPQA and IFEval, leading to tiny independent sets and poorer representativeness [2606.01400].

Metric-Fair Prompting reports a different kind of gain: on MedQA with Qwen3-14B, single-item prompting achieves 68.0 accuracy and the metric-fair two-item prompt achieves 84.0 [2512.07608]. The paper evaluates fairness qualitatively rather than via an explicit fairness metric and notes that performance is “not completely understood from a rigorous theoretical perspective” and that results are “not always stable” [2512.07608].

## 6. Theoretical lenses: robustness, fairness, and representative coverage

One theoretical lens comes from min-max fairness. “Active Sampling for Min-Max Fairness” studies
\[
\min_{\theta \in \Theta} \max_{i\in[g]} L_i(\theta),
\]
where \(L_i(\theta)\) is group-wise expected loss [2006.06879]. The paper’s key algorithmic idea is to identify the worst-off group under the current model and sample from that group for the next update, or to maintain a reweighting distribution
\[
\pi_t(i)\propto \pi_{t-1}(i)\exp(\gamma L_i^{\text{emp}}(\theta_t)).
\]
The data block explicitly maps this to prompts by replacing demographic groups with prompt groups or prompt clusters, maintaining per-group validation buffers, and sampling from the worst-performing prompt group [2006.06879]. This suggests a min-max prompt sampler that does not optimize geometric diversity directly, but instead minimizes the maximum loss across prompt regions.

A second lens is metric fairness. Metric-Fair Prompting adopts the Lipschitz-style condition
\[
D(f(x),f(x')) \le d(x,x'),
\]
typically \(|f(x)-f(x')|\le d(x,x')\), so that similar instances receive similar scores [2512.07608]. In prompting, this is implemented softly by pairing similar items and instructing the model to reason jointly, extract decisive features, and maintain cross-item consistency [2512.07608]. This suggests a local max-similarity principle: within a prompt, similarity is deliberately high so that consistency constraints become salient.

A third lens is robust alignment under worst-case perturbation. SPTR uses optimal transport between the set of hand-crafted textual features and tuned textual features, plus a similarity paradigm
\[
\mathcal{L}_{\text{SP}}=
\mathcal{KL}\Big(\operatorname{sim}(\boldsymbol{t}_{tun}, \boldsymbol{v}_{n}), \operatorname{sim}(\boldsymbol{t}_{hand}, \boldsymbol{v}_{adv})\Big),
\]
where \(\boldsymbol{v}_{adv}\) comes from PGD adversarial perturbation and \(\boldsymbol{t}_{hand}\) is the average of multiple hand-crafted prompts [2502.14376]. The data block interprets OT as a distribution-level regularizer that discourages collapse toward a single prompt and implicitly encourages the tuned prompt to stay close to the whole hand-crafted prompt distribution [2502.14376]. A plausible implication is that prompt selection can be formulated not only over pairwise distances but also over similarity distributions and their worst-case deviations.

## 7. Limitations, misconceptions, and open directions

A common misconception is that maximizing similarity everywhere should improve few-shot prompting. The ICS data directly contradict this: using Similarity strategy to make the three demonstrations inside a prompt very similar to each other can hurt performance compared with a random mix, and the beneficial effect of similarity-based methods is strongest at the candidate-sampling step rather than naive within-prompt concentration [2311.09782]. The data block therefore presents “balancing similarity and diversity, not simply maximizing similarity” as the key lesson [2311.09782].

Another misconception is that max–min prompt selection must take the form of an explicit combinatorial solver. The sources show several alternatives. Even spacing across a similarity ranking can approximate diversity without solving a max–min optimization [2311.09782]. Worst-group active sampling optimizes a min-max objective over prompt groups rather than pairwise distances [2006.06879]. Joint prompting of nearest neighbors prioritizes similarity and consistency rather than coverage [2512.07608]. Conversely, MIS on a similarity graph imposes a hard threshold and maximizes subset size under the independence constraint, which is a packing-style rather than continuous max–min objective [2606.01400].

The limitations recurring across the sources are computational cost, embedding dependence, and instability of overly aggressive adversarial or similarity-only designs. ICS requires \(k\) forward passes per query rather than one, and similarity computation over large pools can be expensive without precomputation [2311.09782]. Its results also depend on the quality and domain alignment of the fixed sentence-transformer embeddings [2311.09782]. MIS selection depends critically on embedding quality and threshold choice; low thresholds produce overly dense graphs and poor representativeness [2606.01400]. Metric-Fair Prompting depends on the quality of the embedding-based clinical similarity metric and only evaluates MedQA [2512.07608]. SPTR incurs extra compute from adversarial training and depends on a bank of hand-crafted prompts [2502.14376]. The clustering-based min–max assignment method requires jointly learned transform spaces and alternating optimization [1901.10760].

The most direct future directions are stated repeatedly in the data block: query-conditioned explicit max–min sampling algorithms, better domain-aligned embeddings, adaptive number of examples per prompt and adaptive \(k\), integration of model uncertainty or confidence, cluster-then-cover and active pairing, weighted or multi-objective MIS, and prompt-group reweighting or sampling policies that directly target worst-case performance [2311.09782] [2006.06879] [2512.07608] [2606.01400]. Taken together, these sources suggest that Max–Min Similarity Prompt Sampling is best understood as a general design principle: select prompts or demonstrations so that they are sufficiently relevant, sufficiently non-redundant, and sufficiently robust under the particular downstream objective being optimized.

Source: https://www.emergentmind.com/topics/max-min-similarity-prompt-sampling