Cosine-Consistency Self-Voting Mechanism

• The paper introduces cosine-consistency self-voting as a method where multiple candidate responses are embedded, and the candidate with the highest average cosine similarity is chosen.
• It improves structured output generation in vision-language models, as demonstrated in precision agriculture with significant gains in diagnostic and treatment accuracy.
• The approach leverages domain-adapted embeddings and clustering to enhance semantic centrality while reducing computational overhead compared to greedy decoding.

to=arxiv_search 天天中彩票谁 {"query":"Cosine-Consistency Self-Voting Mechanism arXiv (Sharma et al., 4 Nov 2025, Petullo et al., 8 May 2026, Gupta et al., 8 Jul 2025, Arzhantsev et al., 27 May 2026, Wen et al., 30 Sep 2025)","max_results":10} Cosine-consistency self-voting mechanism is a self-consistency procedure in which a model generates multiple candidate responses for the same input, embeds each response in a semantic vector space, computes pairwise cosine similarities, and selects the candidate whose embedding is most consistent with the others under average cosine similarity. In the formulation reported for precision agriculture, the mechanism operates over long structured vision-language outputs rather than exact answer strings, so the final prediction is the semantically most central response rather than the most frequent literal response (Gupta et al., 8 Jul 2025).

1. Formal definition and decision rule

In the reported formulation, a vision-LLM generates $N$ candidate responses for the same image,

$r_1, r_2, \dots, r_N,$

and each candidate is encoded by an embedding model into

$$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$

Pairwise cosine similarity is then computed as

$$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$

For each response $i$, the global consistency score is the average cosine similarity to all other responses,

$$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$

and the final response is selected by

$i^\* = \arg\max_i s_i.$

The mechanism therefore chooses the candidate with maximum average cosine similarity to the remaining candidates, which the paper describes as analogous to choosing the medoid or “center” in semantic space (Gupta et al., 8 Jul 2025).

The paper defines cosine-consistency as high average cosine similarity among the embeddings of responses to the same input, and self-voting as the process by which the set of responses jointly selects the most consistent candidate by this similarity. No weighting or normalization other than averaging is described in this formulation. The same full-text embedding and cosine-voting rule are used across the reported subtasks; only the evaluation lens changes (Gupta et al., 8 Jul 2025).

2. Instantiation in precision agriculture

The explicit use of the term appears in a precision-agriculture system for maize leaf disease management built on a fine-tuned PaliGemma vision-LLM. The backbone combines a SigLIP vision encoder and a Gemma LLM. Given a maize leaf image and a fixed prompt, the model produces a structured text string of the form Assessment: ...; Analysis: ...; Treatment: ...; Prevention: .... Training uses 400 farmer-captured maize leaf blight cases with expert-validated four-step text, and the purpose of fine-tuning is to make PaliGemma understand crop-disease terminology, field symptom patterns, and appropriate fungicide recommendations (Gupta et al., 8 Jul 2025).

At inference time, the system uses temperature sampling with $t = 1.0$ to generate multiple candidate reports, embeds them with a domain-adapted all-MiniLM-L6-v2 model, computes cosine-consistency scores, and returns the report with the largest score. The evaluation varies the number of generations over 5, 10, 15, and 20 responses per image. The paper reports that the approach improves diagnostic accuracy from $82.2\%$ to $87.8\%$, symptom analysis from $r_1, r_2, \dots, r_N,$0 to $r_1, r_2, \dots, r_N,$1, and treatment recommendation from $r_1, r_2, \dots, r_N,$2 to $r_1, r_2, \dots, r_N,$3 compared to standard greedy decoding. It also states that performance remains stable across different generation counts and that peak performance typically occurs with 10–15 generations, suggesting an operational balance between diagnostic diversity and computational efficiency (Gupta et al., 8 Jul 2025).

The agricultural study also reports a comparison among greedy decoding, cosine-consistency with non-fine-tuned embeddings, and cosine-consistency with fine-tuned embeddings. For Step 1 – Assessment at 5 generations, the table gives Greedy: 74 correct $r_1, r_2, \dots, r_N,$4, NFT voting: 80 correct $r_1, r_2, \dots, r_N,$5, and FT voting: 79 correct $r_1, r_2, \dots, r_N,$6. For Step 2 – Analysis at 15 generations, it gives Greedy: 35 correct $r_1, r_2, \dots, r_N,$7, NFT voting: 36 correct $r_1, r_2, \dots, r_N,$8, and FT voting: 47 correct $r_1, r_2, \dots, r_N,$9. For Step 3 – Treatment at 20 generations, it gives Greedy: 25 correct $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$0, NFT voting: 32 correct $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$1, and FT voting: 39 correct $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$2 (Gupta et al., 8 Jul 2025).

3. Domain-adapted embedding space and expert supervision

A central component of the mechanism is the use of domain-adapted embeddings rather than generic sentence embeddings. The paper argues that off-the-shelf cosine similarity, paraphrase models, and NLI can be high even when clinical or agronomic content is wrong. It gives the specific observation that a generated answer can get cosine similarity approximately $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$3 to the ground truth while prescribing different fungicides or mis-characterizing severity. This is the immediate reason that semantic centrality must be defined in a domain-specific embedding space rather than by generic text similarity alone (Gupta et al., 8 Jul 2025).

The domain adaptation procedure uses OpenAI o1-mini as a meta-evaluator configured as an expert plant pathologist. The evaluator compares generated text against expert ground truth with a rubric in which $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$4–$$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$5 denotes different diseases or requiring different treatments, $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$6–$$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$7 denotes the same disease but treatment adjustments may be necessary, and $$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$8–$$\mathbf{e}_i = \mathrm{Embed}(r_i) \in \mathbb{R}^d.$$9 denotes essentially the same disease and treatment approach. These prompt-based scores are used both as an evaluation metric and as supervision for embedding-model adaptation (Gupta et al., 8 Jul 2025).

For embedding adaptation, the study uses 10 annotated maize blight samples. Each sample has 21 candidate outputs, comprising 20 sampled responses plus 1 greedy decode, yielding $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$0 unordered pairs per sample and 2,100 pairs in total. Target similarity labels are assigned as follows: if both outputs score at least $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$1, label $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$2; if exactly one output scores at least $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$3, label $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$4; if both outputs score below $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$5, label $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$6. The intent is to tune the embedding model so that

$$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$7

thereby aligning cosine similarity with agronomic equivalence rather than lexical overlap. The resulting fine-tuned all-MiniLM-L6-v2 model is reported as approximately 80 MB, which the paper presents as compatible with mobile deployment (Gupta et al., 8 Jul 2025).

4. Relation to self-consistency and weighted voting

The cosine-consistency mechanism is a semantic analogue of standard self-consistency. In standard self-consistency, one samples multiple reasoning traces and final answers and returns the most frequent answer by majority vote: $$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$8 This exact-string voting rule is well suited to short canonical outputs, but the agricultural paper notes that it is poorly matched to long, structured texts, where exact string matches are rare even for semantically equivalent outputs. In that setting, cosine-consistency operates as a semantic majority vote in an embedding space rather than a literal vote over strings (Gupta et al., 8 Jul 2025, Petullo et al., 8 May 2026).

Broader work on inference-time aggregation locates cosine-consistency within a larger family of weighted or structured self-voting schemes. Confidence-Informed Self-Consistency assigns a critic-derived confidence score to each candidate, softmax-normalizes these scores, and performs weighted majority voting over answers rather than equal voting (Petullo et al., 8 May 2026). “The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute” introduces inverse-entropy weighted voting, where each chain receives weight

$$\cos(\mathbf{e}_i,\mathbf{e}_j) = \frac{\mathbf{e}_i \cdot \mathbf{e}_j}{\|\mathbf{e}_i\|\;\|\mathbf{e}_j\|}.$$9

with $i$0 the average sequence-level entropy. That work reports that sequential reasoning beats parallel reasoning in 43 of 45 configurations, or $i$1, and explicitly describes inverse-entropy weighting and cosine-consistency as conceptually parallel: inverse-entropy weighting uses internal per-token distributions to infer confidence, whereas cosine-consistency would use similarity of representations to infer agreement or alignment with a latent consensus (Sharma et al., 4 Nov 2025).

A different but related reframing appears in “Self-Consistency via Marginal Sharpening,” which argues that self-consistency should target the answer marginal rather than the full-output distribution. In that view, what matters is whether an answer is supported by many plausible reasoning paths, not whether a single trace-answer transcript is individually likely. This does not implement cosine similarity, but it situates self-voting as an answer-level inference objective rather than only a post-hoc heuristic (Arzhantsev et al., 27 May 2026).

5. Cosine-based clustering and representative selection

A more elaborate cosine-based mechanism appears in “VecCISC: Improving Confidence-Informed Self-Consistency with Reasoning Trace Clustering and Candidate Answer Selection.” There, an embedding model maps each reasoning trace $i$2 to $i$3, traces are grouped by final answer, and embeddings within each answer group are clustered. For each cluster, the method computes a centroid

$i$4

and selects the representative trace closest to the centroid in cosine space. The critic LLM is then called only on those representatives, and their confidence scores are aggregated by weighted voting over answers (Petullo et al., 8 May 2026).

The paper explicitly characterizes VecCISC as a cosine-consistency self-voting mechanism in the sense that cosine similarity of reasoning-trace embeddings is used to recognize and compress semantically redundant, degenerate, or hallucinated traces before confidence-based voting. It evaluates KMeans and hierarchical agglomerative clustering with cosine distance and average linkage, tunes the maximum number of clusters per answer $i$5 over $i$6, and avoids DBSCAN because small changes in a distance threshold can drastically change clusters in high-dimensional embedding spaces. The study reports that VecCISC reduces the total token usage by $i$7 while maintaining or exceeding the accuracy of CISC. It also reports average critic-call reductions of $i$8 for VecCISC + KMeans and $i$9 for VecCISC + HAC at budget $$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$0, and emphasizes that random selection without semantic clustering significantly underperforms SC and CISC, indicating that the clustering step is essential rather than incidental (Petullo et al., 8 May 2026).

This clustered variant differs from the simpler agricultural rule. The agricultural mechanism chooses the single candidate with maximum average cosine similarity across the full set of responses. VecCISC instead uses cosine structure to compress the set of reasoning traces, after which a critic supplies answer weights. Both mechanisms, however, treat cosine similarity as a proxy for semantic redundancy and semantic centrality (Gupta et al., 8 Jul 2025, Petullo et al., 8 May 2026).

6. Computational profile, limitations, and broader generalization

The agricultural formulation adds computational overhead relative to greedy decoding because it requires $$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$1 VLM generations, $$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$2 embedding passes, and pairwise cosine computation with $$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$3 similarity operations. The paper nonetheless states that the embedding computations are lightweight, the embedding model is approximately 80 MB, and the overall system remains compact enough for deployment on mobile devices and runs in real time on mid-range smartphones. It recommends 10–15 generations as a practical operating point (Gupta et al., 8 Jul 2025).

Several limitations are also stated or directly implied in the reported studies. A cosine-consistency selector can return a central but wrong answer if all sampled responses are consistently wrong. In the agricultural setting, the training data comprise 400 images for PaliGemma fine-tuning and 10 images for embedding fine-tuning, the evaluation is restricted to maize leaf blight, and prompt-based scoring depends on o1-mini as an expert surrogate. The paper therefore does not empirically establish generalization to other crops, multi-disease settings, or other domains, although it explicitly identifies medical imaging, remote sensing, visual question answering, and other long-form structured-output tasks as natural extensions (Gupta et al., 8 Jul 2025).

Related work suggests that self-voting can be generalized beyond cosine similarity and beyond answer selection. EvoQuality adapts self-consistency to ranking-based image quality assessment by using pairwise majority voting to create pseudo-rankings and then optimizing a vision-LLM with group relative policy optimization. It reports a $$s_i = \frac{1}{N-1}\sum_{j \neq i} \cos(\mathbf{e}_i, \mathbf{e}_j),$$4 boost in PLCC across diverse IQA benchmarks, but it does not explicitly use cosine similarity in its core algorithm (Wen et al., 30 Sep 2025). A plausible implication is that cosine-consistency self-voting is best understood not as a single fixed algorithm, but as one member of a broader class of inference-time consensus mechanisms in which multiple stochastic outputs are aggregated through semantic centrality, confidence weighting, clustering, or answer-level marginalization.