CLUE: Non-parametric Verification from Experience via Hidden-State Clustering (2510.01591v1)

Abstract: Assessing the quality of LLM outputs presents a critical challenge. Previous methods either rely on text-level information (e.g., reward models, majority voting), which can overfit to superficial cues, or on calibrated confidence from token probabilities, which would fail on less-calibrated models. Yet both of these signals are, in fact, partial projections of a richer source of information: the model's internal hidden states. Early layers, closer to token embeddings, preserve semantic and lexical features that underpin text-based judgments, while later layers increasingly align with output logits, embedding confidence-related information. This paper explores hidden states directly as a unified foundation for verification. We show that the correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations. To validate this, we present Clue (Clustering and Experience-based Verification), a deliberately minimalist, non-parametric verifier. With no trainable parameters, CLUE only summarizes each reasoning trace by an hidden state delta and classifies correctness via nearest-centroid distance to success'' andfailure'' clusters formed from past experience. The simplicity of this method highlights the strength of the underlying signal. Empirically, CLUE consistently outperforms LLM-as-a-judge baselines and matches or exceeds modern confidence-based methods in reranking candidates, improving both top-1 and majority-vote accuracy across AIME 24/25 and GPQA. As a highlight, on AIME 24 with a 1.5B model, CLUE boosts accuracy from 56.7% (majority@64) to 70.0% (top-maj@16).

• The paper presents CLUE, a non-parametric, training-free framework that verifies LLM outputs by clustering hidden-state activation deltas.
• It summarizes reasoning traces into success and failure centroids using Euclidean distance, providing robust performance across domains and model scales.
• The method demonstrates that RL-trained models yield enhanced geometric separation of correct reasoning, suggesting new avenues for verifiability-focused LLM training.

Non-parametric Verification of LLM Reasoning via Hidden-State Clustering: The CLUE Framework

Introduction

The paper introduces CLUE, a non-parametric, training-free framework for verifying the correctness of LLM outputs by leveraging the geometric structure of hidden-state trajectories. Unlike prior approaches that rely on surface-level text signals or token-level confidence, CLUE operates directly on the internal activations of the model, hypothesizing that correct and incorrect reasoning traces are geometrically separable in hidden space. The method is minimalist: it summarizes each reasoning trace by the difference (delta) between hidden states at the start and end of the reasoning block, aggregates these deltas into class centroids for successes and failures, and classifies new traces by nearest-centroid distance. This approach is shown to outperform both LLM-as-a-judge and confidence-based baselines, especially on smaller or less-calibrated models, and generalizes across domains and model scales.

Geometric Structure of Hidden-State Trajectories

The central hypothesis is that the correctness of a solution is encoded as a geometrically separable signature within the trajectory of hidden activations. Empirical evidence is provided by projecting hidden-state deltas for correct and incorrect solutions onto two principal components, revealing clear clustering by correctness across models.

Figure 1: Visualization of hidden state trajectories for correct (blue) and incorrect (orange) solutions from the experience set, projected to 2D using PCA. Each panel displays data from a different base model, showing geometric separation.

This geometric separation motivates a non-parametric approach: if correct and incorrect traces occupy distinct regions in hidden space, a simple centroid-based classifier should suffice for verification.

The CLUE Framework

CLUE consists of two phases: centroid construction and inference. During centroid construction, labeled trajectories are summarized by their activation deltas (difference between hidden states at the start and end of the reasoning block, across all layers). These deltas are averaged within each class to form success and failure centroids. At inference, a new trajectory is summarized by its activation delta and classified by the layer-averaged Euclidean distance to the two centroids.

Figure 2: Overview of CLUE. Left: Labeled historical trajectories are summarized by activation deltas and aggregated into success and failure centroids. Right: A new trajectory is classified by its distance to these centroids.

The method is deliberately minimalist, requiring no gradient-based optimization or parameter tuning. The only requirement is access to the model's hidden states at the boundaries of the reasoning block.

Pseudocode

Centroid Construction:

def construct_centroids(trajectories, labels): # trajectories: list of (h_start, h_end) pairs, each of shape [L, D] # labels: list of 0 (fail) or 1 (success) deltas_success = [] deltas_failure = [] for (h_start, h_end), y in zip(trajectories, labels): delta = h_end - h_start # shape [L, D] if y == 1: deltas_success.append(delta) else: deltas_failure.append(delta) V_succ = np.mean(deltas_success, axis=0) V_fail = np.mean(deltas_failure, axis=0) return V_succ, V_fail

Inference:

def verify_trajectory(h_start, h_end, V_succ, V_fail): delta = h_end - h_start # shape [L, D] d_succ = np.mean([np.linalg.norm(delta[l] - V_succ[l]) for l in range(L)]) d_fail = np.mean([np.linalg.norm(delta[l] - V_fail[l]) for l in range(L)]) return int(d_succ < d_fail)

Experimental Results

Binary Classification

CLUE achieves substantial improvements over LLM-as-a-judge baselines (e.g., GPT-4o) in binary classification of solution correctness. Notably, CLUE maintains high true negative rates even on weaker models, where LLM judges exhibit strong optimistic bias and frequently misclassify incorrect solutions as correct. For example, on AIME 2024, CLUE achieves 80.9% accuracy on Nemotron-1.5B outputs, compared to 58.6% for GPT-4o (answer only).

Reranking and Aggregation

CLUE is highly effective as a reranker for candidate solutions. On AIME 24, reranking 64 candidates from Nemotron-1.5B, CLUE's top-maj@16 accuracy is 70.0%, compared to 56.7% for majority voting and 56.7% for DeepConf (a confidence-based baseline). Even top@1 selection with CLUE outperforms majority voting, indicating that the geometric signal is robust and actionable.

Cross-Model and Training Paradigm Analysis

CLUE's effectiveness is strongly influenced by the training paradigm of the base model. RL-trained models (e.g., Nemotron-1.5B, Polaris-4B) exhibit much stronger geometric separation and act as superior verifiers, not only for their own outputs but also for those generated by SFT-trained models.

Figure 3: Cross-model reranking performance on AIME 24. RL-trained models are superior verifiers for both self and SFT-generated trajectories.

This suggests that RL with verifiable rewards induces more contrastive supervision, leading to cleaner separation of correct and incorrect reasoning in hidden space.

Generalization to Out-of-Domain Reasoning

CLUE generalizes to non-mathematical domains, as demonstrated on the WebInstruct-verified benchmark (physics, law, finance, humanities). CLUE outperforms GPT-4o as a judge, especially on the 4B model where the LLM judge's accuracy drops below random, but CLUE maintains 59.2% accuracy.

Layer-wise Analysis

Layer-wise PCA and centroid distance analysis reveal that class separability emerges gradually across depth, peaking in the final layers. This aligns with the hypothesis that deeper layers encode more explicit correctness signals.

Figure 4: Layer-wise separability. PCA projections show increasing class separation from shallow to deep layers; centroid distance $d^{(\ell)}$ increases with depth.

Implementation Considerations

• Data Requirements: CLUE requires a labeled experience set of correct and incorrect trajectories. The method is robust to the size of this set, as it relies on centroid averaging.
• Model Access: Access to hidden states at the start and end of the reasoning block is necessary. This is feasible for open-source models but may be restricted for proprietary APIs.
• Computational Cost: The method is lightweight, involving only mean aggregation and Euclidean distance computations. No training or backpropagation is required.
• Deployment: CLUE can be integrated as a post-hoc verifier or reranker in any LLM pipeline that supports hidden-state extraction. It is especially valuable for smaller or less-calibrated models where confidence-based methods are unreliable.

Theoretical and Practical Implications

The results provide strong evidence that correctness signals are geometrically encoded in LLM hidden states, and that these signals are accessible via simple, non-parametric statistics. The finding that RL-trained models develop more separable internal representations suggests that training objectives can be designed to explicitly optimize for internal verifiability, not just output accuracy. This opens avenues for new training paradigms that directly shape the geometry of reasoning in hidden space.

Practically, CLUE enables robust, domain-agnostic verification without the need for expensive reward models or retraining, and is resilient to distribution shift. The method's simplicity and generality make it a strong candidate for deployment in high-stakes applications where correctness is critical.

Future Directions

• Contrastive Training Objectives: Incorporating explicit contrastive losses to further enhance geometric separation of correct and incorrect reasoning.
• Layer Selection and Weighting: Investigating optimal strategies for layer selection or adaptive weighting in the distance metric.
• Unsupervised or Semi-supervised Extensions: Exploring whether geometric separation can be leveraged in low-label or unsupervised settings.
• Integration with Early Exit and Test-Time Adaptation: Combining CLUE with early stopping or test-time adaptation methods to improve efficiency and reliability.

Conclusion

CLUE demonstrates that the internal computation of LLMs encodes robust, geometrically separable signals of correctness, accessible via simple centroid-based clustering of activation deltas. The method outperforms both LLM-judges and confidence-based verifiers, generalizes across domains and model scales, and reveals a strong connection between training paradigm and internal representational clarity. These findings motivate a shift toward direct analysis and optimization of the reasoning process in hidden space, with significant implications for the design of future LLMs and their verification systems.