Representational Contrastive Scoring (RCS)

• Representational Contrastive Scoring (RCS) is a method that uses contrastive objectives on neural embeddings to explicitly increase separation between target classes.
• It employs architectures like LVLMs and LSTM encoders with tailored loss functions (e.g., Mahalanobis and triplet losses) to capture critical safety and performance signals.
• Empirical results indicate that RCS enhances detection accuracy and text grading reliability with minimal computational overhead in high-risk applications.

Representational Contrastive Scoring (RCS) refers to a family of methodologies that leverage the geometric structure of learned internal representations for evaluating and discriminating among inputs to deep models. Characterized by contrastive objectives, RCS aims to sharpen discriminability by explicitly maximizing representational separation between target classes, conditions, or semantic properties. Practically, this often results in more precise, robust, and interpretable scoring for tasks ranging from multimodal security assessment to qualitative grading of natural language.

1. Conceptual Foundations and Motivation

RCS is predicated on the central observation that the internal representations—i.e., hidden states or embeddings—of deep neural networks encode rich task-relevant and safety-critical information. Effective representational geometry allows downstream contrastive scoring to separate classes or semantic categories with high fidelity. In large vision-LLMs (LVLMs), the activations from suitably chosen layers demonstrate pronounced separability of benign versus malicious (jailbreaking) inputs. Similarly, in educational NLP assessment, LSTM-derived hidden states for student texts form geometric clusters that reflect nuanced grading outcomes (Jiang et al., 2020, Hua et al., 12 Dec 2025).

Classical one-class outlier detectors fail to differentiate truly malicious or low-quality content from merely novel but benign inputs. By explicitly modeling both sides of the contrast—benign/malicious, high/low score, or other salient binary distinctions—RCS approximates optimal likelihood-ratio criteria, as formalized by the Neyman–Pearson lemma (Hua et al., 12 Dec 2025).

2. Canonical Architectures and Representation Extraction

RCS frameworks require principled extraction of fixed-dimensional representations. In multimodal LVLMs, the "last input token" hidden state at a selected intermediate layer is used, as it has been empirically shown to capture aggregate knowledge most relevant for safety signals. Optimal layers are neither too early (under-expressive) nor too deep (over-fitted to generation specifics). Evaluation metrics such as maximum SVM margin and inter- versus intra-class variance guide this selection, often yielding silhouette scores $r>0.8$ correlating strongly with downstream detection performance (Hua et al., 12 Dec 2025).

For written text, LSTM or biLSTM encoders are employed. A notable approach treats the entire set of timestep hidden states $\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$ as support points for empirical measures, preserving the fine-grained temporal dynamics of text sequences. No post-encoding pooling is invoked prior to scoring, allowing the full Wasserstein geometry to be exploited (Jiang et al., 2020).

3. Supervised Contrastive Learning in Representation Space

Representational contrastive learning is effected by objectives that ensure both high intra-class compactness and large inter-class separation. A typical form leverages a two-term loss:

• Cluster Consistency: Samples from the same source or class are pulled together in projected space.
• Contrastive Separation: Centroids of opposing classes (e.g., benign/malicious, different scoring bins) are forced apart by a margin.

For instance, with an MLP projection $g_\theta$ learned atop extracted features, dataset clustering and safety-separation losses are combined as $L = \alpha L_{dataset} + \beta L_{sep}$, with hyperparameters tuned for balance (e.g., $\alpha:\beta=1:5$ yields stable results) (Hua et al., 12 Dec 2025).

Alternatively, in text grading, a triplet loss in Wasserstein space over empirical measures encourages anchor samples and positives (same score) to be close, while anchor-negatives (differing scores) are separated with preference for larger absolute score gaps. The triplet construction leverages sampling proportional to score difference, maximizing discriminative pressure (Jiang et al., 2020).

4. Scoring Mechanisms

Once representations are projected, RCS defines robust scoring rules to quantify contrastive separation.

Mahalanobis Contrastive Detection (MCD)

Each dataset or class $d$ is fit as a Gaussian $\mathcal{N}(\mu_d, \Sigma_d)$ in the projected space. The Mahalanobis distance:

$$D_M(z; \mu_d, \Sigma_d) = \sqrt{(z-\mu_d)^\top \Sigma_d^{-1}(z-\mu_d)}$$

The contrastive score is

$$s_{MCD}(x) = \min_{d \in \text{Benign}} D_M(z; \mu_d, \Sigma_d) - \min_{d' \in \text{Malicious}} D_M(z; \mu_{d'}, \Sigma_{d'})$$

A larger $s_{MCD}$ indicates proximity to malicious clusters (Hua et al., 12 Dec 2025).

K-Nearest Contrastive Detection (KCD)

Projecting onto the unit sphere, RCS computes $\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$0-nearest neighbor radii to benign and malicious sets, defining

$\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$1

This non-parametric score is a monotonic proxy for the likelihood-ratio test (Hua et al., 12 Dec 2025).

Wasserstein Triplet Label Transfer

For grading, the scoring process is non-parametric: each test sample’s encoded empirical measure is compared via (regularized) 2-Wasserstein distances to all training samples. The test label is then assigned via $\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$2-nearest neighbor voting among the smallest-distance matches, closely approximating the label distribution (Jiang et al., 2020).

5. Empirical Evaluation and Applications

RCS demonstrates state-of-the-art performance in diverse contexts:

• Jailbreak Detection for LVLMs: On LLaVA-Vicuna-7B, KCD at the optimal intermediate layer achieves accuracy ≈ 92.0%, F1 ≈ 92.2%, AUROC ≈ 97.7%. MCD reaches similar AUROC ≈ 98.6%, exceeding all canonical anomaly detectors and OOD baselines. Remarkably, RCS generalizes to unseen attack distributions (e.g., "FigStep" attacks) and adapts to novel multi-turn attack families after seeing just 5–10 labeled examples (Hua et al., 12 Dec 2025).
• Automated Text Grading: In undergraduate biology lab reports, contrastive representational approaches using Wasserstein distances and biLSTM encoders achieve Quadratic Weighted Kappa (QWK) ≈ 0.628, nearly matching human–human reliability (QWK = 0.839). Baselines relying on mean-pooling or standard SVR fall markedly below (QWK = 0.46 or less). In several border cases, the RCS model’s decisions prompted reassessment of ground-truth by human coders (Jiang et al., 2020).

Table: Core RCS Instantiations Across Domains

Setting	Encoder/Rep. Layer	Loss/Projection	Score Type
LVLM Jailbreak	LLM (layer $\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$3)	MLP + cluster/separation	MCD/KCD
Text Grading	LSTM/biLSTM (all $\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$4)	Triplet in Wasserstein	$\mathcal{H}=\{\mathbf{h}_1,\dots,\mathbf{h}_N\}$5NN

A plausible implication is that careful layer selection and supervised projection can yield steep performance improvements in low-data or high-risk detection scenarios, due to the low effective rank of safety signals in the learned space.

6. Practical Considerations and Limitations

The computational footprint of RCS is modest: the addition of projection and scoring steps typically increases LVLM inference latency by less than 6%, with detector memory usage below 0.02 GB. RCS can be deployed in scenarios requiring white-box access to internal states and is compatible with early stopping strategies, minimizing unnecessary generation costs (Hua et al., 12 Dec 2025).

In grading scenarios, RCS avoids explicit output layers in favor of non-parametric label transfer, further improving interpretability. However, RCS depends on continuous access to appropriate contrastive sampling distributions (benign/malicious or labeled examples). For robust long-term deployment, periodic recalibration with new examples is required. RCS, in its current forms, assumes white-box access to model internals and may not apply to black-box deployments.

7. Discussion and Outlook

RCS unifies several paradigms in robust model evaluation and security through the lens of representation geometry and contrastive supervision. Its empirical success across LVLM security and qualitative text assessment demonstrates its versatility and effectiveness. Ongoing work focuses on few-shot adaptation, dynamic recalibration, and extending RCS to additional modalities and more complex multi-class scenarios. The ability for RCS-based models to highlight borderline or ambiguous inputs for human review suggests potential roles in high-stakes quality assurance pipelines (Jiang et al., 2020, Hua et al., 12 Dec 2025).