SV-Detect: AI-generated Text Detection with Steering Vectors

Abstract: Detecting machine-generated text is especially difficult under distribution shift, such as transfer across domains, source models, and editing attacks. We propose a fake-text detector based on steering vectors extracted from the hidden representations of a frozen LLM. At each layer, we construct a direction that separates human-written from machine-generated text, and represent each input by its layer-wise alignment with these directions. A lightweight classifier trained on these projection features yields the final detection score. Our method achieves strong performance both in-distribution and under distribution shift, including across domains, source models, and machine-editing transformations such as polishing and rewriting. Interpretation analyses show that the learned directions align with recognizable stylistic cues while capturing substantial additional signal beyond surface features. These results position fake-text detection as a representation-space probing problem and show that steering vectors provide a simple and effective solution.

• The paper introduces a method that uses layerwise steering vectors, computed via logistic regression, to effectively separate machine-generated text from human writing.
• It leverages frozen transformer activations to create a low-dimensional cosine similarity feature vector, achieving near-perfect AUROC scores on challenging benchmarks.
• Experiments confirm that the approach generalizes across diverse domains and adversarial attacks while providing interpretable signals that distinguish formal LLM output from informal human text.

SV-Detect: Detecting Machine-Generated Text with Steering Vectors

Problem Setting and Motivation

The increasing fluency and diversity of LLM outputs have rendered many traditional machine-generated text detectors unreliable under distribution shift. Detecting AI-generated text now requires robustness across domains, models, and attacks, such as paraphrasing and machine-assisted editing. The dominant trend in LLM-generated text detection pivots around surface-level statistics, supervised text classifiers, and zero-shot approaches, all of which lose efficacy under unfamiliar domains or when attacking distributions change (Wu et al., 2024, Fu et al., 15 Sep 2025). SV-Detect reframes fake-text detection as a probing task in the high-dimensional representation space of transformer LMs: it leverages learned steering directions in frozen model activations to robustly distinguish human and machine text.

Method: Layerwise Steering Vectors and Probing

The SV-Detect pipeline consists of extracting hidden representations from a frozen transformer, constructing per-layer steering vectors that maximally separate human and machine examples in those representations, and training a lightweight classifier on the resulting projection features.

Given text $x$, SV-Detect produces mean-pooled residual activations $a_l(x)$ at each of the $L$ model layers. For each layer $l$, a discriminative steering vector $v_l$ is constructed—most effectively via the normal vector of a logistic regression hyperplane trained on the class-separated activations (Figure 1).

Figure 1: The SV-Detect pipeline: mean-pooled layer-wise activations are projected onto steering directions, yielding a feature per layer for the detection head.

Each input is thus represented as a vector of $L$ standardized cosine similarities—the projections of activations onto steering vectors. This low-dimensional, interpretable feature vector is passed to a logistic regression, isolating the effect of the learned directions from head complexity.

This approach does not require fine-tuning the base LM and is computationally efficient relative to perturbation-based detectors—it incurs only a single forward pass with lightweight additional computation.

Main Results: Robustness Across Distribution Shifts

SV-Detect is extensively validated on challenging recent benchmarks, namely DetectRL (Wu et al., 2024) and MIRAGE (Fu et al., 15 Sep 2025). DetectRL spans distribution shift by domain, generator, and attack; MIRAGE focuses on scenarios involving direct generation, machine polishing, and rewriting.

DetectRL: In-Distribution and Cross-Source Transfer

Across all in-distribution settings (Multi-Domain, Multi-LLM, Multi-Attack), SV-Detect achieves near-perfect results (AUROC $>$ 99.8, $F_1$ $>$ 99) matching or exceeding strong RoBERTa-based supervised baselines and outperforming all zero-shot methods (Figure 2).

Figure 2: In-domain AUROC and $F_1$ scores demonstrate the ceiling-level performance of SV-Detect across all DetectRL settings.

On the critical cross-source transfer axis—i.e., training on one domain/model/attack and testing on another—the steering-vector pipeline attains the most stable and high transfer scores among compared methods, indicating that the latent-space signal is robust to plausible distributional perturbations.

MIRAGE: Generalization to Editing and Generation

On MIRAGE, steering vectors trained solely on machine-polished samples generalize to generation tasks (AUROC $a_l(x)$00.98) but not vice-versa. When multi-direction, task-specific steering vectors are constructed (one each for generate, polish, rewrite), SV-Detect yields state-of-the-art performance on all MIRAGE tasks, AUROC $a_l(x)$10.99 for direct generation and $a_l(x)$20.94 for editing attacks (Figure 3).

Figure 3: Multi-vector SV-Detect achieves superior AUROC across MIRAGE’s editing and generation scenarios.

Moreover, steering vectors learned on MIRAGE transfer well to DetectRL, consistently outperforming zero-shot baselines even with significant benchmark and domain shift.

Design Ablations and Efficiency

Ablations demonstrate that:

• Logistic-regression-based direction finding markedly outperforms mean-difference or PCA-based approaches, especially under distribution shift (Figure 4).
• Logistic regression as a downstream classifier offers optimal stability; highly flexible classifiers (e.g., CatBoost) add no value and harm transfer.
• Backbone LLM choice influences robustness; Qwen3-1.7B and Gemma models provide slightly enhanced generalizability over GPT-Neo.

Steering-vector representations contain discriminative signal beyond surface stylistic regex baselines, with a 13–24 AUROC point improvement over strong hand-crafted features.

Figure 4: Ablation reveals the superiority of logistic regression steering and head, especially for hardest transfer pairs; alternatives degrade to nearly chance.

Steering Vector Interpretability: What is Detected?

Layer attribution highlights that useful fake-text signals concentrate in later layers for editing detection (MIRAGE, layers 29–31) and are more distributed in direct detection (DetectRL, layers 0–2 and 14–21). The logit lens analysis projects steering vectors through the unembedding matrix, surfacing interpretable lexical cues: machine leaning is associated with polished, formal register; human leaning recovers colloquial, discourse-oriented tokens (Figure 5, Figure 6).

Figure 5: Top tokens for steering directions elucidate formal LLM register versus informal human traits.

Consistency analysis across four open LMs (GPT-Neo, Qwen3, Gemma-pt, Gemma-it) reveals that the discovered lexical patterns are not model specific; positive-direction words include endeavors, utilization, and leveraging, while negatives are marked by casual connectives and discourse fragments (Figure 7 and Figure 6).

Figure 7: Consensus lexical signatures across LMs reveal convergent stylistic detection cues.

Figure 6: Top tokens for $a_l(x)$3 (“LLM-leaning”) are formal/technical, while $a_l(x)$4 (“human-leaning”) tokens are informal/discourse-like, consistent across models.

Tokenwise analysis visualizes how LLM-generated text produces coherent runs of strongly positive projections, while human writing is variable and heterogeneous (Figure 8).

Figure 8: Per-token steering-vector projections sharply distinguish LLM output from human text.

Practical and Theoretical Implications

SV-Detect’s representation-probing paradigm moves fake-text detection away from unreliable surface markers to robust, interpretable latent features. Its transferability implies it can underpin more general tools for model-authorship attribution even under adversarial post-processing, given adaptation to new generator classes. The strong alignment of learned directions with emergent "LLM register" suggests that steering vectors operationalize the abstract stylistic distinction between current LLMs and humans at the neural representation level.

However, full automation via this pipeline is bottlenecked by its dependence on the probe LLM. While open-weight models suffice for strong performance, shifts in base model or writing scenario may require recomputation of steering directions. Its reliance on a frozen LLM forward pass makes it more expensive at inference than lightweight encoders but significantly cheaper than perturbation-based detectors (Appendix: Inference Latency).

Future directions include multilingual extension, adaptation for more elusive hybrid human–machine texts, and integration with lighter encoders for efficiency. The approach opens avenues for general representation-based synthetic content detection, unifying robustness and interpretability.

Conclusion

SV-Detect demonstrates that fake-text detection can be robustified by probing for model-authorship directions in transformer hidden states. Logistic-regression-derived steering vectors operating on mean-pooled activations enable strong and generalizable separation of human from machine text in open LMs, with interpretability and transfer under distribution shift. This reframes the detection problem as a test of latent stylistic alignment, offering a stable foundation as generative models and attack regimes continue to evolve.