DivEye: Lightweight AI Text Detector

• DivEye is a model-agnostic framework for detecting AI-generated text that uses zero-shot, diversity-aware token surprisal statistics to capture both predictability and rhythmic unpredictability.
• It differentiates human and machine text by analyzing a 9-dimensional feature vector derived from distributional, first-order, and second-order surprisal metrics, achieving significant accuracy gains.
• The framework operates with exceptional speed (<0.01s per sample) and maintains robustness across multiple language models and adversarial attacks while enhancing detection performance.

DivEye is a lightweight, model-agnostic AI-generated text detection framework that leverages zero-shot, diversity-aware statistical features computed over token-level surprisal values extracted from any pretrained autoregressive LLM, such as GPT-2. These features are used to train a binary classifier—typically an XGBoost model—to distinguish between human- and machine-written text, either as a standalone detector or as an auxiliary signal for boosting existing detectors. The framework is motivated by the observation that human-authored text exhibits richer variability and temporal fluctuation in lexical and structural unpredictability than LLM outputs, which are described as more uniform and predictable because they are tuned for likelihood maximization (Basani et al., 23 Sep 2025).

1. Statistical premise and detection objective

The central insight of DivEye is that the detection problem is not exhausted by estimating how predictable a text is on average. Prior detectors based on token-level likelihoods, perplexity, token ranks, entropy, or opaque fine-tuned classifiers are described as limited in distinct ways: perplexity measures only average predictability and is insensitive to rhythmic fluctuation or diversity; LogRank- and entropy-based methods use statics over token ranks or entropy but ignore how unpredictability evolves across the sequence; and black-box classifiers may exploit undiagnosed features, often overfit to domain or generator, and lack transparency (Basani et al., 23 Sep 2025).

DivEye instead quantifies both how predictable and how varied that predictability is. The motivating claim is that humans naturally introduce irregularities in writing—unexpected word choices, topical shifts, or creativity—causing local bursts of unpredictability, whereas LLMs tend to generate tokens with smoother, lower-variance surprisal profiles. In the formulation given for DivEye, these fluctuations are treated as a global and stylometric signature. The paper further reports empirical validation through histograms and distributions showing that human texts have higher mean and variance in surprisal, with heavier tails compared to LLM outputs, and through ablations in which removing temporal features results in a significant accuracy drop (Basani et al., 23 Sep 2025).

A common misconception is that the method is simply a repackaging of perplexity. The description of DivEye explicitly rejects that interpretation: its signal is diversity-aware and temporal, with particular emphasis on second-order variation in surprisal. The paper characterizes this as “rhythmic unpredictability,” and presents it as a powerful and underexplored signal for LLM detection (Basani et al., 23 Sep 2025).

2. Computational pipeline

The operational pipeline begins with token-level log-probability extraction. For an input text $X = \{x_1, \ldots, x_n\}$, the framework computes the token-level log-probabilities $P(x_t \mid x_{<t})$ using a pretrained autoregressive LM. It then computes surprisal for each token as

$S(x_t) = -\log P(x_t \mid x_{<t}).$

From the surprisal sequence, DivEye extracts a 9-dimensional vector $\mathcal{D}$ comprising distributional and temporal statistics. This vector is then used to train a binary classifier, typically XGBoost, to distinguish human from AI text. In prediction mode, the same feature extraction procedure is applied to a new text and the trained classifier outputs an AI or human probability (Basani et al., 23 Sep 2025).

The paper also gives an algorithmic outline in compressed form:

$S(x_t) = -\log P(x_t \mid x_{<t}).$9

This design is explicitly presented as lightweight and backbone-agnostic. The backbone-independence claim is tied to the fact that DivEye uses token probabilities from a wide range of LMs, including GPT-2, Llama, Falcon, and Mistral. A plausible implication is that the method is intended to decouple detector behavior from any single generator family, although the paper’s concrete claim is only that it remains effective across multiple backbones and evaluation regimes (Basani et al., 23 Sep 2025).

3. Feature space and mathematical formulation

The feature vector $\mathcal{D}$ contains nine values organized into three categories: distributional statistics over the surprisal sequence, first-order temporal differences, and second-order temporal differences. The representation is specified as

$$\mathcal{D} = \left\{ \mu_s, \sigma_s^2, \gamma_1, \gamma_2,\,\, \Delta\mu, \Delta\sigma^2,\,\, \sigma^2_{\Delta^2}, H_{\Delta^2}, \rho_{\Delta^2} \right\}.$$

Category	Features	Role stated in the paper
Distributional statistics	$\mu_s$, $\sigma_s^2$, $\gamma_1$, $\gamma_2$	Average surprisal, dispersion, asymmetry, tail heaviness
First-order temporal differences	$P(x_t \mid x_{<t})$0, $P(x_t \mid x_{<t})$1	Average change and variability of token-to-token change
Second-order temporal differences	$P(x_t \mid x_{<t})$2, $P(x_t \mid x_{<t})$3, $P(x_t \mid x_{<t})$4	Rapid stylistic transitions, irregularity or complexity, clustering of bursts

The distributional statistics are defined over all $P(x_t \mid x_{<t})$5. Mean surprisal $P(x_t \mid x_{<t})$6 is the average surprisal; variance $P(x_t \mid x_{<t})$7 measures dispersion; skewness $P(x_t \mid x_{<t})$8 captures asymmetry and is described as high when rare, surprising tokens are present; and kurtosis $P(x_t \mid x_{<t})$9 measures tail heaviness, interpreted as the frequency of outlier unpredictability. The paper writes these statistics as

$S(x_t) = -\log P(x_t \mid x_{<t}).$0

The first-order temporal statistics summarize volatility. The mean of first-order differences, $S(x_t) = -\log P(x_t \mid x_{<t}).$1, is the average change in surprisal from token to token, and the variance of first-order differences, $S(x_t) = -\log P(x_t \mid x_{<t}).$2, captures the variability of these changes and is described as measuring burstiness. The second-order statistics formalize rhythmic unpredictability. Let

$S(x_t) = -\log P(x_t \mid x_{<t}).$3

Then the variance of second-order differences $S(x_t) = -\log P(x_t \mid x_{<t}).$4 measures rapid stylistic transitions, the entropy of second-order differences $S(x_t) = -\log P(x_t \mid x_{<t}).$5 quantifies irregularity or complexity in unpredictability rhythm, and the autocorrelation of second-order differences $S(x_t) = -\log P(x_t \mid x_{<t}).$6 checks whether unpredictability bursts cluster, which the summary associates with human-like creativity (Basani et al., 23 Sep 2025).

4. Empirical performance across benchmarks

On the MAGE benchmark under the arbitrary-domain, arbitrary-model setting, DivEye is reported at 78.15% AvgAcc and 0.88 AUROC, compared with BiScope at 75.12% AvgAcc and 0.86 AUROC, Binoculars at 65.49% AvgAcc and 0.71 AUROC, and Entropy at 62.86% AvgAcc and 0.67 AUROC. The paper states a gain over the best zero-shot baseline of up to +33.2% in AvgAcc (Basani et al., 23 Sep 2025).

On the RAID benchmark, DivEye is reported at 93.6% AvgAcc, compared with Binoculars at 79.0%, GLTR at 59.7%, and a fine-tuned state of the art baseline at 94.9%. The reported gain over zero-shot methods is +13.7% AvgAcc. Under adversarial evaluation, the MAGE paraphrasing setting gives DivEye 76.5% AvgAcc and AUROC 0.87, compared with BiScope at 69.3% AvgAcc and AUROC 0.81, while on RAID adversarial attacks DivEye outperforms Binoculars by +11.2% (Basani et al., 23 Sep 2025).

The paper also reports a detector-boosting regime. When combined with RADAR, DivEye yields +18.7% AvgAcc and raises AUROC from 0.62 to 0.90. When combined with BiScope, it raises AvgAcc by 9.38% and AUROC from 0.86 to 0.93. In cross-model, cross-domain, and cross-language settings, the summary reports that HC3 in English and Chinese yields F1 better than GLTR and OpenAI-Detector; backbone-agnostic evaluation gives AUROC 0.88–0.91 across GPT-2, Llama-3.1-8B, and Falcon-7B; and commercial LLM evaluation gives AUROC $S(x_t) = -\log P(x_t \mid x_{<t}).$7 across GPT-3.5-Turbo, GPT-4o, Claude, and Gemini (Basani et al., 23 Sep 2025).

Computationally, the reported inference time is less than 0.01s per sample, stated as up to 2971x faster than RAiDAR. The paper further states that same-model detection remains effective: even when the same LM is used for both generation and surprisal estimation, DivEye maintains high accuracy in the range 96–99% (Basani et al., 23 Sep 2025).

5. Interpretability and explanatory structure

DivEye is presented as inherently interpretable because each of the nine features has a clear information-theoretic or temporal stylometric meaning traceable to unpredictability, volatility, and rhythmic burstiness. The reported feature importance analysis assigns the greatest weight to second-order features—entropy and autocorrelation—which contribute approximately 39.4% to decisions, followed by distributional and first-order features (Basani et al., 23 Sep 2025).

The interpretability claim is reinforced by several forms of visual and ablation evidence. Distribution plots of mean and variance of surprisal are described as showing clear separation between AI and human texts. Class probability distributions are reported to show high discriminative power. AUROC spider plots are described as visualizing consistent superiority across diverse domains and models. In a representative examples table, AI texts are assigned probabilities around 0.96–0.97, whereas a human-written Yelp example is assigned 0.09 (Basani et al., 23 Sep 2025).

The leave-one-out ablation result is particularly important for interpreting the feature set. Removing any single feature is reported to lead to a statistically significant ($S(x_t) = -\log P(x_t \mid x_{<t}).$8) AUC drop. The paper uses this finding to support the claim that each statistic is meaningful and non-redundant. This suggests that the framework’s explanatory value is not merely post hoc; rather, the design of the detector is itself tied to semantically interpretable measurements over surprisal dynamics (Basani et al., 23 Sep 2025).

6. Robustness, augmentation, and limitations

The paper characterizes DivEye as highly robust to paraphrasing, adversarial mutations, and cross-model or cross-domain perturbations. The listed attack settings include character-level modifications such as homoglyph, misspelling, zero-width, and casing; word-level perturbations such as synonym, paraphrase, insertion, and deletion; distributional perturbations such as sampling temperature shifts; commercial paraphrasers including ZeroGPT, GPTinf, and QuillBot, with more than 80% accuracy across all; and prompt-based obfuscation through creative writing prompts, where DivEye outperforms Binoculars by 18–20% accuracy (Basani et al., 23 Sep 2025).

The salient point offered in the summary is that there is no meaningful performance degradation across these attacks. The reason given is that DivEye relies on global and structural linguistic signatures rather than surface-level artifacts that can be erased by local edits or paraphrases. This is also the rationale for its use as an augmentor: when concatenated with outputs from detectors such as RADAR, Binoculars, or BiScope and fed to a meta-classifier, DivEye’s features consistently boost metrics by up to 18.7%, and feature importance in the joint models indicates that DivEye usually provides a strong, often primary, signal (Basani et al., 23 Sep 2025).

The limitations stated in the summary are narrow but consequential. Effectiveness may reduce on very short texts because there are insufficient statistics. The paper also notes that targeted distribution-matching attacks are theoretically possible, though described as practically non-trivial to implement against the diversity-centric measures used in DivEye. A plausible implication is that the framework’s strongest regime is medium- to long-form text where second-order temporal structure can be estimated reliably, but the explicit claim remains only that very short texts are more challenging and that targeted attacks remain a theoretical possibility (Basani et al., 23 Sep 2025).