SentiDetect: Zero-Shot LLM Text Detection

• SentiDetect is a zero-shot, model-agnostic framework that distinguishes LLM-generated text by measuring the stability of sentiment distribution under controlled low-emotional rewriting.
• It employs two key metrics—sentiment distribution consistency (SDC) and sentiment distribution preservation (SDP)—to capture the differential emotional variability between machine and human text.
• Evaluated across five diverse datasets, SentiDetect demonstrates robust performance and resilience to paraphrasing and adversarial perturbations compared to existing detection methods.

SentiDetect is a model-agnostic, zero-shot framework for detecting LLM-generated text by analyzing divergence in sentiment distribution stability. Its central claim is not that machine-generated text is systematically more positive or more negative than human writing, but that LLM outputs tend to exhibit emotionally consistent patterns under transformations that reduce emotionality while preserving meaning. Human-written texts, by contrast, are described as showing greater emotional variability under the same transformations. On that basis, SentiDetect formulates binary authorship attribution through two complementary metrics—sentiment distribution consistency and sentiment distribution preservation—and reports strong gains over prior detectors across five datasets and several advanced LLMs, including Gemini-1.5-Pro, Claude-3, GPT-4-0613, and LLaMa-3.3 (Li et al., 9 Aug 2025).

1. Problem setting and conceptual basis

SentiDetect addresses authorship attribution between the human distribution $\mathcal{P}$ and an LLM distribution $\mathcal{Q}_\theta$. For a candidate text $x_n = [w_{n,1}, w_{n,2}, \ldots, w_{n,J}]$, the paper writes the decision problem as

$\hat{y}_n = \underset{ y_n \in \{y_\mathcal{P}, y_{\mathcal{Q}_\theta} \} }{\operatorname{argmax} \; P (y_n \mid x_n, N, \mathcal{P}, \mathcal{Q_\theta}).$

When $N=1$, this is single-instance detection; when multiple texts are available, evidence can be aggregated across samples (Li et al., 9 Aug 2025).

The framework is motivated by limitations attributed to watermark-based methods, supervised classifiers, and statistical detectors. In the paper’s formulation, many existing approaches either require model access, require labeled training data, or are fragile under paraphrasing, adversarial edits, and domain shift. SentiDetect instead proposes sentiment stability as a robust, model-agnostic signal. The governing hypothesis is described as a “divergence in low-emotional stability”: LLM-generated text maintains consistent sentiment distribution patterns during low-emotional rewriting, while human-written text exhibits significant shifts in sentiment expression patterns (Li et al., 9 Aug 2025).

A common misconception is that the method is a conventional sentiment classifier. The paper does not frame SentiDetect as predicting whether a text is positive, negative, or neutral in any ordinary sense. Rather, it treats the stability of a 3-class sentiment distribution under controlled transformations as the discriminative signal. This suggests that SentiDetect belongs less to ordinary sentiment analysis than to transformation-based authorship forensics (Li et al., 9 Aug 2025).

2. End-to-end detection procedure

SentiDetect proceeds in three stages. The first stage is low-emotional rewriting (LER), represented by a rewriting function

$F_1(\cdot \mid p_i),$

where $p_i$ is a prompt such as “Rewrite objectively,” “Please rewrite this more straightforwardly,” or “Polish this in a machine-like objective tone.” Applying the transformation yields

$x' = F_1(x \mid p_i).$

The paper states that multiple prompts $p_i$ are used to improve robustness, and that these prompt templates are optimized with AutoPrompt-style methods (Li et al., 9 Aug 2025).

The second stage is sentiment feature extraction. Each original or rewritten text is passed through a sentiment analyzer

$F_2(\cdot \mid p_{sa}),$

implemented in experiments as a standard 3-class sentiment classifier that outputs a probability distribution over negative, neutral, and positive. The resulting sentiment distribution is denoted

$\mathcal{Q}_\theta$0

For each text, sentiment distributions from multiple rewritten variants are collected and concatenated into a fixed-length feature representation (Li et al., 9 Aug 2025).

The third stage is stability divergence analysis. The paper defines a general divergence quantity

$\mathcal{Q}_\theta$1

with a simplified empirical form

$\mathcal{Q}_\theta$2

Classification is then performed by thresholding: $\mathcal{Q}_\theta$3 in which case the text is classified as LLM-generated (Li et al., 9 Aug 2025).

The practical significance of this design is that the detector does not depend on source-model internals, a watermark, or a task-specific fine-tuned classifier. The paper explicitly characterizes the method as requiring no fine-tuned detector, no access to the source model internals, and no watermark (Li et al., 9 Aug 2025).

3. Sentiment Distribution Consistency and Sentiment Distribution Preservation

The framework is organized around two metrics: sentiment distribution consistency (SDC) and sentiment distribution preservation (SDP). The paper treats them as distinct rather than collapsing them into a single score, because combining them yielded only marginal gains while reducing interpretability (Li et al., 9 Aug 2025).

SDC measures how much the sentiment distribution changes when a text is rewritten into a low-emotional, semantically preserved version. The paper defines it as

$\mathcal{Q}_\theta$4

Conceptually, SDC asks how invariant the sentiment distribution remains under low-emotional rewriting. The paper argues that LLM-generated text tends to have smaller changes here, whereas human text tends to shift more (Li et al., 9 Aug 2025).

SDP measures whether the sentiment distribution is preserved under forward and inverse semantic-preserving transformations. The paper introduces opposite transformation functions $\mathcal{Q}_\theta$5 and $\mathcal{Q}_\theta$6 and defines

$\mathcal{Q}_\theta$7

Operationally, SDP starts from the original text, applies low-emotional rewriting, applies a forward semantic-preserving transformation, then applies the inverse transformation, and finally compares the sentiment distribution of the final text to the original. The paper reports that the semantic fidelity of the inverse mapping pairs was validated using BLEURT (Li et al., 9 Aug 2025).

The two metrics target different invariances. SDC captures stability under low-emotional rewriting itself, whereas SDP captures preservation through a transformation cycle. The paper’s interpretation is that both reflect the same underlying principle: LLM outputs are less emotionally volatile than human text under these operations (Li et al., 9 Aug 2025).

4. Evaluation protocol, datasets, and baselines

SentiDetect is evaluated on five datasets spanning different domains. The paper lists News, HumanEval, Student Essay, Yelp Review, and Paper Abstract. News contains 5,000 real news articles from 50 journalists, with AI-generated versions produced via a two-stage title-to-article process. HumanEval contains 164 programming tasks with signatures, docstrings, and tests. Student Essay consists of high school and university-level papers from BAWE paired with LLM-generated counterparts. Yelp Review contains 1,000 human-written Yelp reviews with machine-generated reviews of similar length. Paper Abstract contains 500 ACL 2023–2024 abstracts and synthetic versions generated from the first 15 words (Li et al., 9 Aug 2025).

Dataset	Description	Notes
News	5,000 real news articles from 50 journalists	AI-generated via two-stage title-to-article process
HumanEval	164 programming tasks	Includes signatures, docstrings, and tests
Student Essay	BAWE essays with LLM-generated counterparts	High school and university-level papers
Yelp Review	1,000 human-written Yelp reviews	Machine-generated reviews of similar length
Paper Abstract	500 ACL 2023–2024 abstracts	Synthetic versions from the first 15 words

The evaluated source models include GPT-4-0613, Gemini-1.5-Pro, and LLaMa-3.3, and the abstract also mentions Claude-3 (Li et al., 9 Aug 2025). Baselines comprise LogRank (GPT-2), RoBERTa-base, RoBERTa-large, GPTZero, DetectGPT, Ghostbuster, RAIDAR, Binoculars, and R-Detect (Li et al., 9 Aug 2025).

The paper’s evaluation focus is robustness across domains and perturbations rather than narrowly optimized in-domain accuracy. That emphasis aligns SentiDetect with a broader recent shift in LLM-text detection toward out-of-domain generalization. A closely related but methodologically different example is SENTRA, which models selected-next-token-probability sequences with a Transformer and contrastive pre-training, and likewise emphasizes out-of-domain and out-of-LLM robustness (Plyler et al., 15 Sep 2025).

5. Reported empirical performance

The paper reports that SentiDetect consistently outperforms all baselines across the five datasets. For GPT-4-0613-generated text, it reports F1 improvements of over 11.90% on one variant of the method, up to 15.26 points on individual datasets in some settings, and average F1 around 71.19 or 70.89 depending on the reported variant. For Gemini-1.5-Pro-generated text, the abstract states over 16% F1 improvement, and the paper reports strong gains on datasets such as HumanEval and Yelp, with one reported average around 71.65 and another around 70.39. Against open-source LLaMa-3.3 outputs, it reports average F1 around 85.17 for one variant and 83.84 for the other, with around 13-point average F1 improvement over existing methods (Li et al., 9 Aug 2025).

The most notable per-dataset gains are described for HumanEval, where improvements reach around 10–15 F1 points over the best baseline, and for Yelp Review, where gains are often around 8–11 points. Paper Abstract also shows robust improvements, while News and Student Essay show smaller but still consistent gains (Li et al., 9 Aug 2025).

The paper also reports strong robustness under adversarial perturbation. Using TextAttack-style perturbations, many baselines are said to suffer performance losses often exceeding 70%, whereas SentiDetect degrades much less. SentiDetect-SDC is reported as having the best average robustness under attack, with SentiDetect-SDP also remaining strong (Li et al., 9 Aug 2025).

Under paraphrasing, the authors rewrite generated text using GPT-3.5-Turbo and vary the proportion of changed content. All detectors degrade as paraphrasing increases, but SentiDetect remains consistently better than the baselines and still performs well even when 30% of the content is replaced (Li et al., 9 Aug 2025).

With respect to text length, the paper reports that detection generally improves with longer inputs, but that SentiDetect remains relatively robust even for short texts and performs well even at around 20 words. For very short samples, the authors suggest aggregating multiple short texts to recover more reliable sentiment stability signals (Li et al., 9 Aug 2025).

6. Sensitivity analyses, interpretation, and practical uses

Ablation-style analysis in the paper examines the number of LER prompts $\mathcal{Q}_\theta$8. Results are reported for $\mathcal{Q}_\theta$9 on Student Essay and Yelp. Performance improves as $x_n = [w_{n,1}, w_{n,2}, \ldots, w_{n,J}]$0 increases, the gains plateau after moderate values, and $x_n = [w_{n,1}, w_{n,2}, \ldots, w_{n,J}]$1 is described as typically a good tradeoff (Li et al., 9 Aug 2025).

The paper also emphasizes interpretability. SDC and SDP are retained as separate metrics because their combination yielded marginal benefit but reduced clarity. This separation makes the detector’s decision signal easier to inspect: one may ask whether a text is stable under low-emotional rewriting, stable under transformation cycles, or both (Li et al., 9 Aug 2025).

In practical terms, the paper positions SentiDetect for content moderation, academic integrity checks, fake-news filtering, and platform-level AI-content auditing (Li et al., 9 Aug 2025). Its appeal lies in being model-agnostic and zero-shot with respect to the target generator. This distinguishes it from supervised detectors such as SENTRA, which explicitly trains a Transformer-based encoder over selected-next-token-probability sequences and uses contrastive pre-training on unlabeled data (Plyler et al., 15 Sep 2025). The contrast is methodological rather than adversarial: SentiDetect relies on transformation-induced sentiment invariance, while SENTRA relies on sequence modeling of frozen-LLM probability traces.

The interpretive claim behind SentiDetect should nevertheless be read narrowly. The paper does not argue that emotionality is intrinsically human or that all LLM outputs are affectively uniform. A more precise reading is that, under the specific low-emotional and semantic-preserving transformations employed, LLM-generated text appears to preserve its sentiment distribution more regularly than human text. This suggests a stylometric regularity induced by generation behavior rather than a universal theory of affect in language (Li et al., 9 Aug 2025).

7. Limitations and critical perspective

The paper identifies several limitations. Short texts are harder to detect because they may not contain enough sentiment structure. The method depends on the quality of low-emotional rewrites and inverse transformations, so poor rewrites can weaken the signal. Detection quality can vary with the prompt or instruction set even though prompt sets are optimized. Not every domain yields the same margin; the largest gains appear in HumanEval and Yelp, while gains are smaller in News and Essays. Finally, although the method is model-agnostic with respect to the target text generator, it still relies on LLM-based rewriting and sentiment analysis inside the detection pipeline (Li et al., 9 Aug 2025).

These limitations define the scope of SentiDetect more precisely. It is not a universal replacement for supervised or likelihood-based detectors, and it does not eliminate threshold selection or transformation sensitivity. Its contribution is instead to establish sentiment distribution stability as a usable detection signal that is comparatively resilient to paraphrasing, adversarial perturbation, and text-length variation, while avoiding dependence on source-model internals or watermarking (Li et al., 9 Aug 2025).

In that sense, SentiDetect occupies a distinct position in the LLM-detection landscape. It reframes authorship detection as a question about the invariance of sentiment distributions under controlled transformations, operationalized through SDC and SDP, and reports robust empirical performance across multiple datasets and model families. Whether viewed as a zero-shot forensic heuristic or as a transformation-based behavioral detector, its core contribution is the claim that the stability of sentiment expression can function as a discriminative fingerprint of machine-generated text (Li et al., 9 Aug 2025).