Authorship Impersonation via LLM Prompting does not Evade Authorship Verification Methods

Abstract: Authorship verification (AV), the task of determining whether a questioned text was written by a specific individual, is a critical part of forensic linguistics. While manual authorial impersonation by perpetrators has long been a recognized threat in historical forensic cases, recent advances in LLMs raise new challenges, as adversaries may exploit these tools to impersonate another's writing. This study investigates whether prompted LLMs can generate convincing authorial impersonations and whether such outputs can evade existing forensic AV systems. Using GPT-4o as the adversary model, we generated impersonation texts under four prompting conditions across three genres: emails, text messages, and social media posts. We then evaluated these outputs against both non-neural AV methods (n-gram tracing, Ranking-Based Impostors Method, LambdaG) and neural approaches (AdHominem, LUAR, STAR) within a likelihood-ratio framework. Results show that LLM-generated texts failed to sufficiently replicate authorial individuality to bypass established AV systems. We also observed that some methods achieved even higher accuracy when rejecting impersonation texts compared to genuine negative samples. Overall, these findings indicate that, despite the accessibility of LLMs, current AV systems remain robust against entry-level impersonation attempts across multiple genres. Furthermore, we demonstrate that this counter-intuitive resilience stems, at least in part, from the higher lexical diversity and entropy inherent in LLM-generated texts.

• The paper demonstrates that current authorship verification systems robustly detect LLM-generated impersonations even when advanced prompting techniques are employed.
• It employs diverse datasets and evaluation methods, showing that both neural and non-neural models assign strong negative likelihood ratios to the generated texts.
• Increased lexical diversity and entropy in LLM outputs serve as key discriminative signals, undermining the ability to mimic human idiolects effectively.

Robustness of Authorship Verification Systems Against LLM-Based Impersonation

Introduction

The paper "Authorship Impersonation via LLM Prompting does not Evade Authorship Verification Methods" (2603.29454) rigorously investigates the forensic resilience of authorship verification (AV) systems against adversarially generated texts via state-of-the-art LLM prompting. The authors simulate an adversarial scenario wherein GPT-4o is tasked with impersonating individual authors across emails, SMS/chat, and social media domains using diverse prompting strategies. The evaluation encompasses both non-neural and neural AV methods, leveraging a formal likelihood ratio (LLR) framework to scrutinize the evidential strength and discriminative reliability of verification systems confronted with LLM-driven impersonation attempts.

Experimental Design

The study selects three representative corpora—Enron emails, BOLT SMS/chat, and Twitter posts—reflecting genres frequently encountered in forensic casework. For each target author, GPT-4o receives exemplars and is prompted to rewrite a fixed source text, employing four distinct prompting modalities: naive rewriting, self-prompting, role-play (with linguistic checklists), and tree-of-thoughts (ToT) prompting with explicit planning and multi-stage internal voting.

The authorship verification pipeline evaluates the generated impersonations against six established AV methods:

• Non-neural: n-gram tracing, Ranking-Based Impostors (RBI), and LambdaG (grammar model-based likelihood ratio).
• Neural: AdHominem (attention-based Siamese), LUAR (contrastive authorship representation), and STAR (style-sensitive transformer).

POSNoise preprocessing is applied to non-neural AV models to mitigate topic bias, while neural models operate on raw text.

Results

Efficacy of LLM Impersonation

The evaluation reveals that LLM-generated impersonation texts are consistently rejected by AV systems as non-matching, regardless of genre or prompting sophistication. Across all corpora and AV approaches, LLRs overwhelmingly support the different-author hypothesis, with negative values indicating robust discrimination between genuine and impersonated texts.

Figure 1: Mean LLRs assigned to LLM-generated impersonations by six AV methods under four prompting schemes over Enron (top), BOLT (middle), and Twitter (bottom) corpora.

Among neural models, STAR and LUAR demonstrate heightened robustness, assigning definitively negative LLRs and achieving higher true negative rates (TNR) on impersonation cases than on genuine test negatives. AdHominem, while generally resilient, yields less confident results, with scores clustering near zero.

Non-neural methods (LambdaG, n-gram tracing, RBI) display comparable resilience, though LambdaG and RBI incur minor TNR degradation, suggesting occasional susceptibility, particularly in the email domain.

Prompting strategies, including highly structured and linguistically informed prompts, fail to systematically improve impersonation efficacy; the LLM is unable to consistently replicate the idiolectal features necessary to deceive verification systems.

Impersonation-Induced Performance Variation

Relative TNR degradation quantifies potential method vulnerability; most AV systems maintain or even improve rejection reliability on impersonated texts compared to genuine negatives, especially neural models.

Figure 2: TNR degradation (top) and LLR confidence drop (bottom) for AV methods when evaluating LLM impersonations versus genuine test cases.

STAR demonstrates both increased rejection accuracy and strengthened evidential support, reflected in higher-magnitude negative LLRs, particularly in the Twitter domain. These findings establish a paradoxical result: some AV systems are not only robust but exploit systematic differences in LLM outputs to facilitate detection.

Lexical Diversity and Entropy as Discriminative Signals

Analysis of compressed size, entropy, and type-token-ratio (TTR) reveals that LLM-generated impersonations consistently possess greater lexical diversity and entropy than human-authored texts across all genres.

Figure 3: Comparative analysis of compressed size (top), entropy (middle), and TTR (bottom) for LLM-generated impersonations and genuine texts.

Higher entropy and novelty in LLM outputs undermine their plausibility as idiolectally consistent writing, providing an implicit signal exploited by n-gram tracing and neural methods (notably STAR and LUAR) to distinguish between genuine and fabricated samples. These systematic divergences are not easily mitigated by prompt engineering alone, suggesting inherent limitations in few-shot LLM impersonation.

Implications and Future Directions

The demonstrated resilience of AV systems has several theoretical and practical implications:

• Forensic Reliability: AV systems retain robust discriminative power against entry-level LLM-driven impersonation, supporting their continued evidentiary validity in forensic contexts.
• Adversarial Limitations: Prompting alone cannot induce sufficient idiolectal fidelity in LLM outputs to undermine verification. Adaptive adversarial strategies (e.g., feedback-driven prompt refinement, PEFT) and parameter-level model adaptation may raise new challenges—necessitating further study.
• AI Text Detection Overlap: Neural AV models appear to leverage global stylistic properties (e.g., inflated entropy) symptomatic of LLM-generated text—a dual signal for both authorship and AI detection. Fine-grained adversarial attacks warrant investigation to probe vulnerabilities in this overlap.
• Human Deception Risk: Algorithms strongly reject LLM impersonations, but such texts may still deceive human readers, posing social-engineering risks even absent algorithmic confusion.

Future research should explore:

• Iterative adversarial prompting coupled with AV feedback loops.
• Incorporation of human-perceived authorship plausibility metrics.
• Systematic manipulation of LLM decoding parameters to modulate entropy.
• Evaluation of PEFT and full fine-tuning adversarial attacks.

Conclusion

The paper provides compelling empirical evidence that current AV methods, both neural and non-neural, successfully detect and reject LLM-driven authorship impersonations, with resilience persisting across genres and prompting strategies. Persistent limitations in lexically diverse LLM outputs prevent successful evasion of forensic verification, though algorithmic robustness cannot necessarily prevent deception of human readers. Theoretical insights on entropy and idiolectal stability underscore the discriminative mechanisms exploited by AV systems and highlight directions for adversarial escalation. Continued operational effectiveness of AV methods is affirmed, but vigilance against evolving adversarial tactics remains essential.