DetectAnyLLM: Unified LLM Detection Framework

Updated 2 July 2026

DetectAnyLLM is a unified framework integrating discrepancy learning, stylometric analysis, and fingerprinting to detect machine-generated text across diverse scenarios.
It employs innovations such as Direct Discrepancy Learning and LoRA-based fingerprinting to achieve high AUROC and robust detection under adversarial settings.
The framework supports both binary human-versus-machine classification and fine-grained model attribution with scalable, domain-agnostic methodologies.

DetectAnyLLM is a unified framework and family of methodologies for the detection and attribution of machine-generated text, specifically targeting output from LLMs. These methods are designed to enhance robustness and generalization across domains, model families, and adversarial manipulations. Solutions under the DetectAnyLLM banner combine innovations in discrepancy learning, model fingerprinting, statistical anomaly detection, adversarially robust architectures, and stylometric analysis—addressing both the binary human-vs-machine classification task and fine-grained model/source attribution. State-of-the-art variants achieve high area under the ROC curve and significant robustness improvements over prior detectors in challenging multi-domain and adversarial settings (Fu et al., 15 Sep 2025).

1. Motivation and Problem Landscape

The proliferation of advanced LLMs has intensified challenges across misinformation detection, academic integrity, intellectual property protection, and the control of synthetic content. The core task, machine-generated text detection (MGTD), must now contend with:

Increasing human-likeness of LLM outputs, especially under paraphrasing, code revision, and adversarial finetuning (Fu et al., 15 Sep 2025, Nguyen-Son et al., 23 Jan 2026, Park et al., 25 Feb 2025, Henrique et al., 2023).
Failure modes of previous approaches: traditional zero-shot detectors (e.g., entropy, log-rank, DetectGPT) are brittle to shifts in domain or style, while supervision-based discriminators often overfit to generator distributions rather than task-oriented separation (Fu et al., 15 Sep 2025, Henrique et al., 2023).
Complex attack scenarios, including adaptive paraphrasing, evasion via critic-guided fine-tuning, and adversarially manipulated training (Henrique et al., 2023, Nguyen-Son et al., 23 Jan 2026).
The necessity of both “open-world” detection (robust discrimination without seeing the generator at train time) and fine-grained attribution (identifying the responsible model from among many possible LLMs) (Fu et al., 27 Jan 2025, Wu et al., 2023).

2. Core Direct Discrepancy Learning Approach

DetectAnyLLM (Fu et al., 15 Sep 2025) introduces Direct Discrepancy Learning (DDL), an objective that aligns model training directly with the discrimination of machine-generated (MGT) versus human-written text (HWT). The framework comprises:

Scoring Backbone: A causal LLM (e.g., GPT-Neo-2.7B) with a LoRA adapter (rank=8, α=32, dropout=0.1), serving both for perturbation sampling and detection.
Perturbation and Discrepancy Calculation:

1. For input $x$ , draw $n$ perturbations $\tilde{x}_i$ via stochastic resampling. 2. Compute the normalized log-probability discrepancy:

$d_c(x) = \frac{\log f_\theta(x|x) - \mũ}{\sigmã},$

where $\mũ$ and $\sigmã^2$ are the mean and variance of $\log f_\theta(\tilde{x}|x)$ under the perturbation sampler. 3. Reference clustering: Compute $d_c$ on small labeled reference sets (MGT and HWT), and output $p_m(x)$ based on local density.

Training Objective: DDL removes KL-regularization, instead minimizing:

$\mathcal{L}_{DDL} = \mathbb{E}_{x_h,x_m}\left[|d_c(x_h)| + |\gamma - d_c(x_m)|\right],$

pushing $n$ 0 and $n$ 1 for margin hyperparameter $n$ 2.

Empirical Outcomes: On the MIRAGE benchmark, DetectAnyLLM achieves AUROC ≈ 0.952 and Balanced Accuracy ≈ 0.897–0.906 across 17 LLMs and 5 domains, with robust performance under domain shifts and paraphrasing/rephrasing attacks. The training is efficient (reduced time and memory vs. prior objectives) and generalizes well to unseen models. Stable detection performance holds for a wide γ-range, and removing σ normalization impairs robustness by 10–15 AUROC points (Fu et al., 15 Sep 2025).

3. Fine-Grained Attribution and Model Fingerprinting

DetectAnyLLM incorporates advanced fingerprinting for model attribution across black-box APIs and heterogeneous LLMs. The FDLLM pipeline (Fu et al., 27 Jan 2025) and its envisioned DetectAnyLLM extension feature:

Data Construction: Multilingual/multidomain prompts generate 90,000 samples from 20 diverse LLMs (FD-Dataset).
LoRA-based Fingerprinting: Insert low-rank adapters— $n$ 3—into a foundation encoder (Qwen2.5-7B), with only $n$ 4 trainable to capture distributional quirks of each model.
Classification: Multiclass head infers the responsible model, with a separability regularizer encouraging centroid distance between model embeddings.
Generalization and Robustness: Macro-F1 of 91.1% vs. 74.4% baseline, 95% accuracy on unseen LLMs, and 2× lower adversarial attack success rates under paraphrasing/translation/synonym attacks. Continual learning via adapter updates supports expansion as new LLMs are released (Fu et al., 27 Jan 2025).

4. Specialized Detection Modules and Extensions

DetectAnyLLM is not limited to a single detection paradigm but supports integration of orthogonal modules:

Textual and Stylometric Detection: Hybrid deep learning stacks (embedding, Bi-LSTM, Transformer, CNN) combined with Unicode normalization and aggressive preprocessing yield precision/recall/F1 ≈ 0.99 on medium-scale real-world corpora (Mo et al., 2024).
Anomaly/Token-Level Detection: Black-box anomaly detection by scanning the vocabulary for low-confidence single-token predictions identifies misaligned or anomalous tokens (e.g., under API-only access), using metrics such as confidence, gap, top-K entropy, and tail probability for efficient detection (Witold, 2024).
Paraphrase and Attribution in Code: Stylometric classifiers on code leverage features for naming consistency, indentation structure, and readability, enabling fast and accurate detection of LLM-paraphrased code and LLM-source attribution (F1 up to 96%, $n$ 5200× baseline speed) (Park et al., 25 Feb 2025).
Paraphrase Detection via Web Search and Regeneration: The SearchLLM module uses search engine retrieval and similarity boosting upon paraphrased regeneration, robustly flagging LLM-paraphrased text under both supervised and zero-shot regimes (AUC increases of 0.13–0.41) (Nguyen-Son et al., 23 Jan 2026).
Hallucination Detection via Layerwise Information Deficiency: By analyzing cross-layer information transmission and aggregating usable information differences ( $n$ 6), DetectAnyLLM reliably detects output likely to be hallucinated—even in an entirely training-free, model-agnostic setting (Kim et al., 2024).

5. Security, Adversarial Robustness, and Practical Considerations

The DetectAnyLLM design philosophy directly tackles key vulnerabilities identified in recent works (Henrique et al., 2023):

Adversarial Resistance: Ensembles over multiple detection signals (perplexity under several LMs, stylometric features, semantic coherence, watermarks) prevent single-critic fine-tuned evasion (Henrique et al., 2023). Detectors are adversarially trained on both standard and fine-tuned outputs.
Reference Privacy and Evolving Coverage: Human reference corpora are rotated or credential-protected; inferences do not require access to full LLMs, only precomputed statistics or adapters (Wu et al., 2023, Fu et al., 27 Jan 2025).
Black-Box and API-Only Scenarios: Dedicated fingerprinting (FDLLM) and anomaly frameworks operate without needing internal LLM access, supporting extension to proprietary and closed-source models (Fu et al., 27 Jan 2025, Witold, 2024).
Efficiency: Techniques such as proxy perplexity dictionaries and raw stylometric feature MLPs deliver substantial ( $n$ 7– $n$ 8) speedup versus edit-distance or embedding-based baselines (Wu et al., 2023, Park et al., 25 Feb 2025).

6. Benchmarking and Empirical Outcomes

DetectAnyLLM models are evaluated on the MIRAGE benchmark, which encompasses 93.8K text pairs from 10 human corpora, 5 domains, and 17 LLMs, along with specialized datasets for code, anomaly, and hallucination detection.

Method/Setting	Macro-F1 / AUROC	Robustness / Notes
DDL on MIRAGE	0.952 (AUROC)	+66% vs. previous best (ImBD)
FDLLM (20-way attrib.)	91.1% (Macro-F1)	95% generalization to new LLMs
LPcodedec (code attrib.)	up to 96% (F1)	1,343 $n$ 9 speed over Tree-ED
Hallucination (𝓛I)	+5–15 AUROC over baselines	Model-agnostic, unsupervised
LLMDet	P=98.54%, R=99.00%	Fast, dictionary-based, extensible

Underlying all empirical reports is a consistent pattern: Discrepancy-based, task-aligned (rather than MLE/KL) objectives and lightweight stylometric or LoRA-adapter fingerprinting provide both domain-agnostic and instance-specific robustness (Fu et al., 15 Sep 2025, Fu et al., 27 Jan 2025, Park et al., 25 Feb 2025, Wu et al., 2023).

7. Limitations, Open Challenges, and Future Directions

While DetectAnyLLM advances the state-of-the-art in both detection performance and adversarial resistance, several open frontiers remain:

Multilingual and Cross-Domain Transfer: Most approaches are validated on English; robust cross-lingual expansion is required (Wu et al., 2023, Fu et al., 27 Jan 2025).
Short Texts and Edge Cases: Very short sequences remain challenging for both classifier- and perplexity-based modules (Wu et al., 2023, Fu et al., 15 Sep 2025).
Fine-Tuned/Instruction-Tuned Variants: n-gram/proxy-dist patterns drift, necessitating periodic adaptation (Wu et al., 2023).
Reference Dependence: Reference clustering and calibration require small labeled sets; eliminating this need or developing open-set detectors remains an active direction (Fu et al., 15 Sep 2025).
Hallucination Thresholding: $\tilde{x}_i$ 0 thresholds vary by model and domain; unsupervised calibration techniques and adaptive weighting strategies are future work (Kim et al., 2024).
Complex Hiding/Evasion in Structured Formats: Hidden prompt detection (e.g., PhantomLint) demonstrates that adversarial document engineering can subvert visual inspection and must be paired with model-layer detection (Murray, 25 Aug 2025).

Promising avenues include continual learning for evolving LLMs, integration of auxiliary (syntactic, semantic, watermark) modalities, and hierarchical ensemble frameworks to maintain detection efficacy in an adversarial, rapidly changing LLM ecosystem.