Multi-Level Contextual Token Relation Modeling for Machine-Generated Text Detection

Abstract: Machine-generated texts (MGTs) pose risks such as disinformation and phishing, underscoring the need for reliable detection. Metric-based methods, which extract statistically distinguishable features of MGTs, are often more practical than complex model-based methods that are prone to overfitting. Given their diverse designs, we first place representative metric-based methods within a unified framework, enabling a clear assessment of their advantages and limitations. Our analysis identifies a core challenge across these methods: the token-level detection score is easily biased by the inherent randomness of the MGTs generation process. Then, we theoretically derive the multi-hop transitions of the token-level detection score and explore their local and global relations. Based on these findings, we propose a multi-level contextual token relation modeling framework for MGT detection. Specifically, for local relations, we model them through a lightweight Markov-informed calibration module that refines token-level evidence before aggregation. For global relations, we introduce a rule-support reasoning module that uses explicit logical rules derived from contextual score statistics. Finally, we combine the local calibrated score and the global rule-support reasoning signal in a joint multi-level inference framework. Extensive experiments show broad and substantial improvements across various real-world scenarios, including cross-LLM and cross-domain settings, with low computational overhead.

• The paper presents a novel multi-level detection framework that integrates local Markov calibration and global rule-support reasoning to counter token-level noise.
• It demonstrates substantial improvements in AUROC and TPR@FPR-1% across diverse datasets and adversarial scenarios.
• The method offers robust generalization, practical interpretability, and minimal computational overhead for real-world machine-generated text detection.

Multi-Level Contextual Token Relation Modeling for Machine-Generated Text Detection

Introduction and Problem Setting

The proliferation of high-capacity LLMs has rendered the detection of machine-generated text (MGT) indispensable for combating disinformation, IP violations, and automated phishing. Conventional watermark-based methods require intrusive model modifications and are largely impractical in black-box scenarios. Model-based (detector-classifier) approaches suffer from overfitting and reduced generalization, especially for out-of-distribution and cross-domain cases. Metric-based methods—exploiting the intrinsic statistical signals in generation (e.g., log-likelihood, entropy, log-rank)—are inherently more robust but remain limited by the high variance and stochasticity in token-level scores induced by autoregressive sampling.

The authors identify a foundational weakness shared by most metric-based detectors: token-level scores are individually noisy and aggregation fails to compensate for dependencies and systematic biases introduced by the generation process. Addressing this, the paper introduces a comprehensive multi-level context-aware modeling framework for enhanced MGT detection, operationalized with a Markov-inspired local calibration and a symbolic global rule-support reasoning module.

Unified Analysis of Metric-Based Detection

Metric-based detectors, irrespective of the specific signal (likelihood, entropy, log-rank, perturbation-based probes such as DetectGPT, DNA-DetectLLM), conform to a shared two-stage pipeline: generate token-level scores (direct or via auxiliary perturbed/rewritten texts), and aggregate these into a text-level statistic for thresholding. The authors systematically demonstrate that most improvements in the literature target either auxiliary data generation or modified aggregation rules, but do not address the correlation and instability inherent to token-level evidence.

This unified view reveals that error propagation occurs due to three main factors:

• Token-level score randomness: The generation process induces nontrivial stochasticity, especially at initial positions.
• Aggregation limitations: Direct averaging or thresholding cannot recover lost contextual information.
• Contextual dependencies: LLMs induce strong statistical associations between neighbors and global text structure.

Theoretical and Empirical Analysis of Contextual Token Relations

The authors formalize the local and global dependencies among detection scores using a tractable, single-layer Transformer analysis. The core theoretical results can be summarized as:

• Local Smoothness: The absolute difference between adjacent token scores is tightly bounded, implying strong neighbor similarity. The initial tokens manifest heightened instability due to smaller context windows (Figure 1, Figure 2).

  Figure 1: Mean absolute difference in detection scores as a function of token hop in Essay dataset, highlighting rapid locality decay.

  

  Figure 2: Mean absolute difference trajectories for adjacent tokens across positions; instability is concentrated at the text onset and stabilizes later.

• Global Structural Boundaries: Extending to multi-hop differences, the score transitions between non-adjacent tokens are also bounded logarithmically in hop distance, when early tokens are ignored. Empirically, MGT exhibits lower variance in the latter-part statistics of detection scores, higher adjacent-difference stability, and more constrained long-range transitions, compared to HGT (Figure 3).

  Figure 3: Comprehensive visualization of global statistics—variance and mean in score, adjacent-difference, and long-range-difference—contrasting MGT and HGT; MGT is consistently more stable in later positions.

These observations motivate separate modeling strategies for local versus global relations, supplementing any base metric-based detector.

Multi-Level Contextual Relation Modeling Framework

The proposed framework incorporates two orthogonal enhancement strategies (Figure 4):

Figure 4: The overall workflow: raw detector output passes through local Markov calibration and global rule-support reasoning before final fusion.

Local Relation Modeling: Markov-Informed Calibration

Local smoothness and initial instability are encoded via a pairwise Markov random field (MRF), where each token is a node with binary labels (MGT/HGT) and edge potentials penalize label differences for neighbors. The pairwise weights are modulated by a position-dependent function (sigmoid ramp), suppressing the effect of unreliable early tokens. Inference is executed via mean-field updates, yielding locally calibrated scores that are subsequently aggregated.

Global Relation Modeling: Rule-Support Reasoning

Global context is captured by constructing explicit logical rules over binned global statistics derived from calibrated token scores (e.g., latter-part variance, adjacent or long-range differences). For each text, statistics are bucketized; the activated pattern corresponds to a conjunction rule, whose empirical MGT-support is computed from the training data—integrating evidence that is robust to local perturbations or adversarial attacks.

Fusion is performed by a linear combination of the locally calibrated score and global rule-support confidence, with the fusion hyperparameter tuned on validation data. This joint inference mechanism is both computationally efficient (complexity $O(NT + NM + MK)$) and detector-agnostic.

Experimental Results

The framework is instantiated over a diverse suite of recent metric-based detectors (Likelihood, Log-Rank, Entropy, DetectGPT, FastDetectGPT, Binoculars, FourierGPT, AdaDetectGPT, DNA-DetectLLM), with systematic evaluation across four high-impact datasets: Essay, Reuters, TruthfulQA, and DetectRL (which contains adversarial and mixed-text subsets).

The key results are:

• Substantial Gains in Detection Accuracy: Across all LLMs and domains, both TPR@FPR-1% and AUROC substantially increase post-enhancement, with the multi-level model often producing double-digit absolute improvements (Tables with detailed numbers in the text).
• Robust Generalization and Transfer: The framework yields consistent cross-LLM and cross-domain generalization, supporting deployment against unknown generators (Figure 5).

  Figure 5: Relative improvement in AUROC over original detectors across training/test LLMs for DetectLLM base.

• Extreme Robustness to Paraphrasing and Adversarial Attacks: Both paraphrase-rewritten and character-level adversarial texts are detected with markedly higher AUROC, demonstrating resilience to state-of-the-art evasion tactics (Figures 7 and 8).

  Figure 6: AUROC under Dipper and Polish paraphrasing—multi-level modeling consistently dominates original and single-level baselines.

  

  Figure 7: Adversarial resilience: AUROC on character-perturbed texts for all detectors.

• Mixed-Text Detection: The framework remains effective when machine and human content are blended within a document, outperforming prior approaches in this challenging regime (Figure 8).

  Figure 8: AUROC for various proportions of mixed human/AItokens in text.

• Global Rule Interpretability: Visualization demonstrates that rule-support scores are maximized in regions with higher global stability in detection statistics, confirming the theoretical predictions (Figure 9).

  Figure 9: Empirical MGT support as a function of atomized global statistics—rules encoding tight stability reliably indicate MGT.

Ablation, Sensitivity, and Runtime Analysis

Ablation experiments verify that both local Markov calibration and the newly introduced global rule-support module are independently beneficial, with maximal performance when fused (Figure 10). Sensitivity analyses reveal stable performance for moderate hyperparameter choices (Figure 11, Figure 12), and measured runtime overhead is negligible relative to detector scoring (runtime table in the text).

Figure 10: Ablation results demonstrating the necessity of both local and global components for peak AUROC.

Implications and Outlook

This framework provides a principled, practical, and modular approach to closing the gap in MGT detection performance caused by stochasticity and contextual dependencies. By explicitly modeling both local smoothness and global structural statistics in detection scores, it enhances the efficacy and reliability of all token-based metric detectors.

Theoretical implications include: demonstrating the value of tractable surrogate analyses (single-layer attention) for downstream detection theory, and motivating further work on global context modeling in unsupervised detection.

Practically, the method supports deployment in high-stakes environments where model internals are inaccessible and data domains are heterogeneous or adversarially manipulated.

Anticipated research directions include adversarially robust calibration under score perturbation attacks designed to disrupt global stability, as well as the extension of symbolic rule-support reasoning to more complex statistics and multimodal settings.

Conclusion

This work identifies and fills a critical gap in the MGT detector stack by introducing context-sensitive multi-level modeling. The integration of lightweight local MRF calibration and explicit global rule-support reasoning delivers substantial improvements for a broad class of detectors, with theoretical support, strong empirical validation, negligible computational overhead, and high interpretability. The approach sets a new standard for robust, cross-domain, cross-model, and adversarially resilient MGT detection.