CodeGPTSensor+: Adversarial Code Provenance

• CodeGPTSensor+ is an adversarially trained detector that differentiates LLM-generated code from human-written code even when subtle, semantics-preserving edits are applied.
• It employs the MIST module to generate realistic adversarial samples via identifier renaming and structure transformation for robustness-oriented fine-tuning.
• The enhanced model maintains high performance on pristine code while significantly boosting detection accuracy on adversarially modified samples.

CodeGPTSensor+ is an adversarially trained detector for distinguishing LLM-generated code from human-written code when the generated code has undergone subtle, semantics-preserving modification. It is introduced as an enhanced version of CodeGPTSensor and retains the original provenance-classification objective while changing the training regime: adversarial samples generated by MIST, a Multi-objective Identifier and Structure Transformation module, are mixed with the original training data and used for robustness-oriented fine-tuning. The resulting model is evaluated on the original HMCorp test sets and on HMCorp-adv, an adversarially modified counterpart, with the stated goal of maintaining high performance on pristine code while resisting realistic manual edits such as identifier renaming and structure rewrites (Yin et al., 17 Jul 2025).

1. Problem setting and nomenclature

The problem addressed by CodeGPTSensor+ is not generic AI-generated text detection, but code provenance detection under perturbation. The underlying premise is that detecting LLM-generated code is a different problem from detecting LLM-generated natural language, because code has strong syntax, structure, naming, and semantic constraints. Existing AIGC detectors designed for natural language do not transfer well to code, and the original CodeGPTSensor work argues that naive natural-language detectors are unreliable for this setting (Xu et al., 2024).

This challenge is amplified by the fact that LLM-generated code is often manually edited before submission or publication. The adversarial-training paper states that state-of-the-art LLM-generated code detectors are brittle under realistic, small edits: identifiers are renamed, loops are rewritten, branches are refactored, and constants are wrapped in variables, while the program logic remains unchanged. CodeGPTSensor+ is the response to that operational setting, not a new provenance task (Yin et al., 17 Jul 2025).

A common source of confusion is historical. The original CodeGPTSensor paper explicitly states that it does not introduce a system named CodeGPTSensor+; the only close naming variants in that paper are CodeGPTSensor and CodeGPTSensor w/o Contrast (Xu et al., 2024). Accordingly, CodeGPTSensor+ is properly associated with the later adversarially trained extension rather than with the 2024 base model.

2. Baseline lineage: CodeGPTSensor and HMCorp

CodeGPTSensor+ inherits its detector family from CodeGPTSensor, which is a binary classifier that predicts whether a function is human-written or generated by ChatGPT. CodeGPTSensor is code-oriented rather than a generic text detector, and its core architecture combines UniXcoder as the semantic encoder with a contrastive learning training scheme that pulls same-class embeddings closer and pushes different-class embeddings apart (Xu et al., 2024).

The original system uses UniXcoder in encoder-only mode by setting the prefix to [\mathit{Enc}], and, following the UniXcoder protocol, keeps only the AST leaves corresponding to source code during fine-tuning. A projector maps encoder outputs to embeddings $E_f$, $E_{f^+}$, and $E_{f^-}$, where $f^+$ is from the same class and $f^-$ is from a different class. Positive pairs are constructed by passing the same input twice through UniXcoder with different dropout masks, using dropout probability 0.1. The optimization objective is described as

$$\max(||E_{f} - E_{f^+}|| - ||E_f - E_{f^-}|| + \epsilon, 0)$$

with margin $\epsilon = 1$ by default, after which a classification layer and softmax produce class probabilities, and a threshold of 0.5 yields the final binary decision (Xu et al., 2024).

The data substrate for both the baseline and the extension is HMCorp, the Human and Machine comparison Corpus. HMCorp contains 550K pairs of aligned human-written and ChatGPT-generated code, specifically 288K Python code pairs and 222K Java code pairs. The human-written side is derived from CodeSearchNet after noise removal and docstring-length filtering; the machine-generated side is obtained by prompting gpt-3.5-turbo with docstrings as functional requirements via the OpenAI API in April 2023, followed by response cleaning and retention of only successful generations. The dataset is split 8:1:1 into train, validation, and test (Xu et al., 2024).

The baseline’s empirical motivation is strong. On HMCorp, CodeGPTSensor achieves Accuracy 0.992, F1 0.992, AUC 0.999 for Python and Accuracy 0.967, F1 0.968, AUC 0.995 for Java, outperforming selected baselines. At the same time, the same paper shows why robustness is necessary: humans perform at essentially chance in distinguishing code origin, with 50% accuracy for Python and 49.75% accuracy for Java (Xu et al., 2024).

3. Adversarial training pipeline in CodeGPTSensor+

CodeGPTSensor+ is not a detector architecture from scratch; it is an enhanced training pipeline around the original CodeGPTSensor classifier. The pipeline is defined in five stages: train the original detector on the HMCorp training set to obtain baseline classifier $M$; generate adversarially modified code samples using MIST; augment the training set by mixing original and adversarial samples; fine-tune the detector on the augmented data to obtain robust model $M^+$; and evaluate both CodeGPTSensor and CodeGPTSensor+ on the original HMCorp test set and on HMCorp-adv, which is produced by applying MIST to the original test set (Yin et al., 17 Jul 2025).

The adversarial training objective is standard fine-tuning on an augmented dataset. Let $D$ denote the original training set, $E_{f^+}$0 the adversarial samples, and $E_{f^+}$1 the mixed dataset. Adversarial examples are assigned the same labels as their originals, and the paper uses the usual cross-entropy loss:

$E_{f^+}$2

The robustness mechanism is therefore attributed to adversarial data augmentation rather than to a specialized robust-loss term (Yin et al., 17 Jul 2025).

The mixing policy is explicit. During fine-tuning, the augmented training set contains 70% original samples and 30% adversarial samples. To control computational cost, only 10% of the original training set is randomly selected as the target pool for adversarial generation. For each selected sample, if MIST successfully flips the model prediction, the first successful adversarial sample is kept; otherwise, the sample that minimizes the model’s confidence in the true class the most is retained (Yin et al., 17 Jul 2025).

The reported fine-tuning setup is also fixed: classification threshold 0.5, batch size 8, learning rate 2e-5, optimizer Adam, and fine-tuning epochs 1. The experiments are implemented in PyTorch and run on a workstation with an Intel Xeon Gold 6226R CPU, 192 GB RAM, 10 × NVIDIA RTX 3090 GPUs, and Ubuntu 20.04.1 LTS (Yin et al., 17 Jul 2025).

4. MIST: adversarial sample generation by identifier and structure transformation

MIST, the Multi-objective Identifier and Structure Transformation module, is the key mechanism that differentiates CodeGPTSensor+ from the baseline. It is designed to better resemble real manual code edits than prior attacks by combining identifier replacement and code structure transformation, and by optimizing them jointly under the objectives of attack success, semantic consistency, and small perturbation magnitude (Yin et al., 17 Jul 2025).

At the search level, MIST is implemented as an evolutionary procedure. Given input code $E_{f^+}$3, population size $E_{f^+}$4, and maximum iterations $E_{f^+}$5, it initializes a population of mutated samples, then repeatedly applies crossover between randomly chosen parents, mutation to offspring, and NSGA-II non-dominated sorting for selection. The best $E_{f^+}$6 samples are preserved at each generation. Mutation is probabilistic:

$E_{f^+}$7

and if $E_{f^+}$8, then

$E_{f^+}$9

otherwise

$E_{f^-}$0

with identifier-renaming rate $E_{f^-}$1 in experiments (Yin et al., 17 Jul 2025).

Identifier transformation proceeds in two stages. First, identifiers are extracted using Tree-sitter and ranked by importance. For identifier $E_{f^-}$2, the paper replaces that identifier with <UNK> to obtain $E_{f^-}$3, then computes

$E_{f^-}$4

Identifiers with larger $E_{f^-}$5 are treated as more influential, and the probability of selecting identifier $E_{f^-}$6 is

$E_{f^-}$7

Second, after selecting an identifier, MIST uses CodeT5 to generate context-aware replacement names. The chosen identifier is masked with <extra_id_0>, tokenized with T5Tokenizer, fed into CodeT5, and one candidate is randomly chosen from the top-$E_{f^-}$8 predictions, with $E_{f^-}$9. The replacement must satisfy language naming rules and must not duplicate an existing identifier in the snippet (Yin et al., 17 Jul 2025).

Structure transformation supplies semantics-preserving rewrites beyond renaming. The paper groups these into three categories. Control-flow equivalence transformations include loop conversion for $f^+$0 while and branch conversion if-else $f^+$1 if-if. Expression equivalence transformations include rewrites such as j-- $f^+$2 j = j - 1 and x += y $f^+$3 x = x + y, together with analogous operators like -=, *=, /=, %=, <<=, and >>=. Constant/variable equivalence transformations include replacing print("Hello, World!") with a variable binding followed by print(message). The paper explicitly excludes dead-code insertion because it hurts naturalness (Yin et al., 17 Jul 2025).

A distinctive feature of MIST is its style-aware probabilistic transformation policy. It builds a style reference table from HMCorp by splitting the training data by language and source, then counting how frequently original structure $f^+$4 and transformed structure $f^+$5 occur for each rule. Transformation probabilities are derived from

$f^+$6

When attacking a sample, the rule probability is taken from the target source class’s style distribution; for example, a Java LLM-generated sample may be transformed according to statistics observed in Java human-written code, with the aim of drifting the sample toward human style (Yin et al., 17 Jul 2025).

The optimization is multi-objective. For adversarial sample $f^+$7, the objectives are

$f^+$8

which minimizes confidence in the true label,

$f^+$9

which constrains semantic drift, and

$f^-$0

which constrains perturbation magnitude. The overall problem is

$f^-$1

and NSGA-II is used instead of a single scalarized objective (Yin et al., 17 Jul 2025).

5. Empirical behavior on original and adversarial code

The central empirical claim is that CodeGPTSensor+ preserves baseline performance on unmodified code while substantially improving robustness on adversarially modified code. On the original HMCorp test sets, the performance difference is minimal: for HMCorp-Java, CodeGPTSensor reports Accuracy 0.967, F1 0.968, AUC 0.995, while CodeGPTSensor+ reports Accuracy 0.970, F1 0.970, AUC 0.996; for HMCorp-Python, both report Accuracy 0.992, F1 0.992, AUC 0.999 (Yin et al., 17 Jul 2025).

On HMCorp-adv, the gap is large.

Setting	CodeGPTSensor	CodeGPTSensor+
Java-adv Accuracy / F1 / AUC	0.325 / 0.120 / 0.822	0.944 / 0.961 / 0.983
Python-adv Accuracy / F1 / AUC	0.260 / 0.412 / 0.140	0.969 / 0.960 / 0.995

For HMCorp-Java-adv, CodeGPTSensor records Recall 0.065 and Precision 0.785, whereas CodeGPTSensor+ records Recall 0.974 and Precision 0.949. For HMCorp-Python-adv, the corresponding values are Recall 0.694, Precision 0.293 for CodeGPTSensor and Recall 0.979, Precision 0.941 for CodeGPTSensor+. The paper reports relative improvements of +190.8% Accuracy, +699.6% F1, and +19.7% AUC on Java adversarial samples, and +272.8% Accuracy, +132.8% F1, and +608.9% AUC on Python adversarial samples (Yin et al., 17 Jul 2025).

The paper also evaluates MIST as an attack generator against CodeGPTSensor and GPTSniffer, comparing it with ALERT, MOAA, and CODA using ASR, AMQ, ICR, SD, and ED. Against CodeGPTSensor on Java, MIST achieves ASR 53.80, ICR 9.25, SD 0.53, ED 94.68, AMQ 58.87; on Python, it achieves ASR 60.15, ICR 6.14, SD 0.42, ED 144.48, AMQ 49.14. These figures are presented as the best or near-best tradeoff among the compared attack methods (Yin et al., 17 Jul 2025).

The ablation and robustness analysis attributes the gains to the quality of adversarial samples rather than to generic noise injection. Three findings are emphasized: matching train and test attack styles usually yields the highest accuracy; multi-objective adversarial examples are harder than single-objective ones; and MIST-generated samples are especially effective for adversarial training because they combine attack success with low perturbation and realistic, semantically consistent rewrites. This suggests that CodeGPTSensor+ benefits from exposure to transformations that resemble actual human cleanup and disguise behavior (Yin et al., 17 Jul 2025).

6. Scope, limitations, and related research uses of the term

The scope of CodeGPTSensor+ is explicitly limited. The experiments only study Java and Python, and the generated code originates from a specific ChatGPT version, gpt-3.5-turbo from April 2023. The transformation space mainly covers identifier renaming and equivalent loop, branch, expression, and constant-style rewrites; it does not model more complex real-world edits such as function splitting, introducing third-party libraries, heavier refactoring, or larger semantic reorganizations (Yin et al., 17 Jul 2025).

These limitations connect directly to the baseline’s own assumptions. CodeGPTSensor and HMCorp are based on open-source GitHub projects, and the original paper notes that results may not transfer directly to other programming languages, commercial codebases, newer versions of ChatGPT, or other LLMs. That paper also emphasizes that ChatGPT-generated code can be semantically very close to human code while differing only in naming style, implementation pattern, control-flow shape, verbosity, and use of library calls, which is precisely why brittle provenance detectors are vulnerable to small modifications (Xu et al., 2024).

A broader research context shows that code-origin detection is feasible but method-dependent. One study frames the task as supervised binary classification over Python code using black-box embeddings, white-box formatting features, GMMs, and an interpretable Bayes classifier, with the strongest result reported as XGB + TF-IDF at 98.28% accuracy and 99.84% AUC on unseen problems; the same study reports human performance at 48.7% ± 14.7 accuracy, effectively random guessing (Oedingen et al., 2024). This suggests that CodeGPTSensor+ occupies a specific subproblem within a larger literature: robust code-origin detection under adversarially modified input rather than provenance detection on pristine samples alone.

Separately, one interpretive usage associates the name with a CGEMs-based sensing and evaluation framework for GPT-3-generated code quality rather than provenance. In that usage, CGEMs denotes Code Generation Evaluation Metrics, combining compilation, functionality, edits, sequence similarity, execution time, coverage, static code metrics, and NLP-style overlap metrics, then feeding these features into an ANN and using LIME for interpretability. The best-performing model in that study reports 76.92% test accuracy and 55.56% F1 for acceptable/not acceptable quality classification on 80 GPT-3-generated Python programs (Narasimhan et al., 2021). This suggests that “CodeGPTSensor+” may appear in two different conceptual registers: formally, as the adversarially trained extension of CodeGPTSensor; and interpretively, as a broader sensing framework for generated-code assessment.