---
title: 'CodeGPTSensor+: Adversarial Code Provenance'
url: https://www.emergentmind.com/topics/codegptsensor
type: topic
---

# CodeGPTSensor+: Adversarial Code Provenance

CodeGPTSensor+ is an adversarially trained detector for distinguishing LLM-generated code from human-written code when the generated code has undergone subtle, semantics-preserving modification. It is introduced as an enhanced version of CodeGPTSensor and retains the original provenance-classification objective while changing the training regime: adversarial samples generated by MIST, a Multi-objective Identifier and Structure Transformation module, are mixed with the original training data and used for robustness-oriented fine-tuning. The resulting model is evaluated on the original HMCorp test sets and on HMCorp-adv, an adversarially modified counterpart, with the stated goal of maintaining high performance on pristine code while resisting realistic manual edits such as identifier renaming and structure rewrites [2507.13123].

## 1. Problem setting and nomenclature

The problem addressed by CodeGPTSensor+ is not generic AI-generated text detection, but code provenance detection under perturbation. The underlying premise is that detecting LLM-generated code is a different problem from detecting LLM-generated natural language, because code has strong syntax, structure, naming, and semantic constraints. Existing AIGC detectors designed for natural language do not transfer well to code, and the original CodeGPTSensor work argues that naive natural-language detectors are unreliable for this setting [2411.04704].

This challenge is amplified by the fact that LLM-generated code is often manually edited before submission or publication. The adversarial-training paper states that state-of-the-art LLM-generated code detectors are brittle under realistic, small edits: identifiers are renamed, loops are rewritten, branches are refactored, and constants are wrapped in variables, while the program logic remains unchanged. CodeGPTSensor+ is the response to that operational setting, not a new provenance task [2507.13123].

A common source of confusion is historical. The original CodeGPTSensor paper explicitly states that it does **not** introduce a system named CodeGPTSensor+; the only close naming variants in that paper are CodeGPTSensor and CodeGPTSensor w/o Contrast [2411.04704]. Accordingly, CodeGPTSensor+ is properly associated with the later adversarially trained extension rather than with the 2024 base model.

## 2. Baseline lineage: CodeGPTSensor and HMCorp

CodeGPTSensor+ inherits its detector family from CodeGPTSensor, which is a binary classifier that predicts whether a function is human-written or generated by ChatGPT. CodeGPTSensor is code-oriented rather than a generic text detector, and its core architecture combines UniXcoder as the semantic encoder with a contrastive learning training scheme that pulls same-class embeddings closer and pushes different-class embeddings apart [2411.04704].

The original system uses UniXcoder in encoder-only mode by setting the prefix to `[\mathit{Enc}]`, and, following the UniXcoder protocol, keeps only the AST leaves corresponding to source code during fine-tuning. A projector maps encoder outputs to embeddings \(E_f\), \(E_{f^+}\), and \(E_{f^-}\), where \(f^+\) is from the same class and \(f^-\) is from a different class. Positive pairs are constructed by passing the same input twice through UniXcoder with different dropout masks, using dropout probability **0.1**. The optimization objective is described as
$$
\max(||E_{f} - E_{f^+}|| - ||E_f - E_{f^-}|| + \epsilon, 0)
$$
with margin \(\epsilon = 1\) by default, after which a classification layer and softmax produce class probabilities, and a threshold of **0.5** yields the final binary decision [2411.04704].

The data substrate for both the baseline and the extension is HMCorp, the Human and Machine comparison Corpus. HMCorp contains **550K pairs** of aligned human-written and ChatGPT-generated code, specifically **288K Python code pairs** and **222K Java code pairs**. The human-written side is derived from CodeSearchNet after noise removal and docstring-length filtering; the machine-generated side is obtained by prompting **`gpt-3.5-turbo`** with docstrings as functional requirements via the OpenAI API in **April 2023**, followed by response cleaning and retention of only successful generations. The dataset is split **8:1:1** into train, validation, and test [2411.04704].

The baseline’s empirical motivation is strong. On HMCorp, CodeGPTSensor achieves **Accuracy 0.992, F1 0.992, AUC 0.999** for Python and **Accuracy 0.967, F1 0.968, AUC 0.995** for Java, outperforming selected baselines. At the same time, the same paper shows why robustness is necessary: humans perform at essentially chance in distinguishing code origin, with **50% accuracy for Python** and **49.75% accuracy for Java** [2411.04704].

## 3. Adversarial training pipeline in CodeGPTSensor+

CodeGPTSensor+ is not a detector architecture from scratch; it is an enhanced training pipeline around the original CodeGPTSensor classifier. The pipeline is defined in five stages: train the original detector on the HMCorp training set to obtain baseline classifier \(M\); generate adversarially modified code samples using MIST; augment the training set by mixing original and adversarial samples; fine-tune the detector on the augmented data to obtain robust model \(M^+\); and evaluate both CodeGPTSensor and CodeGPTSensor+ on the original HMCorp test set and on HMCorp-adv, which is produced by applying MIST to the original test set [2507.13123].

The adversarial training objective is standard fine-tuning on an augmented dataset. Let \(D\) denote the original training set, \(D^{adv}\) the adversarial samples, and \(D^+\) the mixed dataset. Adversarial examples are assigned the same labels as their originals, and the paper uses the usual cross-entropy loss:
$$
\mathcal{L}_{ce} = -\sum_i y_i \log(\hat{y}_i).
$$
The robustness mechanism is therefore attributed to adversarial data augmentation rather than to a specialized robust-loss term [2507.13123].

The mixing policy is explicit. During fine-tuning, the augmented training set contains **70% original samples** and **30% adversarial samples**. To control computational cost, only **10% of the original training set** is randomly selected as the target pool for adversarial generation. For each selected sample, if MIST successfully flips the model prediction, the **first successful adversarial sample** is kept; otherwise, the sample that minimizes the model’s confidence in the true class the most is retained [2507.13123].

The reported fine-tuning setup is also fixed: classification threshold **0.5**, batch size **8**, learning rate **2e-5**, optimizer **Adam**, and fine-tuning epochs **1**. The experiments are implemented in PyTorch and run on a workstation with an Intel Xeon Gold 6226R CPU, **192 GB RAM**, **10 × NVIDIA RTX 3090 GPUs**, and Ubuntu 20.04.1 LTS [2507.13123].

## 4. MIST: adversarial sample generation by identifier and structure transformation

MIST, the Multi-objective Identifier and Structure Transformation module, is the key mechanism that differentiates CodeGPTSensor+ from the baseline. It is designed to better resemble real manual code edits than prior attacks by combining **identifier replacement** and **code structure transformation**, and by optimizing them jointly under the objectives of attack success, semantic consistency, and small perturbation magnitude [2507.13123].

At the search level, MIST is implemented as an evolutionary procedure. Given input code \(x\), population size \(N\), and maximum iterations \(\text{max\_iter}\), it initializes a population of mutated samples, then repeatedly applies crossover between randomly chosen parents, mutation to offspring, and **NSGA-II non-dominated sorting** for selection. The best \(N\) samples are preserved at each generation. Mutation is probabilistic:
$$
p \sim \mathcal{U}(0,1)
$$
and if \(p < r\), then
$$
x_{new} = \text{rename\_identifier}(x),
$$
otherwise
$$
x_{new} = \text{transform\_structure}(x),
$$
with identifier-renaming rate \(r = 0.5\) in experiments [2507.13123].

Identifier transformation proceeds in two stages. First, identifiers are extracted using Tree-sitter and ranked by importance. For identifier \(i\), the paper replaces that identifier with `<UNK>` to obtain \(x_i^*\), then computes
$$
IS_i = M(x)[y_{\text{truth}] - M(x_i^*)[y_{\text{truth}] .
$$
Identifiers with larger \(IS_i\) are treated as more influential, and the probability of selecting identifier \(i\) is
$$
P(i) = \frac{IS_i}{\sum_{j=1}^{n} IS_j}.
$$
Second, after selecting an identifier, MIST uses **CodeT5** to generate context-aware replacement names. The chosen identifier is masked with `<extra_id_0>`, tokenized with T5Tokenizer, fed into CodeT5, and one candidate is randomly chosen from the top-\(k\) predictions, with \(k = 40\). The replacement must satisfy language naming rules and must not duplicate an existing identifier in the snippet [2507.13123].

Structure transformation supplies semantics-preserving rewrites beyond renaming. The paper groups these into three categories. **Control-flow equivalence transformations** include loop conversion `for` \(\leftrightarrow\) `while` and branch conversion `if-else` \(\leftrightarrow\) `if-if`. **Expression equivalence transformations** include rewrites such as `j--` \(\leftrightarrow\) `j = j - 1` and `x += y` \(\leftrightarrow\) `x = x + y`, together with analogous operators like `-=`, `*=`, `/=`, `%=`, `<<=`, and `>>=`. **Constant/variable equivalence transformations** include replacing `print("Hello, World!")` with a variable binding followed by `print(message)`. The paper explicitly excludes dead-code insertion because it hurts naturalness [2507.13123].

A distinctive feature of MIST is its style-aware probabilistic transformation policy. It builds a style reference table from HMCorp by splitting the training data by language and source, then counting how frequently original structure \(S^b\) and transformed structure \(S^a\) occur for each rule. Transformation probabilities are derived from
$$
\frac{N^a}{N^a + N^b}, \qquad \frac{N^b}{N^a + N^b}.
$$
When attacking a sample, the rule probability is taken from the target source class’s style distribution; for example, a Java LLM-generated sample may be transformed according to statistics observed in Java human-written code, with the aim of drifting the sample toward human style [2507.13123].

The optimization is multi-objective. For adversarial sample \(x^{adv}\), the objectives are
$$
f_1(x^{adv}) = M(x^{adv})[y_{\text{truth}],
$$
which minimizes confidence in the true label,
$$
f_2(x^{adv}) = \sum_{i=1}^{n} 1 - \cos(v_i, v_i^{adv}),
$$
which constrains semantic drift, and
$$
f_3(x^{adv}) = \text{EditDistance}(x, x^{adv}),
$$
which constrains perturbation magnitude. The overall problem is
$$
\min_{x^{adv}\in\mathcal{X} \left( f_1(x^{adv}), f_2(x^{adv}), f_3(x^{adv}) \right),
$$
and NSGA-II is used instead of a single scalarized objective [2507.13123].

## 5. Empirical behavior on original and adversarial code

The central empirical claim is that CodeGPTSensor+ preserves baseline performance on unmodified code while substantially improving robustness on adversarially modified code. On the original HMCorp test sets, the performance difference is minimal: for **HMCorp-Java**, CodeGPTSensor reports **Accuracy 0.967, F1 0.968, AUC 0.995**, while CodeGPTSensor+ reports **Accuracy 0.970, F1 0.970, AUC 0.996**; for **HMCorp-Python**, both report **Accuracy 0.992, F1 0.992, AUC 0.999** [2507.13123].

On HMCorp-adv, the gap is large.

| Setting | CodeGPTSensor | CodeGPTSensor+ |
|---|---:|---:|
| Java-adv Accuracy / F1 / AUC | 0.325 / 0.120 / 0.822 | 0.944 / 0.961 / 0.983 |
| Python-adv Accuracy / F1 / AUC | 0.260 / 0.412 / 0.140 | 0.969 / 0.960 / 0.995 |

For **HMCorp-Java-adv**, CodeGPTSensor records **Recall 0.065** and **Precision 0.785**, whereas CodeGPTSensor+ records **Recall 0.974** and **Precision 0.949**. For **HMCorp-Python-adv**, the corresponding values are **Recall 0.694**, **Precision 0.293** for CodeGPTSensor and **Recall 0.979**, **Precision 0.941** for CodeGPTSensor+. The paper reports relative improvements of **+190.8% Accuracy**, **+699.6% F1**, and **+19.7% AUC** on Java adversarial samples, and **+272.8% Accuracy**, **+132.8% F1**, and **+608.9% AUC** on Python adversarial samples [2507.13123].

The paper also evaluates MIST as an attack generator against CodeGPTSensor and GPTSniffer, comparing it with **ALERT**, **MOAA**, and **CODA** using **ASR**, **AMQ**, **ICR**, **SD**, and **ED**. Against CodeGPTSensor on Java, MIST achieves **ASR 53.80**, **ICR 9.25**, **SD 0.53**, **ED 94.68**, **AMQ 58.87**; on Python, it achieves **ASR 60.15**, **ICR 6.14**, **SD 0.42**, **ED 144.48**, **AMQ 49.14**. These figures are presented as the best or near-best tradeoff among the compared attack methods [2507.13123].

The ablation and robustness analysis attributes the gains to the quality of adversarial samples rather than to generic noise injection. Three findings are emphasized: matching train and test attack styles usually yields the highest accuracy; multi-objective adversarial examples are harder than single-objective ones; and MIST-generated samples are especially effective for adversarial training because they combine attack success with low perturbation and realistic, semantically consistent rewrites. This suggests that CodeGPTSensor+ benefits from exposure to transformations that resemble actual human cleanup and disguise behavior [2507.13123].

## 6. Scope, limitations, and related research uses of the term

The scope of CodeGPTSensor+ is explicitly limited. The experiments only study **Java and Python**, and the generated code originates from a specific ChatGPT version, **`gpt-3.5-turbo` from April 2023**. The transformation space mainly covers identifier renaming and equivalent loop, branch, expression, and constant-style rewrites; it does **not** model more complex real-world edits such as function splitting, introducing third-party libraries, heavier refactoring, or larger semantic reorganizations [2507.13123].

These limitations connect directly to the baseline’s own assumptions. CodeGPTSensor and HMCorp are based on open-source GitHub projects, and the original paper notes that results may not transfer directly to other programming languages, commercial codebases, newer versions of ChatGPT, or other LLMs. That paper also emphasizes that ChatGPT-generated code can be semantically very close to human code while differing only in naming style, implementation pattern, control-flow shape, verbosity, and use of library calls, which is precisely why brittle provenance detectors are vulnerable to small modifications [2411.04704].

A broader research context shows that code-origin detection is feasible but method-dependent. One study frames the task as supervised binary classification over Python code using black-box embeddings, white-box formatting features, GMMs, and an interpretable Bayes classifier, with the strongest result reported as **XGB + TF-IDF** at **98.28% accuracy** and **99.84% AUC** on unseen problems; the same study reports human performance at **48.7% ± 14.7** accuracy, effectively random guessing [2405.15512]. This suggests that CodeGPTSensor+ occupies a specific subproblem within a larger literature: robust code-origin detection under adversarially modified input rather than provenance detection on pristine samples alone.

Separately, one interpretive usage associates the name with a CGEMs-based sensing and evaluation framework for GPT-3-generated code quality rather than provenance. In that usage, **CGEMs** denotes **Code Generation Evaluation Metrics**, combining compilation, functionality, edits, sequence similarity, execution time, coverage, static code metrics, and NLP-style overlap metrics, then feeding these features into an ANN and using **LIME** for interpretability. The best-performing model in that study reports **76.92% test accuracy** and **55.56% F1** for acceptable/not acceptable quality classification on **80 GPT-3-generated Python programs** [2108.10168]. This suggests that “CodeGPTSensor+” may appear in two different conceptual registers: formally, as the adversarially trained extension of CodeGPTSensor; and interpretively, as a broader sensing framework for generated-code assessment.

Source: https://www.emergentmind.com/topics/codegptsensor