---
title: 'AITDNA: AI Text Detection Under Natural Assumptions'
url: https://www.emergentmind.com/topics/aitdna
type: topic
---

# AITDNA: AI Text Detection Under Natural Assumptions

AITDNA, short for **AI Text Detection under Natural Assumptions**, is a benchmark and dataset for evaluating AI-generated text detection under realistic human–machine co-writing conditions rather than simplified synthetic assumptions. It was introduced to address a central problem in AI text detection research: the field often treats “AI-generated text” as if it were a single, stable object, while actual deployment scenarios differ in normative standard, provenance assumptions, and attacker model [2606.04906]. AITDNA is therefore designed as a **notion-agnostic**, provenance-rich corpus of naturally human–AI co-created texts with recorded edit histories, prompts, model responses, and token-level authorship information, enabling multiple formal detection notions to be projected onto the same underlying texts [2606.04906].

## 1. Conceptual scope and motivation

AITDNA is grounded in the claim that current AI-generated text detection benchmarks frequently encode hidden assumptions that are too restrictive for real use [2606.04906]. Typical assumptions include **document-level en-bloc generation**, **synthetic mixing**, **no policy distinction**, and **no provenance ambiguity**. Under such assumptions, a document is often treated as either wholly human or wholly AI, or AI content is inserted into human text in a controlled artificial way. AITDNA is built against that background, focusing instead on iterative and collaborative writing in which a human may draft text, request continuation, request revision, accept some model outputs, reject others, and repeat [2606.04906].

This design responds to a broader definitional issue: the paper argues that **“AI-generated text” is not a single well-defined object** [2606.04906]. Depending on the application, the relevant target may be an entire AI-written document, AI passages inside human text, AI-written sentences, AI content violating a content policy, AI content violating an intent policy, text matching a reference population, or text matching a particular author’s prior writing. AITDNA was introduced to support evaluation across these distinct notions using texts produced through actual human–LLM interaction rather than post hoc synthetic construction [2606.04906].

A plausible implication is that benchmark performance without explicit notion specification can be misleading. If a detector is evaluated on a dataset whose labels reflect only one hidden interpretation of “AI text,” its reported quality may not transfer to another deployment setting with different granularity or policy constraints. This implication is central to the benchmark’s design and evaluation agenda [2606.04906].

## 2. Formalization: genesis, notions, and policy-sensitive labeling

A core construct in AITDNA is **genesis**, defined as the recorded origin and history of each token in a text [2606.04906]. At the token level, genesis assigns labels from $\{H, AI, MX\}$, where **H** denotes human-originated tokens, **AI** denotes AI-originated tokens, and **MX** denotes mixed origin, such as human text corrected or transformed by AI. For tokens labeled $AI$ or $MX$, the dataset also records prompt provenance, linking those tokens to the underlying prompt that contributed to their generation [2606.04906]. The benchmark treats genesis as an **objective trace of what happened**, rather than as a normative decision about whether the observed AI use should count as disallowed or harmful.

On top of genesis, the paper defines a **notion** as a mapping from tokens to binary labels $\{H, AI\}$ through segmentation and labeling functions [2606.04906]. The segmentation function $B$ partitions a text into connected spans, and the labeling function $M$ assigns a detection label to each span. For genesis-based notions, the paper introduces a threshold $\tau \in [0,1]$ and labels a span as AI if its AI-token ratio exceeds $\tau$ [2606.04906]. Conceptually, this converts token-level provenance into span-level decisions, making it possible to study document-level, sentence-level, and boundary-level settings within a common formal scheme.

For policy-sensitive variants, the paper defines **content-based AITD** and **intent-based AITD** by combining genesis-based labeling with a policy function $\rho$ [2606.04906]. In these notions, a span is labeled AI only if it is AI-generated under the genesis criterion and violates the relevant policy. In content-based AITD, the policy depends on the span’s content; in intent-based AITD, it depends on the underlying prompt. This separates factual provenance from policy judgment and makes explicit that not every instance of AI assistance is necessarily treated as equally problematic [2606.04906].

The framework distinguishes two broad families of notions:

- **Genesis-based notions**: document-level AITD, boundary-level AITD, sentence-level AITD, intent-based AITD, and content-based AITD [2606.04906].
- **Population-based notions**: membership-based AITD and authorship-ID-based AITD, which judge whether text matches a reference population rather than whether it is AI-originated in an absolute sense [2606.04906].

This separation is methodologically important because it prevents the benchmark from collapsing fundamentally different tasks into a single undifferentiated detection problem.

## 3. Dataset design and provenance capture

AITDNA is described as **notion-agnostic** because it was not collected to fit one specific labeling scheme [2606.04906]. Instead, it captures sufficient interaction and edit provenance to allow different notions to be instantiated afterward. The dataset was collected through a dedicated writing interface based on **CARE**, where participants co-wrote with an LLM in a text editor using two main interaction modes: **Continuation** and **Revision** [2606.04906]. In continuation mode, a participant selects a position and the LLM continues the text from that point, optionally guided by a prompt. In revision mode, a participant selects a span and the LLM revises that span, again optionally guided by a prompt. The proposed output is displayed next to the editor, and the participant may accept or reject it [2606.04906].

The benchmark’s most distinctive technical asset is its provenance logging. It records the **entire edit history**, **all insertions and deletions**, **all user prompts**, **all LLM responses**, **interaction traces over time**, **token-level authorship labels**, and **token-level prompt provenance** [2606.04906]. From these traces, the authors derive per-token authorship and prompt information, making it possible to reconstruct genesis at fine granularity. This design is substantially different from datasets constructed only from final text snapshots, because it retains the latent process by which a document was produced.

The benchmark thereby supports analyses at multiple levels of granularity. A document can be evaluated as a whole; spans can be segmented by sentences or boundaries; content and prompts can be linked to policy-based decisions; and, in principle, author-specific reference populations can be defined for authorship-style comparisons. This suggests that AITDNA functions not merely as a corpus but as a structured observational substrate for studying the interaction between provenance, policy, and detection under realistic assumptions [2606.04906].

## 4. Collection protocol and corpus composition

The collection protocol involved multiple writing sessions in which participants typically spent about **90 minutes**, targeting **350–450 words per text** [2606.04906]. Each session included **one human-only text** and **three human–LLM co-created texts**. The human-only condition served both as a purely human reference set and as an interface familiarization phase [2606.04906].

The writing scenarios sampled from **argumentative**, **creative**, and **explanatory** tasks, and the study also included **two scientific-writing sessions** with scientific argumentative writing, scientific explanatory writing, impact statement or creative-style scientific writing, and peer review [2606.04906]. Co-writing used one of five models: **Llama4-Scout**, **Qwen2.5:7B**, **DeepSeek-V3.2 (Non-Thinking Mode)**, **GPT-5.2**, and **Gemini-3-flash**, with temperatures **0 or 1** [2606.04906].

Participants comprised **99 participants** across **8 sessions**, all with at least **B1 English proficiency**; scientific scenarios used postgraduate or PhD students, while non-scientific scenarios involved crowd workers filtered for essay-writing performance [2606.04906]. To maintain data quality, the authors removed texts containing copy-paste from external sources when copied spans exceeded **60 characters** from outside the interface [2606.04906].

The final corpus statistics reported for AITDNA are as follows:

| Property | Value |
|---|---:|
| Texts collected | 452 |
| Texts remaining after filtering | 362 |
| Human-only samples | 95 |
| Human–AI co-written share | 74% |
| Essay-style share | 71% |
| Scientific setting share | 29% |
| Average length | 459 words |

These figures place AITDNA in a middle regime of human–AI mixing rather than at either extreme of fully human or fully synthetic AI-only composition [2606.04906].

The paper also compares AITDNA with earlier datasets including **CoAuthor**, **SenDetEx**, **BD**, **Mixset**, and **DetectRL** [2606.04906]. Two diagnostics are highlighted: average AI-token ratio and number of human–AI boundaries. AITDNA is reported to have **50.91% AI tokens** and **23.33 boundaries**, compared with **24.09% / 20.32** for CoAuthor, **30.96% / 13.57** for SenDetEx, **65.68% / 2.75** for BD, **64.57% / 31.51** for Mixset, and **63.69% / 50.33** for DetectRL [2606.04906]. The paper interprets BD, Mixset, and DetectRL as more synthetic and assumption-heavy, while AITDNA is positioned as a neutral reference point because it is based on natural interaction rather than adversarial or one-shot generation [2606.04906].

## 5. Benchmarking methodology and empirical findings

AITDNA is used to benchmark eight detectors under multiple notions of AI-generated text [2606.04906]. The evaluated detectors are **Min-K**, **Likelihood**, **Log Rank**, **Binoculars**, **modernBERT-ai-detection-raid-mage (moBERT)**, **FastDetectGPT**, and the proprietary systems **GPTZero** and **Pangram** [2606.04906]. For GPTZero and Pangram, non-binary outputs are mapped into $\{H, AI\}$ by treating anything not explicitly labeled $H$ as $AI$ [2606.04906]. Reported metrics are **AUROC**, **F1-score**, and **False Positive Rate (FPR)** [2606.04906].

The benchmark comprises three main experimental protocols: evaluation of all detectors on all notions within AITDNA, comparison across datasets under a fixed notion, and analysis of performance as the threshold $\tau$ varies [2606.04906]. For population-based notions, authorship-ID-based AITD could not be meaningfully evaluated because most participants contributed only one text [2606.04906].

Several findings are emphasized. First, **notion choice strongly changes detector performance** [2606.04906]. On AITDNA, **document-level AITD** is the easiest setting, whereas **sentence-level** and **boundary-level** settings are harder, and **content-based** and **intent-based** AITD are the hardest for most detectors [2606.04906]. The reported interpretation is that current detectors mostly exploit surface-level statistical cues and are comparatively weak when the target depends on semantic intent or policy violation rather than purely stylistic or distributional evidence [2606.04906].

Second, **higher granularity lowers performance** [2606.04906]. The paper notes that this observation is consistent with earlier work, but argues that the difficulty is not only a matter of granularity; it also reflects whether the target notion aligns with the detector’s implicit assumptions. Third, **false positive rates on hybrid texts are often high** [2606.04906]. For many detectors, human–AI co-written documents are over-labeled as AI, which has direct deployment relevance because benign AI-assisted writing may be misclassified as malicious or disallowed [2606.04906].

Fourth, **no detector is best across all datasets** [2606.04906]. With $\tau = 0.5$ for document-level evaluation, **Log Rank** is best on AITDNA, **Likelihood** is best on CoAuthor, and **moBERT** is strongest on DetectRL and BD [2606.04906]. The paper uses this result to argue that detector rankings are highly sensitive to dataset assumptions and hidden label distributions. Fifth, **threshold choice matters substantially** [2606.04906]. Varying $\tau$ from **0.1 to 0.9** changes the recall–FPR balance: low $\tau$ gives higher recall but can induce very high FPR, whereas high $\tau$ lowers FPR but sharply reduces F1 [2606.04906]. The explicit conclusion is that $\tau$ must be stated, because otherwise evaluation results are not directly comparable across studies [2606.04906].

## 6. Interpretation, limitations, and significance for AI text detection

AITDNA’s principal significance lies in redefining evaluation around explicit assumptions rather than around a single implicit notion of AI-generated text [2606.04906]. The benchmark shows that deployment-relevant questions are inseparable from three dimensions: the **normative standard**, the **genesis assumptions**, and the **attacker model**. As a result, a detector that performs well on synthetic whole-document AI-versus-human classification may not perform well on boundary localization, policy-based screening, or mixed-authorship settings [2606.04906].

The paper is also explicit about limitations. Data were collected in a **lab setting** rather than in a real-world application such as ChatGPT, and conversational multi-turn interaction styles common in the wild are absent [2606.04906]. The dataset is **relatively small**, which limits its use as a large-scale training resource and positions it primarily as an analysis and evaluation benchmark [2606.04906]. **Authorship-ID-based AITD** could not be fully tested because of insufficient multi-sample-per-author coverage [2606.04906]. The copy-paste filter may be **conservative**, potentially discarding valid examples [2606.04906]. Finally, **population-based**, **content-based**, and **intent-based** notions remain underexplored relative to more conventional document-level settings [2606.04906].

The broader implication is that AI text detection should not be framed as a monolithic binary classification problem. AITDNA instead encourages explicit declaration of what is being detected, under which threshold, with which provenance assumptions, and for which policy context [2606.04906]. This suggests a shift from the coarse question “Was this text written by AI?” toward the more technically precise question of whether a text or span should be treated as AI-generated under a specified notion and operational standard. In that sense, AITDNA is both a benchmark and a formal intervention in the methodology of AI-generated text detection [2606.04906].

Source: https://www.emergentmind.com/topics/aitdna