---
title: 'ARForensics: AR Image Forensics Benchmark'
url: https://www.emergentmind.com/topics/arforensics
type: topic
---

# ARForensics: AR Image Forensics Benchmark

ARForensics is a benchmark dataset for the forensic detection of images generated by visual autoregressive (AR) models. It was introduced to make the problem of distinguishing **real images from AR-generated images** concrete and measurable, in a setting where existing synthetic-image forensics benchmarks and detectors had been developed mainly for **GANs** and later **diffusion models**, but not for **visual autoregressive generators**. ARForensics is presented as the **first benchmark specifically targeted at visual autoregressive image forensics**, and its design emphasizes cross-model generalization, cross-paradigm transfer, and robustness under realistic post-processing [2510.05891].

## 1. Definition and benchmark scope

ARForensics targets a specific forensic problem: **detection of autoregressive-generated images versus real images**. Its motivating premise is that visual AR systems generate images through **discrete visual tokens** rather than continuous pixel-space synthesis or iterative denoising trajectories, and therefore leave forensic traces that differ from those exploited by prior detectors. The benchmark is built to test whether detectors can recover a forensic principle of AR generation rather than memorize the artifacts of a single model.

The dataset is organized around **7 representative mainstream visual autoregressive models**: **LlamaGen, VAR, Infinity, Janus-Pro, RAR, Switti, and Open-MAGVIT2**. These models span different AR design families, including **token-based** and **scale-based** autoregressive generation, and they also differ in discretization strategies and codebook-related design choices. This diversity is central to the benchmark’s stated purpose, because the underlying claim is that autoregressive generators share a common forensic signature in their **discrete latent/token statistics**, even when their generation order, scale hierarchy, or prompting mode differs.

The benchmark is category-aligned with ImageNet classes and supports labels at least for **real vs. fake** and **source generator identity** at test time. It therefore supports both standard binary detection and more structured analyses of detector behavior across generators.

## 2. Dataset construction and partitioning

ARForensics contains **152,000 real images and 152,000 generated images**. The real images come from **ImageNet**, which is described as large, manually annotated, and quality controlled. The generated side is produced by the seven AR generators.

Its construction protocol is explicit. The dataset is split into training, validation, and test subsets, with independently sampled real images across subsets to avoid overlap-driven bias.

| Split | Real images | Generated images |
|---|---:|---:|
| Training | 100,000 | 100,000 LlamaGen-generated |
| Validation | 10,000 | 10,000 |
| Test | 42,000 | 42,000 |

The training set consists of **100,000 LlamaGen-generated images** paired with **100,000 ImageNet real images**, with the real subset sampled as **100 images per category**. The validation set contains **10,000 image pairs**. The test set includes **6,000 generated samples from each of the 7 AR models**, balanced with corresponding ImageNet test samples, yielding **42,000 fake images** and **42,000 real images**.

Generation protocols differ by model type. **Infinity, Janus-Pro, and Switti** are treated as text-to-image models and are prompted with the fixed template **“A photo of [class]”**, where the class is the ImageNet label. **LlamaGen, VAR, RAR, and Open-MAGVIT2** use their **ImageNet-pretrained class-conditional versions**. This keeps generated images semantically aligned with ImageNet classes while preserving model diversity [2510.05891].

## 3. Forensic rationale: discrete tokenization and codebook bias

The forensic rationale for ARForensics is tied to the image formation process of visual autoregressive models. A typical visual AR pipeline first encodes an image into a continuous latent map, then **vector-quantizes** that map using a learned **codebook**, and finally predicts the next token or next scale autoregressively. Because generation is constrained by a **finite discrete vocabulary of visual tokens**, the paper argues that AR-generated images exhibit a forensic trace that is distinct from GAN upsampling/frequency artifacts and diffusion denoising traces.

Three sources of AR-specific forensic traces are repeatedly emphasized. First, **finite codebook capacity** cannot perfectly capture the long-tailed richness of natural images. Second, during VQ training, many visually distinct but infrequent patterns are mapped into the same higher-frequency codebook entries, producing a latent-space compression bias. Third, AR sampling strategies such as **top-k** and **top-p** explicitly truncate the long tail of token probabilities during generation.

The resulting claim is that generated images tend to use codebook tokens in a more **concentrated**, less natural manner than real images. Real data is described as having a more pronounced **long-tail token distribution**, whereas AR-generated samples place more probability mass in a smaller set of high-frequency codebook entries. In qualitative terms, real images are said to have relatively balanced codebook utilization, while generated samples show **severe polarization**: high-frequency entries become abnormally dominant and low-frequency regions are underused.

This makes ARForensics more than a collection of images. It is a benchmark explicitly designed around the proposition that autoregressive image generation leaves forensic evidence in **discrete latent representation**, especially in **codebook frequency distribution bias** rather than only in superficial pixel-space texture.

## 4. Supported tasks and evaluation protocol

The most direct task supported by ARForensics is **binary real-vs-fake detection**. More importantly, its train/test protocol directly measures **cross-model generalization within the AR family**. Training is performed only on **LlamaGen-generated images** versus ImageNet real images, while testing is performed on all seven AR models. This setting asks whether a detector trained on one AR generator can recover **generator-agnostic AR forensic cues**.

The benchmark also functions as a source dataset for **cross-paradigm transfer**. The associated detector is trained on ARForensics and then evaluated on external **GAN** and **diffusion** datasets, so the benchmark supports analysis of how AR-trained forensic features transfer beyond the AR family. In addition, ARForensics is used under image degradations such as **JPEG compression** and **center cropping with resizing**, so it also supports **robustness testing under realistic post-processing**.

The evaluation protocol uses standard binary forensic metrics: **Accuracy (Acc.)** with a threshold of 0.5 and **Average Precision (AP)** over decision thresholds. This makes ARForensics comparable to established forensic detection benchmarks while preserving a detection regime specialized to visual autoregressive generators.

## 5. Baseline method and benchmarked performance

ARForensics was introduced together with **D\(^3\)QE**, short for **Discrete Distribution Discrepancy-aware Quantization Error**, a detector built around the claim that AR-generated images leave traces in **quantization error** and in **real-vs-fake codebook frequency discrepancies**. The method uses a frozen **LlamaGen visual tokenizer** with **16× downsampling** and a **codebook size of 16,384**, computes the quantization residual
\[
\hat{z} = (z_q - z) \in \mathbb{R}^{h \times w \times c},
\]
tracks global training-time codebook statistics for real and fake images, and injects their normalized discrepancy prior into a **Discrete Distribution Discrepancy-Aware Transformer** through **Discrete Distribution Discrepancy-Aware Self-Attention**. It also fuses discrepancy-aware latent features with frozen **CLIP-ViT** semantic features [2510.05891].

On ARForensics, D\(^3\)QE achieves the reported best mean performance among the listed baselines, with **82.11% accuracy** and **92.07% AP** averaged over the seven AR models. The per-model results are:

- **LlamaGen**: 97.19 Acc / 99.43 AP  
- **VAR**: 85.33 / 95.30  
- **Infinity**: 62.88 / 79.39  
- **Janus-Pro**: 92.28 / 97.53  
- **RAR**: 91.69 / 97.77  
- **Switti**: 75.31 / 89.09  
- **Open-MAGVIT2**: 70.08 / 85.98  

The strongest baseline overall is **UnivFD** with **79.56% accuracy** and **90.44% AP**, so D\(^3\)QE improves the mean by about **2.55 points in Acc** and **1.63 points in AP**. The paper specifically highlights the gain on **VAR**, where D\(^3\)QE reaches **85.33%** accuracy versus **80.53%** for UnivFD.

The ablation results clarify the benchmark’s intended forensic signal. A **CLIP-only** model reaches **79.56%** accuracy. Adding latent residual information without transformer-based modeling yields **79.92%**. Using discrete quantized features \(z_q\) with a vanilla transformer gives **80.39%**; using residual quantization features \(\hat{z}\) with a vanilla transformer improves this to **80.72%**. Replacing the vanilla transformer with **D\(^3\)AT** raises accuracy to **82.11%**. This supports two conclusions stated in the paper: **quantization residuals are more diagnostic than raw discrete tokens**, and **distribution-aware attention** is the main mechanism turning those residuals into a more generalizable forensic signal.

Under robustness evaluation on ARForensics, D\(^3\)QE maintains **AP above 85%** under **JPEG quality 60**, and keeps **over 80% AP** even under severe cropping with **\(f=0.5\)**. The reported interpretation is that the detector degrades more gracefully than prior methods because its cues are not tied only to fragile pixel artifacts.

## 6. Significance, downstream use, and limitations

ARForensics is significant not only because of scale, but because of its benchmark design. It isolates a detection regime in which training and testing span different AR architectures, including token-based, scale-based, and randomized-order generators. This suggests that the benchmark is intended to test whether a detector has learned a **forensic principle of AR generation** rather than an architecture-specific artifact.

Its downstream use reinforces that role. **UniGenDet**, a later unified generative-discriminative framework for generated-image detection, evaluates in a **zero-shot** setting on ARForensics and reports **98.1** mean accuracy, with per-generator accuracies of **89.4** for LlamaGen, **99.7** for VAR, **99.9** for Infinity, **99.7** for Janus-Pro, **99.5** for RAR, **98.8** for Switti, and **99.5** for OM2. In the same comparison, **D\(^3\)QE** is reported at **82.1** mean accuracy and **FakeVLM** at **97.1**, making ARForensics a useful stress test for zero-shot generalization to modern visual autoregressive generators [2604.21904].

The benchmark also has implicit limitations. Training is based on one tokenizer—the **LlamaGen visual tokenizer**—and one training generator, **LlamaGen**, so transfer still depends on the cross-model stability of tokenizer-induced statistics. Performance is uneven across AR models: **Infinity** remains relatively difficult at **62.88% accuracy** under D\(^3\)QE, indicating that not all AR generators expose the same degree of detectable discrete bias. More broadly, as visual AR models evolve, especially if they reduce or eliminate vector quantization, the exact forensic cues emphasized by ARForensics may need revision.

Within the scope defined by current tokenized visual AR systems, however, ARForensics establishes a benchmark centered on a specific forensic thesis: autoregressive image generators leave traces in their **discrete latent token statistics**, and those traces can be studied through controlled cross-model evaluation, cross-paradigm transfer, and robustness testing.

Source: https://www.emergentmind.com/topics/arforensics