DFQ-ViT: Data-Free Quantization for ViTs

Updated 6 July 2026

The paper introduces DFQ-ViT, which quantizes pre-trained Vision Transformers without original data by leveraging synthetic samples and internal attention statistics.
It details strategies such as patch-similarity synthesis, semantic prompting, and activation correction to overcome the lack of BatchNorm priors inherent to ViTs.
Empirical results demonstrate that DFQ-ViT achieves low-bit quantization with minimal performance loss across tasks like classification, detection, and segmentation.

Data-Free Quantization for Vision Transformers (DFQ-ViT) denotes a line of post-training quantization methods for Vision Transformers (ViTs) that operate without access to original training or calibration images. In this setting, a pre-trained full-precision model is quantized by using synthetic samples, internal attention statistics, teacher–student matching, or generated calibration corpora instead of real data. The problem became distinct from earlier convolutional DFQ because standard zero-shot quantization pipelines rely on BatchNorm statistics, whereas ViTs typically use LayerNorm and therefore do not expose the same calibration priors. The literature has consequently evolved from patch-similarity-driven sample synthesis to adaptive minimax calibration, contrastive generation, semantic prompting, inter-head attention alignment, activation correction, learned diffusion prompts, and informative-region masking (Li et al., 2022, Li et al., 2022, Ramachandran et al., 2024, Zhong et al., 2024, Choi et al., 2024, Tong et al., 19 Jul 2025, Li et al., 21 Feb 2026, Qian et al., 3 Jun 2026).

1. Problem setting and formalization

The basic DFQ-ViT problem is to start from a pre-trained full-precision ViT and construct a low-bit quantized model that preserves the original model’s behavior without using real images. PSAQ-ViT formulates this as choosing quantized weights, activation quantizers, and a synthetic calibration set so that the approximation error between the full-precision model $P$ and quantized model $Q$ is minimized under a $k$ -bit constraint (Li et al., 2022). A related formulation in later work is to freeze the original full-precision ViT, build a quantized clone, and optimize quantization parameters jointly over intermediate features and final logits (Li et al., 21 Feb 2026).

A recurring motivation across the literature is architectural. Standard DFQ relies on BatchNorm statistics, but ViTs use LayerNorm, so BatchNorm-stored statistics are unavailable; this is stated explicitly in semantic prompting work for ViTs and is also the reason earlier BN-regularization methods do not transfer directly (Zhong et al., 2024). As a result, ViT-oriented DFQ methods typically exploit self-attention outputs, patch tokens, attention maps, or internal feature distributions as substitute priors.

Uniform quantization remains the dominant hardware-oriented backbone. One explicit formulation uses an $N$ -bit uniform quantizer with step size $\Delta$ and zero-point $z$ ,

$q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$

with per-tensor activation quantization and channel-wise weight quantization (Li et al., 21 Feb 2026). Other DFQ-ViT systems adopt symmetric MinMax quantization for weights, asymmetric quantization for activations, or uniform symmetric operators with zero-point fixed to $0$, but the central challenge is not the existence of a quantizer; it is how to calibrate that quantizer without real data (Li et al., 2022).

The scope of the area is broader than classification alone. Reported evaluations cover ImageNet-1K classification, COCO object detection, and ADE20K semantic segmentation, indicating that DFQ-ViT has been studied both as a model compression problem and as a downstream backbone deployment problem (Li et al., 2022).

2. Synthetic-sample priors and semantic modeling

The defining technical question in DFQ-ViT is how to synthesize calibration data that induces the internal statistics expected by a ViT. Different methods answer this by imposing structure on patch similarity, attention maps, local semantics, or prompt-conditioned generative outputs.

Method	Synthetic prior	Key mechanism
PSAQ-ViT (Li et al., 2022)	Patch similarity entropy	Maximize diversity of inter-patch similarities in MSA outputs
PSAQ-ViT V2 (Li et al., 2022)	Patch similarity + discrepancy	Adaptive teacher–student minimax loop with cyclic sample evolution
CLAMP-ViT (Ramachandran et al., 2024)	Patch-level contrastive learning	Contrastive generation on MHSA patch embeddings with cyclic adaptation
Semantic prompting method in (Zhong et al., 2024)	Attention priors + local semantics	Attention Priors Alignment, Multi-Semantic Reinforcement, Soft-Label Learning
MimiQ (Choi et al., 2024)	Inter-head attention similarity	Align head-wise attention outputs for each spatial query patch
DFQ-ViT (Tong et al., 19 Jul 2025)	Easy-to-Hard curriculum	Increasing-difficulty synthesis with a global–local composite loss
Joint prompt-guided PTQ (Li et al., 21 Feb 2026)	Learned diffusion prompts	Stable Diffusion Turbo conditioned on learned multi-mode prompts
MaskAQ (Qian et al., 3 Jun 2026)	Informative-region masking	Differential entropy maximization and masked attention alignment

The earliest PSAQ-ViT line is built on a specific observation: self-attention outputs on real images exhibit more diverse patch similarities than outputs on Gaussian noise. PSAQ-ViT therefore defines a patch-similarity matrix from post-MSA outputs, estimates the distribution of pairwise cosine similarities, and maximizes the corresponding differential entropy to push Gaussian noise toward “real-like” samples. PSAQ-ViT V2 preserves this patch-similarity prior but adds an adaptive teacher–student strategy in which the generated samples and quantized student evolve cyclically under supervision from the full-precision teacher. Its stated goal is to improve accuracy while remaining compatible with classification, detection, and segmentation and without relying on auxiliary category guidance (Li et al., 2022).

Later work increasingly characterizes the failure mode of synthetic images as a semantic one. The paper “Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers” states that existing DFQ methods suffer from semantic distortion and semantic inadequacy. In its detailed technical report, the method is described through three modules: Attention Priors Alignment (APA), which aligns synthetic attention maps to randomized structure priors in deeper blocks; Multi-Semantic Reinforcement (MSR), which performs localized patch optimization by cropping and resizing non-overlapping patches; and Soft-Label Learning (SL), which replaces one-hot targets with soft semantic targets to avoid semantic collapse in multi-object images (Zhong et al., 2024).

CLAMP-ViT also frames prior limitations semantically, but at patch granularity. Its first stage starts from Gaussian-noise “images,” freezes both the full-precision and quantized ViTs, extracts per-patch embeddings from each MHSA layer’s outputs, and applies a patch-level contrastive loss $L^{C_1}$ . In parallel it imposes an output loss

$L^O = (1/n_c)\bigl\|Q(X)-T_G\bigr\|_1 + \bigl\|FP(X)-T_G\bigr\|_1,$

where $Q$ 0 are random task-specific targets: one-hot for classification, box maps for detection, and pixel maps for segmentation. The total generation loss is

$Q$ 1

and is back-propagated directly into $Q$ 2, which serves as the generator (Ramachandran et al., 2024).

MimiQ shifts the emphasis from patch similarity to agreement among attention heads. For each spatial query patch and layer, it computes head-wise attention score maps and defines an inter-head similarity score using the absolute structural-similarity index (SSIM). Synthetic images are then optimized with an inter-head coherency loss, a class-matching loss, and total variation regularization. During calibration, MimiQ further aligns the quantized model’s attention maps to those of the full-precision teacher via head-wise structural attention distillation based on DSSIM (Choi et al., 2024).

The 2025 paper explicitly titled “DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning” introduces an Easy-to-Hard curriculum. It defines a cropping ratio $Q$ 3 that decreases according to a cosine schedule so that sample difficulty increases from global structure toward local detail over synthesis iterations. Its loss combines Patch-Similarity Entropy, one-hot classification loss, and total variation. This suggests a direct attempt to balance global and local structure in synthetic images, rather than optimizing a single static image prior (Tong et al., 19 Jul 2025).

Prompt-guided generation introduces a different class of prior. In joint post-training quantization for ViTs, synthetic calibration samples are produced by Stable Diffusion Turbo using learned multi-mode prompts. Each ImageNet class is assigned $Q$ 4 prompts, the first $Q$ 5 CLIP tokens are initialized from the textual class name, and the remaining $Q$ 6 are learnable context vectors. Diversity is encouraged jointly in prompt embeddings, generated image features, and attention maps (Li et al., 21 Feb 2026). MaskAQ, by contrast, argues that semantics are concentrated in sparse informative regions. It decouples such regions from background by maximizing differential entropy over patch similarity and then aligns the full-precision and quantized models only on masked informative patches (Qian et al., 3 Jun 2026).

3. Calibration, optimization, and correction mechanisms

DFQ-ViT methods differ not only in how they synthesize data but also in how they calibrate or optimize the quantized model.

PSAQ-ViT is a calibration-oriented baseline. After synthetic images are generated, it collects per-layer activation minima and maxima, fixes clipping thresholds, and performs quantization without fine-tuning. PSAQ-ViT V2 turns this into an alternating optimization problem with a teacher–student minimax loop. It defines the model discrepancy as

$Q$ 7

and alternates between sample generation,

$Q$ 8

and quantization learning,

$Q$ 9

The paper emphasizes that MAE gives more stable gradients than KL-divergence in the adversarial loop (Li et al., 2022).

CLAMP-ViT uses a two-stage cycle. After synthetic data generation, it performs quantization adaptation by maintaining a small population of candidate quantization parameter sets $k$ 0, where each layer has a bit-width $k$ 1 and a scale $k$ 2. Quantization is applied by the uniform symmetric operator

$k$ 3

with zero-point fixed to $k$ 4. Candidate fitness is evaluated by a local contrastive loss over pooled intermediate activations plus the same output loss $k$ 5, yielding

$k$ 6

The paper then performs layer-wise evolutionary search with parent selection, regeneration, diversity promotion, evaluation, and periodic jumps back to data re-synthesis (Ramachandran et al., 2024).

The semantic prompting framework described in (Zhong et al., 2024) separates synthesis and quantized-model update into two stages. Stage I optimizes synthetic images from $k$ 7 using

$k$ 8

with APA, MSR, SL, and TV smoothing inside each step. Stage II quantizes the model using a linear quantizer for weights and activations and a $k$ 9 quantizer for attention, then performs block-wise reconstruction in BRECQ style: $N$ 0 updating only the weights in block $N$ 1.

MimiQ quantizes weights and activations with integer-only quantization and then fine-tunes the student network on the synthetic dataset. Its training objective is

$N$ 2

where $N$ 3 averages DSSIM-based discrepancies between teacher and student attention outputs over all heads and layers (Choi et al., 2024).

The 2025 DFQ-ViT method is notable for explicitly avoiding fine-tuning. Instead, after quantization parameter calibration it computes an Activation Correction Matrix (ACM) for each layer: $N$ 4 During inference, each corrected layer applies

$N$ 5

No retraining is required; only a single pass is used to accumulate $N$ 6 offline (Tong et al., 19 Jul 2025).

Joint end-to-end PTQ extends optimization further by learning quantization parameters, channel-wise rescaling variables, and a small additive refinement $N$ 7 over all blocks at once. The total loss is

$N$ 8

with $N$ 9, $\Delta$ 0, $\Delta$ 1, and $\Delta$ 2 as typical hyper-parameters. This formulation is explicitly presented as an alternative to block-wise reconstruction, with the claim that inter-block dependencies are optimized jointly (Li et al., 21 Feb 2026).

4. Representative empirical results

The empirical literature shows a progression from 8-bit no-fine-tuning baselines to mixed-precision, 4-bit, 3-bit, and even $\Delta$ 3 settings, with evaluations spanning classification, detection, and segmentation.

Paper	Setting	Representative result
PSAQ-ViT (Li et al., 2022)	ImageNet, DeiT-S, W8/A8	$\Delta$ 4 top-1; Swin-T, W4/A8 reaches $\Delta$ 5
PSAQ-ViT V2 (Li et al., 2022)	Swin-S, 8-bit	$\Delta$ 6 top-1, $\Delta$ 7 box AP, $\Delta$ 8 mask AP, $\Delta$ 9 mIoU
CLAMP-ViT (Ramachandran et al., 2024)	DeiT-S, fixed 4/8 W/A	$z$ 0 top-1 vs. PSAQ-ViT v2’s $z$ 1; UperNet + DeiT-S mixed $z$ 2 gives $z$ 3 mIoU vs. $z$ 4
Semantic prompting method in (Zhong et al., 2024)	ViT-B, W4A4	PSAQ-ViT $z$ 5 SPDFQ $z$ 6 $z$ 7
MimiQ (Choi et al., 2024)	ImageNet, W4/A4	ViT-Base $z$ 8, DeiT-Base $z$ 9, Swin-Base $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 0
DFQ-ViT (Tong et al., 19 Jul 2025)	DeiT-T, ImageNet	W3/A8 $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 1 vs. PSAQ-ViT $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 2; W4/A8 $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 3 vs. real-data $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 4
Joint prompt-guided PTQ (Li et al., 21 Feb 2026)	ViT-S, W4A4 and W1.58A8	W4A4 synth $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 5 vs. real $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 6; W1.58A8 synth $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 7
MaskAQ (Qian et al., 3 Jun 2026)	DeiT-T, 3w3a	$q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 8 vs. MimiQ $q=\mathrm{clip}\Bigl(\mathrm{round}(x/\Delta)+z,\;0,\;2^N-1\Bigr), \qquad \hat x=(q-z)\Delta,$ 9; ADE20K mIoU $0$0 vs. $0$1

Several results are especially indicative of the field’s trajectory. PSAQ-ViT established that a purely data-free method tailored to self-attention could outperform Gaussian-noise calibration and, in some W4/A8 and W8/A8 settings, even surpass a real-data Standard baseline (Li et al., 2022). PSAQ-ViT V2 showed that the same paradigm could extend from ImageNet classification to COCO detection and ADE20K segmentation under a general data-free framework (Li et al., 2022). CLAMP-ViT reported improvements of up to $0$2 in top-1 accuracy for classification, $0$3 mAP for object detection, and $0$4 mIoU for segmentation at similar or better compression ratio over existing alternatives, while its representative experiments reported $0$5 top-1 on DeiT-S fixed $0$6 W/A and $0$7 on ViT-B (Ramachandran et al., 2024).

In low-bit regimes, the gains are larger. The semantic prompting paper reports that on ViT-B ($0$8 FP), W4A4 improves from PSAQ-ViT $0$9 to SPDFQ $L^{C_1}$ 0, and on Swin-B ( $L^{C_1}$ 1 FP), W4A4 improves from $L^{C_1}$ 2 to $L^{C_1}$ 3 (Zhong et al., 2024). MimiQ reports that under W4/A4, ViT-Base/16 reaches $L^{C_1}$ 4, DeiT-Base/16 reaches $L^{C_1}$ 5, and Swin-Base/4/4 reaches $L^{C_1}$ 6, with COCO AP $L^{C_1}$ 7 rising to $L^{C_1}$ 8 and ADE20K mIoU to $L^{C_1}$ 9 (Choi et al., 2024). The 2025 DFQ-ViT method reports a $L^O = (1/n_c)\bigl\|Q(X)-T_G\bigr\|_1 + \bigl\|FP(X)-T_G\bigr\|_1,$ 0 gain over PSAQ-ViT on DeiT-T with 3-bit weights quantization and states that its performance is on par with models quantized through real data (Tong et al., 19 Jul 2025).

5. Conceptual distinctions, misconceptions, and limitations

A common misconception is that “data-free” implies the absence of all external priors. The literature does not support that interpretation. PSAQ-ViT V2 explicitly emphasizes task- and model-independent priors and drops class-conditional guidance (Li et al., 2022), but CLAMP-ViT uses random task-specific targets for classification, detection, and segmentation (Ramachandran et al., 2024). The semantic prompting method uses random semantic targets and soft targets (Zhong et al., 2024), and joint prompt-guided PTQ uses class-name-initialized CLIP tokens together with Stable Diffusion Turbo (Li et al., 21 Feb 2026). This suggests that DFQ-ViT is best understood as “no access to the original calibration images,” not necessarily “no semantic prior.”

A second misconception is that synthetic data quality is judged only by photorealism. In the DFQ-ViT literature, realism is usually defined by internal compatibility with a ViT: bimodal or high-entropy patch-similarity statistics, aligned attention priors, semantically rich local crops, coherent inter-head attention maps, or informative regions that preserve mutual information with the evolving quantized model. The emphasis on internal representations is explicit in PSAQ-ViT, MimiQ, and MaskAQ (Li et al., 2022, Choi et al., 2024, Qian et al., 3 Jun 2026).

A third misconception is that DFQ-ViT is uniformly a no-fine-tuning problem. That is true for PSAQ-ViT and for the 2025 DFQ-ViT method, which replaces retraining with ACM-based forward correction (Tong et al., 19 Jul 2025). It is not true for MimiQ, which fine-tunes for $Q$ 13 epochs on the synthetic set (Choi et al., 2024), for PSAQ-ViT V2, which alternates sample generation and quantization learning (Li et al., 2022), or for joint end-to-end PTQ, which trains for $Q$ 14 iterations and introduces $Q$ 15 (Li et al., 21 Feb 2026).

Claims of parity with real-data PTQ also require careful reading. The literature contains several settings in which DFQ-ViT matches or exceeds real-data calibration, including PSAQ-ViT on selected W4/A8 and W8/A8 experiments, DFQ-ViT on DeiT-T W4/A8, and prompt-guided PTQ on W4A4 where synthetic ViT-S is $Q$ 16 and real-data ViT-S is $Q$ 17 (Li et al., 2022, Tong et al., 19 Jul 2025, Li et al., 21 Feb 2026). However, the same literature also shows persistent gaps in harder settings, especially at very low bit-widths. A plausible implication is that reported “parity” is regime-dependent rather than universal.

Limitations are stated openly across the papers. PSAQ-ViT V2 notes that lower-bit quantization below $Q$ 18 bit may require non-uniform schemes and that the iterative sample-generation loop adds overhead (Li et al., 2022). The 2025 DFQ-ViT paper notes that its cosine curriculum may not be optimal for all datasets, that ACM is additive and layer-local, and that hyperparameters such as $Q$ 19 require tuning per architecture (Tong et al., 19 Jul 2025). Prompt-guided PTQ reduces data dependence but introduces a generative prior through Stable Diffusion Turbo and learned prompts, which is a different computational trade-off from calibration on a small synthetic batch (Li et al., 21 Feb 2026).

6. Research directions and broader significance

The DFQ-ViT literature now spans several distinct research directions: internal-statistics priors, semantic prompting, contrastive calibration, attention-structure matching, activation correction, and joint all-layer optimization. The accumulated evidence indicates that patch-level or attention-level structure is the dominant source of usable calibration information when real images are unavailable.

Several future directions are explicitly identified in the primary papers. PSAQ-ViT V2 proposes combining PSAQ-generated samples with advanced PTQ methods such as block reconstruction and mixed precision, and extending the approach to black-box adversarial tasks or data-free knowledge distillation (Li et al., 2022). The 2025 DFQ-ViT paper proposes learnable curriculum scheduling, low-rank or structured ACM, extension to multi-task ViTs, and theoretical analysis of ACM convergence and stability (Tong et al., 19 Jul 2025). Joint PTQ emphasizes sub-4-bit deployment, including $Q$ 20, and frames the overall pipeline as a practical route to efficient edge deployment (Li et al., 21 Feb 2026). MaskAQ adds periodic sample refreshing to preserve mutual information between synthetic samples and an evolving quantized model, suggesting that future DFQ-ViT systems may increasingly treat synthesis and calibration as a closed-loop dynamical process rather than a single calibration pass (Qian et al., 3 Jun 2026).

Taken together, the field has moved from static synthetic calibration toward adaptive, semantics-aware, and structure-matched quantization. Early methods showed that ViT self-attention itself can supply a data-free prior (Li et al., 2022). Subsequent work showed that cyclic adaptation, contrastive objectives, localized semantics, learned prompts, activation correction, and informative-region masking can materially change performance at 4-bit, 3-bit, and even $Q$ 21 precision (Ramachandran et al., 2024, Zhong et al., 2024, Choi et al., 2024, Tong et al., 19 Jul 2025, Li et al., 21 Feb 2026, Qian et al., 3 Jun 2026). This suggests that DFQ-ViT has become a distinct subfield of ViT compression, defined less by any single quantizer than by the problem of reconstructing task-relevant transformer statistics without seeing the original data.