---
title: 'OrbitQuant: Diffusion Transformer Quantization'
url: https://www.emergentmind.com/topics/orbitquant
type: topic
---

# OrbitQuant: Diffusion Transformer Quantization

Searching arXiv for the primary OrbitQuant paper and closely related diffusion-transformer PTQ context.
OrbitQuant is a post-training, training-free, calibration-free weight-activation quantization method for diffusion transformers (DiTs) that quantizes in a normalized, randomly rotated basis rather than by estimating activation ranges from calibration data [2607.02461]. Its central premise is that DiT activations shift across denoising timesteps, prompts, classifier-free guidance branches, checkpoints, and modalities, making conventional calibration-based post-training quantization brittle; OrbitQuant addresses this by using a normalized, rotated representation in which each coordinate follows an almost fixed, known distribution determined primarily by the hidden dimension, so that one precomputed Lloyd–Max scalar codebook per dimension and bit-width can be shared across layers, timesteps, prompts, and even across image and video DiTs with the same hidden size [2607.02461].

## 1. Definition and problem setting

OrbitQuant is presented for post-training quantization of diffusion transformers used for image and video generation, in a setting where inference is expensive because of multi-step denoising and growing parameter count [2607.02461]. The method is explicitly designed for joint weight-activation quantization rather than weight-only compression, because the paper emphasizes that DiT inference is often compute-bound, so weight-only quantization does not deliver measured speedup [2607.02461].

The problem it addresses is activation non-stationarity. The paper states that DiT activations exhibit channel-wise outliers and shift across timesteps, prompts, and classifier-free-guidance branches, and that existing DiT post-training quantization methods absorb this drift with calibration, so each new checkpoint, resolution, or modality requires a calibration set to be re-collected and re-fit [2607.02461]. OrbitQuant instead adopts a data-agnostic construction: no calibration set, no per-layer range fitting, no prompt- or timestep-dependent scale estimation, and no per-checkpoint re-fitting of the quantizer [2607.02461].

This design is applied to both image and video diffusion transformers. The reported image models are FLUX.1-schnell, FLUX.1-dev, and Z-Image-Turbo, while the video models are Wan 2.1-1.3B and CogVideoX-2B; supplementary experiments also include Wan 14B and HunyuanVideo [2607.02461]. Image evaluation is reported on GenEval and video evaluation on VBench [2607.02461].

## 2. Quantization in a normalized, rotated basis

The central methodological move is to quantize after normalization and random orthogonal mixing. The paper observes that if a unit vector is multiplied by a sufficiently random orthogonal matrix, each rotated coordinate has a fixed marginal distribution independent of the original vector direction [2607.02461]. For a Haar-random orthogonal matrix $\boldsymbol{\Phi}_d$, each coordinate of $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$ follows  
$$
f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],
$$  
and for $d \ge 64$ the paper says this is tightly approximated by $\mathcal{N}(0,1/d)$ [2607.02461].

This fixed post-rotation marginal is what permits a shared codebook. OrbitQuant uses a codebook  
$$
\mathcal{C}^{(d,b)} = \{c_1^{(d,b)}, \ldots, c_{2^b}^{(d,b)}\},
$$  
with nearest-centroid scalar quantizer  
$$
\hat{q}_b^{(d)}(t) = \operatorname*{arg\,min}_{c \,\in\, \mathcal{C}^{(d,b)}} |t - c|,
$$  
and coordinate-wise vector quantization  
$$
\hat{Q}_b^{(d)}(\mathbf{u})_k = \hat{q}_b^{(d)}(u_k)
$$  
[2607.02461]. The codebook is obtained by running Lloyd–Max against the fixed marginal $f_d$, offline, and the paper explicitly states that no scales or zero-points are used in the codebook and that the codebook is shared by all layers and rows with the same input dimension [2607.02461].

For activations, OrbitQuant first rotates and normalizes:
$$
\mathbf{x}' = \boldsymbol{\Pi}_d \mathbf{x}, \quad s = \|\mathbf{x}'\|_2, \quad \tilde{\mathbf{x}}' = \mathbf{x}' / (s + \varepsilon),
$$
where $\varepsilon = 10^{-10}$ [2607.02461]. It then quantizes the direction and restores the norm:
$$
\hat{\mathbf{x}}' = s \cdot \hat{Q}_{b_a}^{(d)}(\tilde{\mathbf{x}}').
$$
For a batch of $N$ tokens, the rotation is applied row-wise as $\mathbf{x}\boldsymbol{\Pi}_d^\top$ [2607.02461]. In effect, the method uses per-token normalization, followed by non-uniform scalar quantization in a rotated basis, with one shared codebook per dimension.

A plausible implication is that OrbitQuant reframes activation quantization from a range-estimation problem into a distribution-matching problem. The paper itself states this more concretely by saying that one precomputed Lloyd–Max scalar codebook per dimension and bit-width can be reused across layers, prompts, timesteps, checkpoints, and modalities [2607.02461].

## 3. Randomized permuted block-Hadamard rotation

A dense Haar rotation is too expensive for deployment, so OrbitQuant replaces it with a structured orthogonal transform called randomized permuted block-Hadamard (RPBH) [2607.02461]. The rotation is defined as
$$
\boldsymbol{\Pi}_d = \mathrm{blkdiag}(\mathbf{H}_h \mathbf{D}_1, \ldots, \mathbf{H}_h \mathbf{D}_{d/h}) \cdot \mathbf{P}_\pi,
$$
where $\mathbf{P}_\pi$ is a uniform random permutation matrix, each $\mathbf{D}_i$ is a Rademacher sign diagonal, and $\mathbf{H}_h$ is an $h \times h$ Walsh–Hadamard matrix [2607.02461]. The paper reports that $h$ is chosen as the largest power of two dividing $d$, with $h \in \{128,512,1024,2048,4096\}$ across the evaluated models [2607.02461].

The permutation is integral to the construction. Without permutation, each block-Hadamard only mixes coordinates within its own block, so a block containing outliers remains problematic; with permutation, mass is spread across blocks before Hadamard mixing [2607.02461]. This supports the paper’s claim that the permutation improves low-bit robustness relative to block-only rotations [2607.02461].

Theoretical support is given in the form of a variance-concentration proposition. For a fixed unit vector $\tilde{\mathbf{x}}$, with $\mu_\infty = \|\tilde{\mathbf{x}}\|_\infty^2$ and $d=kh$, the paper states that with probability at least $1-\delta$ over the random rotation, every coordinate $z_i$ of $\boldsymbol{\Pi}_d \tilde{\mathbf{x}}$ satisfies
$$
\mathrm{Var}(z_i \mid \pi) \in \Big[\tfrac{1-\rho}{d},\; \tfrac{1+\rho}{d}\Big], \qquad \rho = d\,\mu_\infty \sqrt{\tfrac{1}{2h}\log\tfrac{4k}{\delta}}.
$$
It also states an $\ell_\infty$ bound,
$$
\|\boldsymbol{\Pi}_d \tilde{\mathbf{x}}\|_\infty \le \sqrt{\tfrac{2}{d}(1 + \rho)\log(4d/\delta)},
$$
and remarks that the variance concentration upgrades to a quantitative Gaussian approximation [2607.02461].

The empirical evidence reported is that after RPBH, the coordinate histograms of both weights and activations match the target $\mathcal{N}(0,1/d)$ nearly as well as dense Haar rotation, while raw activations deviate sharply [2607.02461]. In ablations on FLUX.1-schnell, RPBH achieves overall GenEval scores of 0.690 at W4A4, 0.674 at W3A3, and 0.595 at W2A4, with 0.451 s rotation latency; the corresponding dense Haar latency is 11.65 s [2607.02461].

## 4. Weight quantization and rotation cancellation

OrbitQuant extends the same construction to weight rows offline. For a linear layer
$$
\mathbf{y} = \mathbf{W}\mathbf{x}, \quad \mathbf{W}\in\mathbb{R}^{m\times d},\ \mathbf{x}\in\mathbb{R}^d,
$$
it rotates the weight as
$$
\mathbf{W}' = \mathbf{W}\boldsymbol{\Pi}_d^\top
$$
[2607.02461]. For each row $\mathbf{w}_i'$, it stores
$$
r_i' = \|\mathbf{w}_i'\|_2, \qquad \tilde{\mathbf{w}}_i' = \mathbf{w}_i' / r_i',
$$
and quantizes using
$$
\hat{\mathbf{W}}' = \mathrm{diag}(\mathbf{r}') \cdot \hat{Q}_{b_w}^{(d)}(\tilde{\mathbf{W}}')
$$
[2607.02461]. The row-norm vector is stored in BF16, and the paper states that this adds only $16m$ bits per layer, less than 0.3% relative to $b_wmd$ bits for direction quantization [2607.02461].

The operational advantage is that the forward activation rotation can be absorbed algebraically by the offline weight rotation:
$$
\mathbf{W}'\mathbf{x}' = \mathbf{W}\boldsymbol{\Pi}_d^\top\boldsymbol{\Pi}_d\mathbf{x} = \mathbf{W}\mathbf{x},
$$
since $\boldsymbol{\Pi}_d$ is orthogonal [2607.02461]. With quantization, the layer computes $\hat{\mathbf{W}}'\hat{\mathbf{x}}' \approx \mathbf{W}\mathbf{x}$ [2607.02461]. The inverse rotation is therefore not needed at runtime; only the forward activation rotation remains [2607.02461].

The paper applies this to every linear projection in transformer blocks, including image-side and text-side Q, K, V, output projections, feed-forward layers, and text-conditioning K/V projections for cross/joint attention [2607.02461]. It keeps embeddings, timestep MLP, final un-patchify head, and text encoder in BF16, and handles AdaLN modulation projections separately: weights are quantized with INT4 RTN, group size 64, while activations remain BF16 [2607.02461].

This suggests that OrbitQuant is not merely a quantizer but also a layerwise algebraic reparameterization scheme. The paper’s concrete claim is narrower: the rotation is absorbed into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime [2607.02461].

## 5. Empirical results across image and video diffusion transformers

OrbitQuant is evaluated at several low-bit settings. For image generation on GenEval, the paper reports state-of-the-art post-training quantization results across FLUX.1-schnell, FLUX.1-dev, and Z-Image-Turbo [2607.02461].

For FLUX.1-schnell, the reported overall GenEval scores are 0.664 in FP16, 0.703 at OrbitQuant W4A4, and 0.604 at OrbitQuant W2A4 [2607.02461]. For FLUX.1-dev, the corresponding scores are 0.667 in FP16, 0.633 at W4A4, and 0.475 at W2A4 [2607.02461]. For Z-Image-Turbo, the reported overall scores are 0.754 in FP16, 0.767 at W4A4, and 0.319 at W2A4 [2607.02461].

The paper emphasizes that OrbitQuant pushes post-training quantization of image diffusion transformers to W2A4 with usable generation quality, while the listed low-bit baselines largely collapse at that setting [2607.02461]. Supplementary image results are also reported at W3A3 and W2A3. At W3A3, OrbitQuant achieves overall scores of 0.678 on FLUX.1-schnell, 0.584 on FLUX.1-dev, and 0.740 on Z-Image-Turbo [2607.02461]. At W2A3, the reported scores are 0.517, 0.372, and 0.105 respectively, and the paper explicitly notes that Z-Image-Turbo degrades sharply there [2607.02461].

For video generation on VBench, the method is evaluated on Wan 2.1-1.3B and CogVideoX-2B [2607.02461]. On Wan 2.1-1.3B, the reported Overall Consistency scores are 24.67 in FP16, 24.35 at W4A6, and 23.86 at W4A4 [2607.02461]. On CogVideoX-2B, the corresponding scores are 25.06 in FP16, 24.55 at W4A6, and 23.86 at W4A4 [2607.02461]. Supplementary experiments also report that Wan 14B at W4A4 reaches overall consistency 0.2615 versus BF16 0.2629, and that OrbitQuant is competitive with DVD-Quant on HunyuanVideo [2607.02461].

The paper repeatedly frames this as cross-modality transfer: the same recipe is used unchanged on image and video DiTs, with one codebook per dimension serving all timesteps, prompts, layers, and both modalities [2607.02461]. A plausible interpretation is that the hidden dimension, rather than modality, becomes the dominant quantization unit.

## 6. Runtime characteristics, limitations, and relation to prior PTQ methods

The runtime path involves one forward RPBH rotation on activations, one per-token norm computation, one nearest-centroid lookup per coordinate, and matrix multiplication using dequantized BF16 values in the current fake-quant implementation [2607.02461]. The paper explicitly states that current experiments use fake quantization: weights and activations are dequantized to BF16 and GEMM is computed in BF16, so reported latency measures quantization overhead rather than realized low-bit kernel speedup [2607.02461].

Despite that systems limitation, OrbitQuant is reported to have the lowest overhead among the tested weight-and-activation quantization methods. On image generation with FLUX.1-dev, SmoothQuant is 1.09× slower than OrbitQuant, QuaRot is 1.28× slower, and ViDiT-Q is 1.40× slower [2607.02461]. On video, OrbitQuant uses 20.3 GB peak memory versus 19.3 GB for QuaRot and SmoothQuant and 23.2 GB for ViDiT-Q [2607.02461].

The paper also identifies its own limitations. OrbitQuant requires an online activation rotation; there is no native low-bit GEMM kernel yet for the non-uniform Lloyd–Max centroids; codebooks are dimension-specific; and some layers, notably AdaLN modulation projections, cannot use the same cancellation trick [2607.02461]. It further notes that quality still degrades at extreme low bit-widths, especially W2A3 on some models [2607.02461].

Within the paper’s own comparative framing, OrbitQuant is positioned against calibration-based DiT PTQ methods such as ViDiT-Q, SVDQuant, AdaTSQ, PTQ4DiT, LRQ-DiT, and PermuQuant, against SmoothQuant-style rescaling methods, and against rotation-based methods such as QuaRot [2607.02461]. Its immediate conceptual precursor is TurboQuant, but OrbitQuant’s contribution is to adapt normalization-plus-random-rotation quantization to internal DiT linear layers, replace dense Haar with efficient RPBH, absorb the rotation into weights so that no inverse rotation is needed at runtime, and apply the same codebook family to both activations and weight rows [2607.02461].

The strongest general conclusion stated by the paper is that OrbitQuant is a calibration-free post-training quantizer for diffusion transformers that converts a brittle activation-range problem into a fixed-marginal quantization problem and sets the state of the art for PTQ at several low-bit settings across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX [2607.02461]. A plausible implication is that its most durable significance lies not only in its reported scores but in the abstraction it proposes: quantization by basis normalization and probabilistic regularization rather than by empirical calibration.

Source: https://www.emergentmind.com/topics/orbitquant