OrbitQuant: Diffusion Transformer Quantization

• OrbitQuant is a calibration-free post-training quantization method for diffusion transformers that uses a normalized, randomly rotated basis to handle activation non-stationarity.
• It employs a precomputed Lloyd–Max scalar codebook per dimension shared across layers, timesteps, prompts, and modalities for consistent, efficient quantization.
• The method integrates efficient randomized permuted block-Hadamard rotations and weight-rotation cancellation to reduce runtime overhead and maintain generation quality.

Searching arXiv for the primary OrbitQuant paper and closely related diffusion-transformer PTQ context. OrbitQuant is a post-training, training-free, calibration-free weight-activation quantization method for diffusion transformers (DiTs) that quantizes in a normalized, randomly rotated basis rather than by estimating activation ranges from calibration data (Lee et al., 2 Jul 2026). Its central premise is that DiT activations shift across denoising timesteps, prompts, classifier-free guidance branches, checkpoints, and modalities, making conventional calibration-based post-training quantization brittle; OrbitQuant addresses this by using a normalized, rotated representation in which each coordinate follows an almost fixed, known distribution determined primarily by the hidden dimension, so that one precomputed Lloyd–Max scalar codebook per dimension and bit-width can be shared across layers, timesteps, prompts, and even across image and video DiTs with the same hidden size (Lee et al., 2 Jul 2026).

1. Definition and problem setting

OrbitQuant is presented for post-training quantization of diffusion transformers used for image and video generation, in a setting where inference is expensive because of multi-step denoising and growing parameter count (Lee et al., 2 Jul 2026). The method is explicitly designed for joint weight-activation quantization rather than weight-only compression, because the paper emphasizes that DiT inference is often compute-bound, so weight-only quantization does not deliver measured speedup (Lee et al., 2 Jul 2026).

The problem it addresses is activation non-stationarity. The paper states that DiT activations exhibit channel-wise outliers and shift across timesteps, prompts, and classifier-free-guidance branches, and that existing DiT post-training quantization methods absorb this drift with calibration, so each new checkpoint, resolution, or modality requires a calibration set to be re-collected and re-fit (Lee et al., 2 Jul 2026). OrbitQuant instead adopts a data-agnostic construction: no calibration set, no per-layer range fitting, no prompt- or timestep-dependent scale estimation, and no per-checkpoint re-fitting of the quantizer (Lee et al., 2 Jul 2026).

This design is applied to both image and video diffusion transformers. The reported image models are FLUX.1-schnell, FLUX.1-dev, and Z-Image-Turbo, while the video models are Wan 2.1-1.3B and CogVideoX-2B; supplementary experiments also include Wan 14B and HunyuanVideo (Lee et al., 2 Jul 2026). Image evaluation is reported on GenEval and video evaluation on VBench (Lee et al., 2 Jul 2026).

2. Quantization in a normalized, rotated basis

The central methodological move is to quantize after normalization and random orthogonal mixing. The paper observes that if a unit vector is multiplied by a sufficiently random orthogonal matrix, each rotated coordinate has a fixed marginal distribution independent of the original vector direction (Lee et al., 2 Jul 2026). For a Haar-random orthogonal matrix $\boldsymbol{\Phi}_d$, each coordinate of $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$ follows $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$ and for $d \ge 64$ the paper says this is tightly approximated by $\mathcal{N}(0,1/d)$ (Lee et al., 2 Jul 2026).

This fixed post-rotation marginal is what permits a shared codebook. OrbitQuant uses a codebook $$\mathcal{C}^{(d,b)} = \{c_1^{(d,b)}, \ldots, c_{2^b}^{(d,b)}\},$$ with nearest-centroid scalar quantizer $$\hat{q}_b^{(d)}(t) = \operatorname*{arg\,min}_{c \,\in\, \mathcal{C}^{(d,b)}} |t - c|,$$ and coordinate-wise vector quantization $$\hat{Q}_b^{(d)}(\mathbf{u})_k = \hat{q}_b^{(d)}(u_k)$$ (Lee et al., 2 Jul 2026). The codebook is obtained by running Lloyd–Max against the fixed marginal $f_d$, offline, and the paper explicitly states that no scales or zero-points are used in the codebook and that the codebook is shared by all layers and rows with the same input dimension (Lee et al., 2 Jul 2026).

For activations, OrbitQuant first rotates and normalizes:

$$\mathbf{x}' = \boldsymbol{\Pi}_d \mathbf{x}, \quad s = \|\mathbf{x}'\|_2, \quad \tilde{\mathbf{x}}' = \mathbf{x}' / (s + \varepsilon),$$

where $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$0 (Lee et al., 2 Jul 2026). It then quantizes the direction and restores the norm:

$\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$1

For a batch of $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$2 tokens, the rotation is applied row-wise as $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$3 (Lee et al., 2 Jul 2026). In effect, the method uses per-token normalization, followed by non-uniform scalar quantization in a rotated basis, with one shared codebook per dimension.

A plausible implication is that OrbitQuant reframes activation quantization from a range-estimation problem into a distribution-matching problem. The paper itself states this more concretely by saying that one precomputed Lloyd–Max scalar codebook per dimension and bit-width can be reused across layers, prompts, timesteps, checkpoints, and modalities (Lee et al., 2 Jul 2026).

3. Randomized permuted block-Hadamard rotation

A dense Haar rotation is too expensive for deployment, so OrbitQuant replaces it with a structured orthogonal transform called randomized permuted block-Hadamard (RPBH) (Lee et al., 2 Jul 2026). The rotation is defined as

$\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$4

where $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$5 is a uniform random permutation matrix, each $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$6 is a Rademacher sign diagonal, and $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$7 is an $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$8 Walsh–Hadamard matrix (Lee et al., 2 Jul 2026). The paper reports that $\boldsymbol{\Phi}_d \tilde{\mathbf{x}}$9 is chosen as the largest power of two dividing $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$0, with $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$1 across the evaluated models (Lee et al., 2 Jul 2026).

The permutation is integral to the construction. Without permutation, each block-Hadamard only mixes coordinates within its own block, so a block containing outliers remains problematic; with permutation, mass is spread across blocks before Hadamard mixing (Lee et al., 2 Jul 2026). This supports the paper’s claim that the permutation improves low-bit robustness relative to block-only rotations (Lee et al., 2 Jul 2026).

Theoretical support is given in the form of a variance-concentration proposition. For a fixed unit vector $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$2, with $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$3 and $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$4, the paper states that with probability at least $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$5 over the random rotation, every coordinate $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$6 of $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$7 satisfies

$$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$8

It also states an $$f_d(t) = \frac{\Gamma(d/2)}{\sqrt{\pi}\,\Gamma((d-1)/2)} (1 - t^2)^{(d-3)/2}, \quad t \in [-1, 1],$$9 bound,

$d \ge 64$0

and remarks that the variance concentration upgrades to a quantitative Gaussian approximation (Lee et al., 2 Jul 2026).

The empirical evidence reported is that after RPBH, the coordinate histograms of both weights and activations match the target $d \ge 64$1 nearly as well as dense Haar rotation, while raw activations deviate sharply (Lee et al., 2 Jul 2026). In ablations on FLUX.1-schnell, RPBH achieves overall GenEval scores of 0.690 at W4A4, 0.674 at W3A3, and 0.595 at W2A4, with 0.451 s rotation latency; the corresponding dense Haar latency is 11.65 s (Lee et al., 2 Jul 2026).

4. Weight quantization and rotation cancellation

OrbitQuant extends the same construction to weight rows offline. For a linear layer

$d \ge 64$2

it rotates the weight as

$d \ge 64$3

(Lee et al., 2 Jul 2026). For each row $d \ge 64$4, it stores

$d \ge 64$5

and quantizes using

$d \ge 64$6

(Lee et al., 2 Jul 2026). The row-norm vector is stored in BF16, and the paper states that this adds only $d \ge 64$7 bits per layer, less than 0.3% relative to $d \ge 64$8 bits for direction quantization (Lee et al., 2 Jul 2026).

The operational advantage is that the forward activation rotation can be absorbed algebraically by the offline weight rotation:

$d \ge 64$9

since $\mathcal{N}(0,1/d)$0 is orthogonal (Lee et al., 2 Jul 2026). With quantization, the layer computes $\mathcal{N}(0,1/d)$1 (Lee et al., 2 Jul 2026). The inverse rotation is therefore not needed at runtime; only the forward activation rotation remains (Lee et al., 2 Jul 2026).

The paper applies this to every linear projection in transformer blocks, including image-side and text-side Q, K, V, output projections, feed-forward layers, and text-conditioning K/V projections for cross/joint attention (Lee et al., 2 Jul 2026). It keeps embeddings, timestep MLP, final un-patchify head, and text encoder in BF16, and handles AdaLN modulation projections separately: weights are quantized with INT4 RTN, group size 64, while activations remain BF16 (Lee et al., 2 Jul 2026).

This suggests that OrbitQuant is not merely a quantizer but also a layerwise algebraic reparameterization scheme. The paper’s concrete claim is narrower: the rotation is absorbed into the weights so that it cancels inside each linear layer and only a forward rotation on the activations remains at runtime (Lee et al., 2 Jul 2026).

5. Empirical results across image and video diffusion transformers

OrbitQuant is evaluated at several low-bit settings. For image generation on GenEval, the paper reports state-of-the-art post-training quantization results across FLUX.1-schnell, FLUX.1-dev, and Z-Image-Turbo (Lee et al., 2 Jul 2026).

For FLUX.1-schnell, the reported overall GenEval scores are 0.664 in FP16, 0.703 at OrbitQuant W4A4, and 0.604 at OrbitQuant W2A4 (Lee et al., 2 Jul 2026). For FLUX.1-dev, the corresponding scores are 0.667 in FP16, 0.633 at W4A4, and 0.475 at W2A4 (Lee et al., 2 Jul 2026). For Z-Image-Turbo, the reported overall scores are 0.754 in FP16, 0.767 at W4A4, and 0.319 at W2A4 (Lee et al., 2 Jul 2026).

The paper emphasizes that OrbitQuant pushes post-training quantization of image diffusion transformers to W2A4 with usable generation quality, while the listed low-bit baselines largely collapse at that setting (Lee et al., 2 Jul 2026). Supplementary image results are also reported at W3A3 and W2A3. At W3A3, OrbitQuant achieves overall scores of 0.678 on FLUX.1-schnell, 0.584 on FLUX.1-dev, and 0.740 on Z-Image-Turbo (Lee et al., 2 Jul 2026). At W2A3, the reported scores are 0.517, 0.372, and 0.105 respectively, and the paper explicitly notes that Z-Image-Turbo degrades sharply there (Lee et al., 2 Jul 2026).

For video generation on VBench, the method is evaluated on Wan 2.1-1.3B and CogVideoX-2B (Lee et al., 2 Jul 2026). On Wan 2.1-1.3B, the reported Overall Consistency scores are 24.67 in FP16, 24.35 at W4A6, and 23.86 at W4A4 (Lee et al., 2 Jul 2026). On CogVideoX-2B, the corresponding scores are 25.06 in FP16, 24.55 at W4A6, and 23.86 at W4A4 (Lee et al., 2 Jul 2026). Supplementary experiments also report that Wan 14B at W4A4 reaches overall consistency 0.2615 versus BF16 0.2629, and that OrbitQuant is competitive with DVD-Quant on HunyuanVideo (Lee et al., 2 Jul 2026).

The paper repeatedly frames this as cross-modality transfer: the same recipe is used unchanged on image and video DiTs, with one codebook per dimension serving all timesteps, prompts, layers, and both modalities (Lee et al., 2 Jul 2026). A plausible interpretation is that the hidden dimension, rather than modality, becomes the dominant quantization unit.

6. Runtime characteristics, limitations, and relation to prior PTQ methods

The runtime path involves one forward RPBH rotation on activations, one per-token norm computation, one nearest-centroid lookup per coordinate, and matrix multiplication using dequantized BF16 values in the current fake-quant implementation (Lee et al., 2 Jul 2026). The paper explicitly states that current experiments use fake quantization: weights and activations are dequantized to BF16 and GEMM is computed in BF16, so reported latency measures quantization overhead rather than realized low-bit kernel speedup (Lee et al., 2 Jul 2026).

Despite that systems limitation, OrbitQuant is reported to have the lowest overhead among the tested weight-and-activation quantization methods. On image generation with FLUX.1-dev, SmoothQuant is 1.09× slower than OrbitQuant, QuaRot is 1.28× slower, and ViDiT-Q is 1.40× slower (Lee et al., 2 Jul 2026). On video, OrbitQuant uses 20.3 GB peak memory versus 19.3 GB for QuaRot and SmoothQuant and 23.2 GB for ViDiT-Q (Lee et al., 2 Jul 2026).

The paper also identifies its own limitations. OrbitQuant requires an online activation rotation; there is no native low-bit GEMM kernel yet for the non-uniform Lloyd–Max centroids; codebooks are dimension-specific; and some layers, notably AdaLN modulation projections, cannot use the same cancellation trick (Lee et al., 2 Jul 2026). It further notes that quality still degrades at extreme low bit-widths, especially W2A3 on some models (Lee et al., 2 Jul 2026).

Within the paper’s own comparative framing, OrbitQuant is positioned against calibration-based DiT PTQ methods such as ViDiT-Q, SVDQuant, AdaTSQ, PTQ4DiT, LRQ-DiT, and PermuQuant, against SmoothQuant-style rescaling methods, and against rotation-based methods such as QuaRot (Lee et al., 2 Jul 2026). Its immediate conceptual precursor is TurboQuant, but OrbitQuant’s contribution is to adapt normalization-plus-random-rotation quantization to internal DiT linear layers, replace dense Haar with efficient RPBH, absorb the rotation into weights so that no inverse rotation is needed at runtime, and apply the same codebook family to both activations and weight rows (Lee et al., 2 Jul 2026).

The strongest general conclusion stated by the paper is that OrbitQuant is a calibration-free post-training quantizer for diffusion transformers that converts a brittle activation-range problem into a fixed-marginal quantization problem and sets the state of the art for PTQ at several low-bit settings across FLUX.1, Z-Image-Turbo, Wan 2.1, and CogVideoX (Lee et al., 2 Jul 2026). A plausible implication is that its most durable significance lies not only in its reported scores but in the abstraction it proposes: quantization by basis normalization and probabilistic regularization rather than by empirical calibration.