Channel-wise Power-of-Two Scaling (PTS)

• Channel-wise PTS is a quantization mechanism that rescales activation channels using power-of-two factors to suppress extreme activation outliers.
• It applies fixed, channel-specific scaling and bit-shift compensation to efficiently compress outlier-dominated channels in diffusion-model quantization.
• The method leverages a voting-based calibration and selective deployment, notably on skip connections, to maintain quantization accuracy in low-bit regimes.

Channel-wise Power-of-Two Scaling (PTS) is a post-training quantization mechanism in which selected activation channels are rescaled by channel-specific powers of two before quantization, and the inverse compensation is realized inside the succeeding integer computation by shifting the corresponding quantized weight contributions. In the supplied literature, this formulation appears in diffusion-model quantization, where it is introduced to handle extreme activation outliers in layers with high inter-channel variance, especially skip connections. The acronym is distinct from the older OFDM usage of PTS, where it denotes Partial Transmit Sequence for peak-to-average power ratio reduction rather than channel-wise activation scaling (Lee et al., 17 Jul 2025, Sengar et al., 2012).

1. Definition and problem setting

In DMQ, channel-wise PTS is introduced because static per-tensor activation quantization is mismatched to tensors in which only a small number of channels exhibit extremely large magnitudes. Under a single shared activation scale, those outlier channels either force a very large quantization range, making the majority of channels coarse, or are clamped if the range is kept narrow. The paper identifies this behavior as especially severe in activations and reports that some layers show high inter-channel variance, with skip connections lacking normalization singled out as particularly problematic. It further states that ResBlock skip connections are more severely affected than previously emphasized UNet shortcut layers (Lee et al., 17 Jul 2025).

This mechanism is positioned as complementary to Learned Equivalent Scaling (LES). LES redistributes quantization difficulty between activations and weights, but DMQ explicitly notes that equivalent scaling does not eliminate outliers. PTS is therefore reserved for the remaining extreme cases, especially skip connections where even LES is not sufficient to suppress activation outliers under low-bit quantization (Lee et al., 17 Jul 2025).

The motivating setting is diffusion-model PTQ with static tensor-wise activation quantization and channel-wise weight quantization. Within that setting, PTS is not a general replacement for the quantization pipeline; it is a selective intervention for layers whose activation statistics are dominated by a few pathological channels (Lee et al., 17 Jul 2025).

2. Mathematical form and equivalent execution

PTS in DMQ is an activation-side, channel-wise, power-of-two scaling rule. For input-channel exponents $$\boldsymbol{\delta} = [\delta_1,\delta_2,\dots,\delta_{C_{in}}]$$, the corresponding factors are $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$. Channel $k$ is therefore quantized with an enlarged effective activation scale $2^{\delta_k}s^{(\mathbf X)}$, which compresses that channel before quantization (Lee et al., 17 Jul 2025).

For a linear layer, DMQ writes the quantized computation as

$$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$

Because the compensation factor is restricted to a power of two, the same computation is rewritten as

$$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$

where $\ll$ denotes left bit shift. The appendix gives the corresponding implementation-oriented identity

$$\tilde{\mathbf W}^{\text{shifted}}_{kj} = 2^{\delta_k}\tilde{\mathbf W}_{kj} = \tilde{\mathbf W}_{kj}\ll \delta_k.$$

This is the defining hardware argument for PTS: the activation-side compression is compensated without a general multiplication, but by an exact integer shift on the quantized weight term (Lee et al., 17 Jul 2025).

DMQ is explicit that PTS applies only to activations, while weights otherwise follow the ordinary quantization rule. In practice, the bit-shift is applied immediately after loading weights at kernel execution rather than during multiply-accumulation, which the paper states is intended to keep overhead minimal (Lee et al., 17 Jul 2025).

3. Factor selection and calibration

The PTS exponents in DMQ are not learned by gradient descent. Instead, the paper introduces a voting algorithm designed for robust channel-wise factor selection under a small calibration set. The candidate set is discrete: $\{2^0,2^1,\dots,2^D\},$ or equivalently exponents $d\in\{0,\dots,D\}$. In the reported experiments, $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$0, so the search space is $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$1 (Lee et al., 17 Jul 2025).

For each calibration sample $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$2 and each channel $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$3, the paper selects the best exponent by minimizing quantization error: $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$4 The per-sample winners are then aggregated channel-wise by a mode rule,

$$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$5

followed by an agreement ratio

$$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$6

The final exponent is chosen conservatively: $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$7 DMQ uses $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$8 in all experiments and describes the rule as a safeguard against significant activation distortion by rejecting low-agreement channel factors (Lee et al., 17 Jul 2025).

The selected $$2^{\boldsymbol{\delta}} = [2^{\delta_1},2^{\delta_2},\dots,2^{\delta_{C_{in}}}]$$9 is static at inference time. The paper does not define timestep-specific PTS factors. Instead, the exponents are chosen from calibration activations collected over multiple denoising steps, so timestep variation affects factor selection indirectly through the calibration data rather than through a timestep-indexed inference rule (Lee et al., 17 Jul 2025).

4. Relation to unconstrained channel-wise scaling and to conventional PTS

The current literature uses several distinct notions of channel-wise scaling. Channel-wise PTS in DMQ should not be conflated either with unconstrained real-valued per-channel scaling in LLM quantization or with conventional OFDM Partial Transmit Sequence (Lee et al., 17 Jul 2025, Czakó et al., 4 Jun 2025, Wang et al., 2024, Sengar et al., 2012).

Method	Coefficient form	Operational role
DMQ	$k$0 on activation channels	Compensated by integer weight left shifts; used selectively on skip connections
SmoothRot	$k$1	Unconstrained per-channel scaling before Hadamard rotation; fused into adjacent linear layers
OutlierTune	Real-valued per-channel activation scales $k$2	Folded into downstream weights $k$3, with symmetrization
OFDM PTS	$k$4	Subblock phase rotation for PAPR reduction

SmoothRot is especially close in spirit but not in constraint set. It uses channel-wise scaling to mitigate massive activation outliers in LLMs, yet its scales are standard SmoothQuant-style real values calibrated from activation and weight maxima,

$k$5

with no power-of-two restriction, no rounding to $k$6, and no shift-only implementation. Its efficiency claim comes from offline fusion into neighboring weights, not from power-of-two arithmetic (Czakó et al., 4 Jun 2025).

OutlierTune likewise makes per-channel activation quantization hardware-friendly by absorbing activation-channel scales into weights,

$k$7

and then stabilizing the transformed weights through symmetrization. Again, the scales are ordinary calibration-derived real values rather than powers of two (Wang et al., 2024).

The older OFDM PTS literature uses the same acronym but denotes a different construction. There, the data block is partitioned into subblocks and recombined as

$k$8

with the phase vector chosen to minimize PAPR. This is a constrained phase-rotation method for multicarrier transmission, not channel-wise activation scaling for neural network quantization (Sengar et al., 2012).

5. Empirical behavior in diffusion-model PTQ

The clearest direct evidence for channel-wise PTS comes from DMQ’s ablations on FFHQ, LDM-4, W4A8. In the component study, the reported progression is: full precision with IS $k$9, FID $2^{\delta_k}s^{(\mathbf X)}$0, sFID $2^{\delta_k}s^{(\mathbf X)}$1; baseline with IS $2^{\delta_k}s^{(\mathbf X)}$2, FID $2^{\delta_k}s^{(\mathbf X)}$3, sFID $2^{\delta_k}s^{(\mathbf X)}$4; $2^{\delta_k}s^{(\mathbf X)}$5 LES with IS $2^{\delta_k}s^{(\mathbf X)}$6, FID $2^{\delta_k}s^{(\mathbf X)}$7, sFID $2^{\delta_k}s^{(\mathbf X)}$8; $2^{\delta_k}s^{(\mathbf X)}$9 Adaptive Timestep Weighting with IS $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$0, FID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$1, sFID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$2; and $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$3 Power-of-Two Scaling with IS $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$4, FID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$5, sFID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$6. Within that sequence, the isolated addition of PTS on top of LES and adaptive weighting improves FID from $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$7 to $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$8, sFID from $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} 2^{\delta_k}\,\tilde{\mathbf X}_{ik}\cdot \tilde{\mathbf W}_{kj}\right).$$9 to $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$0, and IS from $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$1 to $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$2 (Lee et al., 17 Jul 2025).

A second ablation probes how PTS should be selected and where it should be applied. Again on FFHQ LDM-4 W4A8, the reported results are: no PTS, IS $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$3, FID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$4, sFID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$5; MSE selection on all layers, IS $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$6, FID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$7, sFID $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$8; voting on all layers, IS $$\mathbf{Y}_{ij} \approx s_i^{(\mathbf X)} s_j^{(\mathbf W)} \left(\sum_{k=1}^{C_{in}} \tilde{\mathbf X}_{ik}\cdot \left(\tilde{\mathbf W}_{kj}\ll \delta_k\right)\right),$$9, FID $\ll$0, sFID $\ll$1; and voting on skip connections, IS $\ll$2, FID $\ll$3, sFID $\ll$4. These numbers support three paper-specific conclusions: PTS helps relative to no PTS, voting is preferable to direct MSE-based factor selection, and selective deployment on skip connections is better than applying PTS to all layers (Lee et al., 17 Jul 2025).

DMQ also reports broader low-bit gains in regimes such as W4A6 and W4A8, including FFHQ, ImageNet class-conditional LDM-4, and Stable Diffusion on MS-COCO. Those results are not attributable to PTS alone, because DMQ combines LES, adaptive timestep weighting, and PTS, but the paper explicitly treats PTS as one of the components enabling stable performance in these low-bit settings (Lee et al., 17 Jul 2025).

6. Limitations, boundaries, and research significance

The literature block imposes several clear boundaries on what channel-wise PTS currently establishes. First, in DMQ it is selective rather than universal: applying PTS to all layers is worse than restricting it to skip connections, so it is not presented as an always-on scaling rule (Lee et al., 17 Jul 2025). Second, the formulation assumes an adjacent affine or convolutional computation in which activation-side powers-of-two can be compensated by weight-side shifts. The paper provides linear and GEMM-equivalent formulations and states that convolution follows the same logic, but it does not provide a general operator-by-operator recipe for every network pattern (Lee et al., 17 Jul 2025).

Third, PTS uses static, globally selected channel factors rather than timestep-specific inference-time factors. This improves deployability but leaves open how to handle layers whose outlier structure changes strongly across denoising steps. Fourth, the power-of-two restriction is intentionally coarse. DMQ addresses that loss of flexibility by combining PTS with LES: LES supplies fine-grained floating-point balancing across all layers, whereas PTS supplies stronger but discrete activation-side suppression only where outliers are extreme (Lee et al., 17 Jul 2025).

Related LLM quantization work clarifies what PTS is not. SmoothRot shows that channel-wise scaling before Hadamard rotation is effective for severe 4-bit activation quantization, but its scales are unconstrained real values fused into neighboring weights, not powers of two (Czakó et al., 4 Jun 2025). OutlierTune shows that per-channel activation scales can be absorbed into downstream weights and that symmetrization can be critical at low precision, yet again the scaling factors are standard real-valued calibration outputs rather than $\ll$5 (Wang et al., 2024). Those papers therefore provide evidence for the value of channel-wise preconditioning and offline folding, but not for the sufficiency of power-of-two-constrained scales.

Taken together, the supplied sources support a narrow but technically precise characterization. Channel-wise PTS, in the strict sense, is an activation-only, per-channel, power-of-two scaling mechanism whose inverse compensation is realized by exact integer shifts in the succeeding operator. In the provided literature, that definition is instantiated explicitly in diffusion-model PTQ by DMQ. Other channel-wise scaling methods may share the same objective—suppressing outlier-driven activation ranges and improving low-bit quantization—but they should not be classified as true PTS unless the per-channel factors are actually restricted to powers of two (Lee et al., 17 Jul 2025, Czakó et al., 4 Jun 2025, Wang et al., 2024).