Decoupled Scale Search (DSS) for NVFP4 Quantization

Updated 5 July 2026

Decoupled Scale Search (DSS) is a reconstruction-aware method that decouples the high-precision quantization scale from the hardware-constrained dequantization scale to minimize block-wise weight reconstruction error.
It functions as the refinement step following Closed-form Joint Scale Optimization (CJSO) in the SOAR framework, using a light discrete local search while preserving the standard NVFP4 storage format.
Empirical results demonstrate that DSS, alone or combined with CJSO, improves accuracy and perplexity metrics in NVFP4 and MXFP4 settings, validating its practical impact on large-scale model quantization.

Searching arXiv for the cited SOAR paper and closely related NVFP4 quantization work to ground the article. Decoupled Scale Search (DSS) is the discrete refinement component of SOAR, an NVFP4 post-training quantization framework for LLMs. In this setting, DSS is a reconstruction-aware procedure for choosing NVFP4 block scales that explicitly separates the scale used to quantize weights into FP4 from the scale used to dequantize on hardware, and then jointly searches over both to minimize block-wise weight reconstruction error under NVFP4 constraints. Its defining feature is that the deployed representation remains unchanged—FP4 weights, FP8 block scales, and an FP32 global scale—while the offline optimization is allowed to use a high-precision quantization scale that is not itself stored at inference time (Bao et al., 12 May 2026).

1. Definition and placement within SOAR

SOAR consists of two tightly coupled components. The first is Closed-form Joint Scale Optimization (CJSO), a continuous analytic procedure that, assuming fixed FP4 codes, solves for the global scale and block scales in closed form by minimizing squared reconstruction error. The second is DSS, a discrete local search that refines block-wise scales by decoupling the quantization scale $\Delta_i^q$ from the dequantization scale $\Delta_i^d$ , and then jointly searching over $(\Delta_i^q,\Delta_i^d)$ for each block (Bao et al., 12 May 2026).

Within the SOAR pipeline, DSS follows a CJSO update. The iteration for a weight tensor $W$ is: initialize the global scale $\alpha$ and block scale $\Delta_i^d$ with standard NVFP4 max-based rules and set $\Delta_i^q \leftarrow \Delta_i^d$ ; perform a CJSO step to update $\alpha$ and the block scales; then run DSS around each updated $\Delta_i^d$ by searching over nearby FP8 candidates for $\Delta_i^d$ and continuous multiplicative perturbations for $\Delta_i^d$ 0. These stages repeat until an MSE convergence criterion is met. CJSO provides an analytic center in scale space, and DSS locally refines around that point.

The decoupling is specific. In standard NVFP4, a single FP8 block scale is used both to generate FP4 codes through $\Delta_i^d$ 1 and to reconstruct through $\Delta_i^d$ 2. DSS breaks that coupling by introducing a high-precision offline quantization scale $\Delta_i^d$ 3 and a hardware-constrained dequantization scale $\Delta_i^d$ 4. The stored model format is unchanged; the decoupling exists only during offline optimization.

2. NVFP4 setting and the coupled-scale limitation

The NVFP4 representation used here combines a tensor-wide global FP32 scale $\Delta_i^d$ 5, per-block FP8 (E4M3) scales $\Delta_i^d$ 6, and FP4 (E2M1) data, with each block containing 16 elements. The paper states the maximum representable magnitudes as $\Delta_i^d$ 7 and $\Delta_i^d$ 8. In the baseline coupled formulation, the global scale is computed as

$\Delta_i^d$ 9

and each block scale is quantized to FP8 as

$(\Delta_i^q,\Delta_i^d)$ 0

The FP4 data are then produced by

$(\Delta_i^q,\Delta_i^d)$ 1

and dequantization is

$(\Delta_i^q,\Delta_i^d)$ 2

For weights, the per-block form is $(\Delta_i^q,\Delta_i^d)$ 3 (Bao et al., 12 May 2026).

The central problem is scale quantization error. The true continuous block scale that minimizes reconstruction error may lie between representable FP8 values. In the coupled scheme, quantizing that scale to FP8 distorts both the code assignment step and the reconstruction step, because the same rounded $(\Delta_i^q,\Delta_i^d)$ 4 appears in both $(\Delta_i^q,\Delta_i^d)$ 5 and $(\Delta_i^q,\Delta_i^d)$ 6. The paper explicitly notes that this can be especially problematic with heavy-tailed weight distributions and with blocks containing a mix of small and large magnitudes.

DSS addresses this by allowing the quantizer to use a high-precision $(\Delta_i^q,\Delta_i^d)$ 7 to determine FP4 codes, while enforcing that the stored inference-time scale $(\Delta_i^q,\Delta_i^d)$ 8 remains FP8. This changes the optimization problem without changing the deployment format.

3. Mathematical formulation and relation to CJSO

CJSO optimizes the coupled reconstruction objective

$(\Delta_i^q,\Delta_i^d)$ 9

Given fixed FP4 codes, the objective is quadratic in $W$ 0 and each $W$ 1, yielding closed-form updates. The global scale update is

$W$ 2

and the block-scale update is

$W$ 3

followed by projection of $W$ 4 to the nearest FP8 (E4M3) value (Bao et al., 12 May 2026).

DSS reformulates the problem by distinguishing the scale used for quantization from the scale used for dequantization. The decoupled tensor-level objective becomes

$W$ 5

For each block, DSS minimizes the local loss

$W$ 6

This decomposition is the core mathematical statement of DSS. Only $W$ 7 must be representable in FP8 and stored; $W$ 8 is transient and exists solely to improve code assignment. The paper’s interpretation is that DSS compensates for FP8 scale quantization and for the asymmetry between offline quantization and inference-time dequantization.

4. Search procedure and algorithmic characteristics

After a CJSO update at iteration $W$ 9, DSS initializes the decoupled search from the coupled point by setting

$\alpha$ 0

It then constructs two search spaces per block. The dequantization-scale candidate set is restricted to the two nearest FP8 neighbors of the current $\alpha$ 1: $\alpha$ 2 The quantization-scale candidate set is a multiplicative grid around the same center: $\alpha$ 3 For every candidate pair $\alpha$ 4, DSS computes

$\alpha$ 5

reconstructs

$\alpha$ 6

and evaluates

$\alpha$ 7

The selected update is

$\alpha$ 8

This objective is purely local and block-wise, and the paper emphasizes that no calibration data are required (Bao et al., 12 May 2026).

The search is intentionally small. Per block, $\alpha$ 9 and $\Delta_i^d$ 0, giving approximately 202 candidate pairs. Because the block size is 16, each evaluation touches only 16 elements. The paper therefore characterizes the search as light, and reports that overall quantization remains practical, with Qwen3-8B quantized in approximately 36 minutes on an A800 GPU for the full SOAR procedure.

Algorithmically, DSS is a local search, not a globally optimal solver for the joint discrete-continuous problem. The paper states this limitation explicitly. It also states that SOAR uses up to 15 outer iterations with early stopping when relative MSE improvement is less than $\Delta_i^d$ 1.

5. Empirical behavior and ablations

The paper reports a component ablation on LLaMA-3.2-3B-Instruct under NVFP4. Relative to the NVFP4 baseline, CJSO alone improves the 5-task average accuracy from 65.02 to 65.64, while DSS alone improves it to 65.50. The combined SOAR system reaches 66.00, with WikiText2 perplexity reduced from 11.98 to 11.88 and C4 perplexity reduced from 15.53 to 15.44 (Bao et al., 12 May 2026).

Method	WikiText2 / C4 PPL	5-task Avg. Acc
NVFP4 baseline	11.98 / 15.53	65.02
+CJSO	12.04 / 15.53	65.64
+DSS (without CJSO)	11.96 / 15.45	65.50
SOAR (CJSO + DSS)	11.88 / 15.44	66.00

The numerical pattern matters. CJSO alone gives a +0.62 point gain over the NVFP4 baseline, DSS alone gives +0.48, and the combination gives +0.98. The paper interprets this as evidence that DSS is complementary to CJSO rather than redundant with it.

DSS is also reported to generalize beyond NVFP4 to MXFP4. On LLaMA-3.2-3B-Instruct, the MXFP4 baseline average accuracy is 62.46 and MXFP4+DSS reaches 63.32. On Qwen3-4B, the average improves from 61.80 to 62.27. On LLaMA-3.1-1B-Instruct, it improves from 53.75 to 54.77. The paper presents this as evidence that scale quantization is a general limitation of microscaling formats and that decoupling systematically helps.

For Qwen3-8B W4A4 NVFP4, SOAR achieves average accuracy 70.68, compared with 68.75 for the baseline NVFP4 implementation, 69.15 for 4over6, and 70.12 for RaZeR. The paper states that these gains are achieved with the same memory footprint and no extra runtime overhead, because the extra quantization scale $\Delta_i^d$ 2 is not stored.

6. Scope, limitations, and terminological boundaries

Several properties of DSS are easy to misstate. First, DSS does not alter the inference-time NVFP4 storage layout. The hardware-visible representation remains FP4 weights, FP8 block scales, and an FP32 global scale. The additional quantization scale is an offline optimization variable only. Second, DSS is calibration-free by default: both DSS and CJSO are weight-reconstruction-based, and the paper only notes activation-aware objectives with calibration data as a possible extension (Bao et al., 12 May 2026).

The method’s limitations are also explicit. DSS is a local search around CJSO solutions, not a globally optimal algorithm. Its search ranges are tied to the E2M1 FP4 mapping and E4M3 FP8 scale format described in the paper, so other FP4 variants may require adjusted ranges. Although the costs are described as reasonable, DSS adds overhead relative to CJSO alone; the paper’s position is that this is a one-time offline cost.

The acronym itself is overloaded in adjacent literatures. “DSS” in this context means Decoupled Scale Search within SOAR and NVFP4 quantization (Bao et al., 12 May 2026). In other papers contained in the same source set, “DSS” denotes “Deep Scale-spaces” for scale-equivariant CNNs (Worrall et al., 2019), a “DSS-indicator” for training-free Transformer architecture search (Zhou et al., 2022), and “Decoupled Synchronisation” in federated causal diffusion (Li et al., 21 Jun 2026). Those uses are terminologically unrelated to SOAR’s scale-decoupling procedure.

Taken in its own setting, DSS is best understood as a hardware-compatible refinement layer for microscaling quantization. It exploits a specific asymmetry: the scale used to choose FP4 codes need not be numerically identical to the scale the hardware stores and applies at inference. By converting that asymmetry into a discrete joint search over $\Delta_i^d$ 3 and $\Delta_i^d$ 4, DSS mitigates scale quantization error while preserving the standard NVFP4 deployment format.