Closed-form Joint Scale Optimization (CJSO)

• CJSO is a closed-form analytical method that optimizes NVFP4 quantization scales by minimizing the reconstruction error with least-squares updates.
• It jointly refines a tensor-wide FP32 global scale and block-wise FP8 scales under fixed FP4 assignments, replacing heuristic methods with a rigorous quadratic minimization.
• The method efficiently computes scale updates while respecting hardware constraints (E4M3 representation), making it integral to the SOAR post-training quantization framework.

Closed-form Joint Scale Optimization (CJSO) is the analytical core of SOAR, a post-training quantization method for the NVFP4 microscaling format used for LLMs. In SOAR, CJSO replaces heuristic scale selection with a reconstruction-driven optimization of the two scale levels inherent to NVFP4: a tensor-wide global scale and block-wise local scales. Its defining feature is that, under fixed FP4 assignments, the reconstruction objective becomes quadratic in the scaling factors, which yields analytical least-squares updates for both the global and block-wise scales (Bao et al., 12 May 2026). The method is therefore “closed-form” at the level of the continuous scale subproblem, “joint” because the two scale levels are optimized within one coupled objective, and specific to NVFP4 because one of the scales must ultimately be projected to the FP8 (E4M3) format required by hardware.

1. Quantization setting and reconstruction objective

SOAR formulates CJSO in the standard NVFP4 hierarchical microscaling format. A full-precision tensor is partitioned into blocks of 16 elements, and quantization uses a tensor-wide FP32 global scale $\alpha$, a block-wise FP8 (E4M3) scale $\Delta_i$ for block $i$, and FP4 (E2M1) quantized values for the tensor entries (Bao et al., 12 May 2026). In the paper’s formulation, each weight $W_{ij}$ in block $i$ is reconstructed as

$\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$

where $Q_{ij}$ is the FP4 code value obtained by quantizing $W_{ij}/(\alpha\Delta_i)$.

CJSO directly minimizes the reconstruction error between original weights and dequantized weights. The objective is

$$\min_{\alpha,\{\Delta_i\}} \mathcal{L} = \sum_i \left\| W_i - Q_i \cdot (\alpha \Delta_i) \right\|_2^2,$$

and, in expanded elementwise form,

$$\min_{\alpha,\{\Delta_i\}} \mathcal{L} = \sum_{i=1}^N \sum_{j \in \text{block}_i} \left( W_{ij} - Q_{ij}\cdot (\alpha \Delta_i) \right)^2.$$

Here $\Delta_i$0 is the number of blocks, $\Delta_i$1 is weight $\Delta_i$2 in block $\Delta_i$3, $\Delta_i$4 is the tensor-wide global scale, $\Delta_i$5 is the block-wise scale for block $\Delta_i$6, and $\Delta_i$7 is the discrete FP4 quantization assignment.

A central assumption is stated explicitly: directly optimizing the reconstruction objective is challenging because $\Delta_i$8 is discrete and depends on both $\Delta_i$9 and $i$0; however, under a fixed assignment of $i$1, the reconstruction objective becomes a quadratic function with respect to the scaling factors (Bao et al., 12 May 2026). This fixed-$i$2 assumption is the condition that makes the closed-form derivation possible.

2. Jointness and coupling structure

CJSO is “joint” because it does not optimize the tensor-wide scale and the block scales independently via separate heuristics. Instead, it treats them as coupled variables in the same reconstruction objective. Both scales affect the normalized value $i$3, which determines the discrete FP4 assignment $i$4, and both scales affect the reconstruction magnitude $i$5 (Bao et al., 12 May 2026).

Even when $i$6 is temporarily held fixed, $i$7 and $i$8 remain multiplicatively coupled through $i$9. The global scale controls the tensor-wide overall dynamic range, while each $W_{ij}$0 adjusts local block-level fit. The paper’s coordinate-wise view is to fix current $W_{ij}$1, solve for optimal $W_{ij}$2 given all $W_{ij}$3, solve for each $W_{ij}$4 given $W_{ij}$5, and then recompute $W_{ij}$6 under updated scales. This means that “joint” refers to joint treatment of the two scale levels inside one reconstruction-minimization framework, even though the updates are applied in coordinate-wise closed form (Bao et al., 12 May 2026).

The paper is equally explicit about what CJSO does not provide. It is not the global optimum of the original mixed discrete-continuous problem; rather, it is the exact minimizer of the continuous subproblem conditioned on the current FP4 assignments. This makes the method piecewise-analytic rather than globally analytic over the full quantization problem.

3. Closed-form derivation and updates

Given fixed block scales $W_{ij}$7, the optimal global scale is

$W_{ij}$8

The appendix derives this by differentiating the fixed-$W_{ij}$9 objective with respect to $i$0, setting the derivative to zero, and rearranging the resulting normal equation (Bao et al., 12 May 2026). The paper interprets this as a least-squares scalar fit over the whole tensor, with the current block scales acting as weights.

With $i$1 fixed, each block scale can be optimized independently: $i$2 The appendix derives this by differentiating $i$3 with respect to $i$4, setting the derivative to zero for block $i$5, and solving the resulting scalar equation (Bao et al., 12 May 2026). Because NVFP4 requires block scales to be hardware-storable in E4M3, the analytical $i$6 is subsequently projected onto the nearest value in the FP8 (E4M3) format. The appendix further indicates a round-and-clip step to the valid E4M3 range.

Under fixed $i$7, the loss is quadratic in $i$8 and in each $i$9, so these updates are the least-squares minimizers. The paper therefore describes CJSO as providing exact continuous minimization for the current quantization assignments.

Several assumptions bound the validity of these formulas. The block partition is fixed at 16 elements. The derivation depends on fixed quantized assignments $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$0. The formulas assume the denominators are nonzero,

$\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$1

The paper does not explicitly discuss edge handling for all-zero $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$2 blocks or degenerate cases, although it notes that a practical implementation would need to guard against them (Bao et al., 12 May 2026).

4. Algorithmic procedure and relation to DSS

The practical CJSO procedure begins from standard NVFP4 max-based initialization. The tensor is partitioned into blocks of 16 elements, $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$3 is initialized via the standard NVFP4 max-based global rule, and $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$4 is initialized via the per-block max-based rule (Bao et al., 12 May 2026). Iteration then alternates between discrete reassignment of FP4 codes and analytical scale updates.

In the main text and appendix, the per-iteration structure is: $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$5 followed by the analytical global update

$\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$6

and the analytical block update

$\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$7

In pure CJSO, a single block scale conceptually participates in both quantization and dequantization. In the full SOAR method, this becomes separated because of Decoupled Scale Search (DSS). DSS introduces a high-precision quantization-side scale $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$8 and a hardware-constrained dequantization-side scale $\hat W_{ij} = Q_{ij}\cdot (\alpha \Delta_i),$9, and optimizes candidate pairs by discrete local search (Bao et al., 12 May 2026). The paper states the division of labor directly: CJSO is continuous, analytical optimization of the scale structure, whereas DSS compensates for the precision loss introduced by hardware-constrained FP8 scales.

The DSS search uses the two nearest E4M3 values around the CJSO analytical solution for dequantization candidates, and a multiplicative search range $Q_{ij}$0 with $Q_{ij}$1 step for quantization-scale candidates (Bao et al., 12 May 2026). In the full SOAR pipeline, the practical stopping rule is 15 iterations maximum with early stopping if relative MSE improvement falls below $Q_{ij}$2.

5. Empirical behavior and quantitative evidence

The clearest direct evidence attributable to CJSO is the ablation study “Effect of CJSO and DSS” on LLaMA-3.2-3B-Instruct. The paper reports the following results (Bao et al., 12 May 2026):

Method	Wiki2	C4	Avg. zero-shot
NVFP4 baseline	11.98	15.53	65.02
+CJSO	12.04	15.53	65.64

The text states that introducing CJSO yields consistent improvements over the NVFP4 baseline, particularly in downstream accuracy $Q_{ij}$3. The same ablation lists per-task changes from NVFP4 to $Q_{ij}$4CJSO as WinoGrande $Q_{ij}$5, PIQA $Q_{ij}$6, HellaSwag $Q_{ij}$7, Arc-E $Q_{ij}$8, and Arc-C $Q_{ij}$9 (Bao et al., 12 May 2026).

Figure 1 is described as showing convergence of reconstruction MSE, and the paper states that SOAR consistently reaches lower error floors than using CJSO strategy only. This identifies CJSO as a useful intermediate baseline and initialization for the complete SOAR procedure. An iteration study on Qwen3-8B further reports that the NVFP4 baseline average is $W_{ij}/(\alpha\Delta_i)$0, SOAR with Iter$W_{ij}/(\alpha\Delta_i)$1 reaches $W_{ij}/(\alpha\Delta_i)$2, and SOAR with Iter$W_{ij}/(\alpha\Delta_i)$3 reaches $W_{ij}/(\alpha\Delta_i)$4, which the paper uses as indirect evidence that the analytical CJSO step is effective early in the iteration schedule (Bao et al., 12 May 2026).

The paper also emphasizes that the full method preserves the same inference format as standard NVFP4, with no memory overhead: $W_{ij}/(\alpha\Delta_i)$5 is not stored, and the dequantization scale remains FP8 E4M3. Quantization times reported for the full SOAR method are $W_{ij}/(\alpha\Delta_i)$6 min for LLaMA-3.2-1B, $W_{ij}/(\alpha\Delta_i)$7 min for LLaMA-3.2-3B, $W_{ij}/(\alpha\Delta_i)$8 min for LLaMA-3.1-8B, $W_{ij}/(\alpha\Delta_i)$9 min for Qwen3-4B, and $$\min_{\alpha,\{\Delta_i\}} \mathcal{L} = \sum_i \left\| W_i - Q_i \cdot (\alpha \Delta_i) \right\|_2^2,$$0 min for Qwen3-8B, with the paper stating that quantization for all evaluated models finishes within 40 minutes (Bao et al., 12 May 2026).

6. Scope, limitations, and related closed-form joint optimization literature

CJSO is a quantization-specific method for NVFP4. It does not optimize block structure, because the block partition is fixed by the format at 16 elements. It is not a global solver for the original mixed discrete-continuous problem, because the closed form applies only after freezing the FP4 assignments $$\min_{\alpha,\{\Delta_i\}} \mathcal{L} = \sum_i \left\| W_i - Q_i \cdot (\alpha \Delta_i) \right\|_2^2,$$1. It is also constrained by hardware, because the block dequantization scale must be E4M3-representable at inference time (Bao et al., 12 May 2026).

These limitations are central to a common misconception. CJSO is not a generic “joint scale optimization” method over arbitrary multiscale decision variables. It is a piecewise-analytic least-squares solver for NVFP4 scales inside a specific post-training quantization framework. This suggests that its main conceptual contribution lies in showing how alternating discrete reassignment and exact continuous minimization can outperform deterministic max-based scaling without changing the deployed inference format.

The term CJSO appears explicitly in SOAR, but related papers use analogous closed-form joint optimization ideas in other domains without the same name. In robust control, a closed-form controller for a class of full-information $$\min_{\alpha,\{\Delta_i\}} \mathcal{L} = \sum_i \left\| W_i - Q_i \cdot (\alpha \Delta_i) \right\|_2^2,$$2 problems yields an explicit global optimum for structured interconnected systems, but the paper is framed as controller synthesis rather than generic joint scale optimization (Bergeling et al., 2019). In radial power distribution, analytical node-level solutions replace embedded nonlinear solvers in a distributed online OPF framework, but the closed form is local and objective-specific rather than a tensor-wide scale optimization rule (Sadnan et al., 2021). In fixed-noise probabilistic PLS, exact Stiefel optimization is combined with closed-form updates for diagonal latent scale parameters, but the paper does not introduce a method called CJSO (Hu et al., 12 May 2026). In positioning-assisted beamforming, closed-form optimization of a joint Gaussian beam pattern matrix yields explicit width and rotation parameters, again strongly related in structure but not in terminology (Liu et al., 4 Mar 2026).

Within quantization research, SOAR’s presentation is more specific. CJSO is the stage that analytically optimizes the scale structure, while DSS addresses the mismatch created by storing scales in low-precision FP8. Taken together, they form SOAR; taken alone, CJSO is best understood as the exact minimization of the current quadratic reconstruction subproblem under fixed FP4 assignments (Bao et al., 12 May 2026).