HiFloat4: 4-bit Floating-Point Quantization

• HiFloat4 is a family of 4-bit floating-point schemes that use hierarchical block scaling and shared exponents for effective low-precision inference and training.
• It employs a three-level scaling hierarchy with 64-element blocks and metadata to balance dynamic range and precision across various quantization workflows.
• Variations of HiFloat4 include fully specified Ascend formats, challenge-driven block-floating models, and toolkit-defined operators, each optimizing memory and bandwidth usage.

HiFloat4, usually abbreviated HiF4, denotes a family of 4-bit floating-point or block-floating numerical schemes used for low-precision inference and training. In the most explicit published formulation, HiF4 is an Ascend-oriented block format that packs 64 low-precision values together with 32 bits of shared scaling metadata, averaging 4.5 bits per value (Luo et al., 11 Feb 2026). In other contexts, especially Wan2.2 quantization challenge reports, the name denotes either a block-wise shared-exponent W4A4 runtime or a toolkit-supplied quantize–dequantize operator whose internal codebook is not redefined by the method (Chen et al., 28 Jun 2026, Feng et al., 26 May 2026). Across these usages, the unifying objective is to retain the memory and bandwidth advantages of 4-bit arithmetic while preserving more dynamic range than conventional low-bit integer quantization.

1. Scope and terminological usage

The literature does not use the term HiFloat4 in a completely uniform way. Some works treat it as a fully specified numerical format, some as a challenge-mandated runtime abstraction, and some as a broader shorthand for low-bit floating-point quantization.

Context	HiFloat4 meaning	Reported treatment
Ascend LM inference/training	Concrete hierarchical 4-bit format	64 values + 32-bit metadata; three-level scaling (Luo et al., 11 Feb 2026, Taghian et al., 9 Apr 2026)
Wan2.2 W4A4 challenge	Block-floating shared-exponent format	Per-block exponent metadata with 4-bit payloads (Chen et al., 28 Jun 2026)
Wan2.2 PTQ toolkit usage	Operator view	Fixed black-box quantize–dequantize operator (Feng et al., 26 May 2026)

This terminological spread is explicit in the source material. The Wan2.2 PTQ report states that HiFloat4 is treated as a fixed, black-box operator and does not expose internal bitfields or exact encoding rules (Feng et al., 26 May 2026). By contrast, the Ascend inference paper gives a full data-format specification, including group size, metadata layout, scaling hierarchy, and a decode rule (Luo et al., 11 Feb 2026). The Wan2.2 W4A4 submission occupies an intermediate position: it does specify the runtime semantics used in the paper, but those semantics are block-floating rather than per-value floating-point, with the exponent shared across a block and the 4-bit payload functioning as a signed or effectively signed coefficient (Chen et al., 28 Jun 2026).

A related source of ambiguity is that some FP4 training papers use “HiFloat4” only interpretively rather than as a separately specified standard. One such paper explicitly states that it does not define HiFloat4 and maps the term to NVFP4 for the purposes of its discussion (Chmiel et al., 25 May 2025). This suggests that the name has both a narrow meaning, tied to Ascend HiFloat formats, and a wider colloquial meaning, referring to practical FP4 deployment schemes.

2. Numerical organization and representation

In the Ascend inference formulation, one HiF4 unit contains 64 in-group elements encoded in 4-bit S1P2 sign-magnitude form and 32 bits of metadata carrying a three-level scaling hierarchy: an unsigned FP8 E6M2 global scale, eight 1-bit micro-exponents over 8-element groups, and sixteen 1-bit micro-exponents over 4-element groups (Luo et al., 11 Feb 2026). The decoded value of element $i$ is

$$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$

where $s_0$ is the E6M2 global scale, $e_1$ and $e_2$ are the micro-exponents, and $a_i$ is the S1P2 element value (Luo et al., 11 Feb 2026). The training-oriented Ascend description is closely aligned: it also uses 64-value blocks, 32 bits of metadata, and a three-level hierarchy, but describes the per-value code as S1P2 equivalent to E1M2, with the reconstructed value written as $$V(i) = S_0 \cdot 2^{e_2[g(i)] + e_3[h(i)]} \cdot (-1)^s \cdot 2^{e-1} \cdot (1 + m \cdot 2^{-2})$$ (Taghian et al., 9 Apr 2026).

This design is distinct from the block-floating HiF4 used in the Wan2.2 W4A4 challenge submission. There, HiF4 is used for weights in block-floating fashion: a block shares an exponent-derived scale $s_B$, each element stores a 4-bit payload, and quantization is described by

$$q = \operatorname{clip}(\operatorname{round}(w/s_B), q_{\min}, q_{\max}), \qquad \hat{w} = s_B \cdot q.$$

Under that construction, the sign and mantissa-like coefficient are encoded in the 4-bit payload, while the exponent is stored once per block as metadata; subnormals, NaNs, and infinities are not represented in the element payload (Chen et al., 28 Jun 2026). The same report emphasizes that it does not use per-value exponents.

The operator-view Wan2.2 PTQ report is more agnostic. It defines HiF4 as an external codebook $S_{\mathrm{HiF4}}$ and models fake quantization as nearest-level projection with immediate dequantization, $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$0 and $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$1, while stating that the internal bit allocation, exact dynamic range, rounding policy, and level spacing are not specified in the paper and must be taken from the toolkit (Feng et al., 26 May 2026). The practical implication is that “HiFloat4” can denote either a concrete hierarchical format or an abstract low-bit operator interface, depending on the system under study.

3. Quantization workflows and inference pipelines

HiFloat4 is typically embedded in broader quantization pipelines rather than used in isolation. In Wan2.2-I2V-A14B W4A4 inference, the reported pipeline combines SmoothQuant-style per-channel smoothing, MixQ-style mixed precision for sparse activation outliers, and block-wise HiF4 packing of feed-forward linear layers in both high-noise and low-noise Transformer stacks (Chen et al., 28 Jun 2026). Calibration on representative OpenS2V-5M batches collects per-input-channel activation maxima and weight-column ranges; heavy-tailed channels are scored by $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$2 and a static top-$$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$3 outlier set is formed per FFN layer (Chen et al., 28 Jun 2026). Smoothing folds per-channel scales into the weights through

$$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$4

after which the bulk path runs in strict W4A4 and a narrow dual-branch GEMM preserves outlier columns in FP16/BF16 (Chen et al., 28 Jun 2026).

The same Wan2.2 model was also quantized in a PTQ regime that adapts ViDiT-Q to HiFloat4 by using per-channel balancing and a percentile-based activation statistic. For each target layer and input channel, the activation proxy is defined as $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$5 rather than a hard maximum, the weight proxy is $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$6, and the balancing mask is

$$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$7

with $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$8 (Feng et al., 26 May 2026). The layer is then rewritten through $$V_i = s_0 \cdot 2^{e_1[g(i)] + e_2[h(i)]} \cdot a_i,$$9, quantized as $s_0$0 and $s_0$1, while normalization, embeddings, patch embedding, final projection, and output-head-style modules remain in BF16 for stability (Feng et al., 26 May 2026). This design is explicitly motivated by reducing the influence of rare calibration outliers without changing the runtime sampler or HiFloat4 arithmetic path.

For LLM inference on Ascend NPUs, HiF4 is presented as fully compatible with state-of-the-art PTQ frameworks. The Ascend evaluation integrates HiF4 with SmoothQuant, tuning $s_0$2 by grid search in $s_0$3 with step $s_0$4, and with SVDQuant using a rank-16 low-rank component (Zhao et al., 13 Feb 2026). The same work recommends choosing a quantization dimension divisible by 64 to match HiF4’s block requirement and using the native 64/8/4 scaling hierarchy for weights, activations, and KV-cache blocks (Zhao et al., 13 Feb 2026). A separate inference paper also reports an optional HiGPTQ, a GPTQ-style PTQ adapted to HiF4, as a further accuracy refinement path (Luo et al., 11 Feb 2026).

4. Training use, stabilization, and grid geometry

In large-scale LM pre-training on Ascend NPUs, HiFloat4 is used as a training-time FP4 format for linear and expert GEMM operations across dense and MoE architectures (Taghian et al., 9 Apr 2026). The reported recipe quantizes attention QKV/output projections, FFN FC1/FC2, and MoE expert up/down projections to FP4, keeps the output linear in higher precision, and applies Random Hadamard Transform to weight-gradients along the inner dimension with block size $s_0$5 (Taghian et al., 9 Apr 2026). The paper states that HiF4 requires only RHT for stabilization and that nearest rounding on gradients works best empirically; stochastic rounding degraded performance in its ablations (Taghian et al., 9 Apr 2026).

The same study contrasts HiF4 with MXFP4. HiF4’s three-level hierarchy is described as reducing the need for truncation-free scaling and lowering bias, whereas MXFP4 requires stochastic rounding, RHT, and truncation-free scaling to approach similar stability (Taghian et al., 9 Apr 2026). Reported training-loss relative errors against BF16 were 1.19% for OpenPangu-1B, 0.85% for Llama3-8B, and 0.88% for Qwen3-MoE-30B under HiF4, compared with 1.79%, 1.44%, and 1.55% for MXFP4 (Taghian et al., 9 Apr 2026).

A complementary theoretical analysis of FP4 training identifies Shrinkage Bias as a systematic toward-zero rounding bias caused by asymmetric bins in non-uniform grids such as E2M1 under RTNE (Zhao et al., 18 Jun 2026). That paper argues that uniform grids such as E1M2 or INT4 eliminate this geometric bias, and explicitly notes that Ascend’s HiFloat4 adopts a uniform S1P2 data element and is therefore naturally aligned with uniform-grid FP4 training recipes (Zhao et al., 18 Jun 2026). The broader claim is not that every system labeled “HiFloat4” is uniform, but that HiFloat-like formats with symmetric 4-bit codebooks and blockwise scaling are better matched to RHT-heavy training paths than E2M1-centered recipes (Zhao et al., 18 Jun 2026).

The wider FP4 training literature reinforces the point that nomenclature and implementation vary. One study presents fully quantized LLM training in predominantly 4-bit floating point but interprets “HiFloat4” as NVFP4, specifically E2M1 values with per-16-element E4M3 scales (Chmiel et al., 25 May 2025). Another introduces an E2M1 training framework with vector-wise scaling, a differentiable quantization estimator, and outlier clamping and compensation, again using “HiFloat4” only in a generic FP4 sense rather than as the Ascend hierarchical format (Wang et al., 28 Jan 2025). Taken together, these works suggest that HiFloat4 is best understood as a format family or deployment idiom rather than a universally fixed training standard.

5. Empirical performance across workloads

The strongest Wan2.2 W4A4 result reported in the provided literature comes from the structured HiF4 pipeline that combines SmoothQuant, MixQ, and block packing. On official VBench I2V metrics, the FP16 baseline scored Aesthetic 0.5445, I2V subject 0.9626, Imaging 0.7086, Motion 0.9730, and Subject consistency 0.9199. A native HiFloat4 baseline with uniform W4A4 degraded by approximately 5.0% on each reported axis. The structured HiF4 pipeline achieved Aesthetic 0.5274, I2V subject 0.9375, Imaging 0.6936, Motion 0.9769, and Subject consistency 0.8875, which was within 2–3.5% of FP16 on most axes and exceeded FP16 on motion smoothness by +0.4% (Chen et al., 28 Jun 2026).

A different Wan2.2 PTQ report produced a more mixed picture. In that setting, BF16 scored Imaging quality 0.7027, Aesthetic quality 0.5456, Overall consistency 0.2263, Subject consistency 0.9331, Motion smoothness 0.9923, and Unweighted mean 0.6800, while HiF4 W4A4 scored 0.6507, 0.5458, 0.2308, 0.5324, 0.9803, and 0.5880 respectively (Feng et al., 26 May 2026). The reported pattern was that aesthetic quality and motion smoothness remained close to BF16, whereas subject consistency suffered the dominant degradation (Feng et al., 26 May 2026). This suggests that runtime format choice alone does not determine quality; calibration strategy, boundary-module retention, and outlier handling materially affect outcomes.

In LLM inference on Ascend, HiF4 is reported to prevent the severe failure modes observed for INT4 in 4-bit weight–activation and KV-cache quantization. On Qwen3-8B W4A4 with RTN, NVFP4 achieved PPL 10.16/16.24 and average zero-shot score 0.971, while HiF4 achieved PPL 10.30/16.55 and average 0.965; with SmoothQuant, NVFP4 reached 0.983 average and HiF4 0.978; with SVDQuant, both reached 0.980 (Zhao et al., 13 Feb 2026). On openPangu-7B W4A4, however, HiF4 limited degradation to 3.0% versus 5.7% for NVFP4 and 8.6% for MXFP4 under RTN, and retained an advantage under SmoothQuant and SVDQuant (Zhao et al., 13 Feb 2026). For KV-cache quantization on Qwen3-8B with SmoothQuant W-A, HiF4 was also best at 3.15% loss for Q16KV4 and 3.92% for QKV4 (Zhao et al., 13 Feb 2026).

A second inference study reports average downstream accuracy advantages for HiF4 over NVFP4 across both small and very large LMs. On the four small-model average, BF16 scored 72.99, NVFP4+PTS scored 71.38, HiF4 scored 71.87, and HiF4+HiGPTQ scored 72.23 (Luo et al., 11 Feb 2026). On DeepSeek-V3.1, the mean score was 85.44 for BF16, 85.24 for NVFP4+PTS, and 86.42 for HiF4; on LongCat, BF16 scored 81.32, NVFP4+PTS 77.81, and HiF4 81.80 (Luo et al., 11 Feb 2026). These results are reported alongside a synthetic Gaussian quantization-error comparison in which the MSE ratios normalized to HiF4 were 1 for HiF4, 1.32 for NVFP4, and 1.89 for MXFP4 (Luo et al., 11 Feb 2026).

In training, the Ascend HiF4 results are framed in terms of BF16-relative loss degradation rather than downstream inference metrics. The reported relative errors below 1% on Llama3-8B and Qwen3-MoE-30B, with lower degradation than MXFP4, position HiF4 as a practically stable FP4 training substrate under the specific recipe used there (Taghian et al., 9 Apr 2026).

6. Comparisons, limitations, and research directions

A recurring comparison in the literature is between HiFloat4 and low-bit integer formats. The Ascend evaluation summarizes three empirical claims: INT8 suits narrow-range data, floating-point formats excel with high-variance data, and HiF4’s hierarchical scaling prevents the accuracy collapse seen in 4-bit integer formats for weight–activation and KV-cache tasks (Zhao et al., 13 Feb 2026). That conclusion is consistent with earlier LLM PTQ work showing that at low bit-width the optimal format varies by layer and that floating-point and integer schemes should be treated as complementary rather than universally ordered alternatives (Zhang et al., 2023).

Against other floating formats, HiF4 is usually compared to MXFP4 and NVFP4. The Ascend training paper argues that HiF4’s hierarchy distributes precision across both block scale and element mantissae, while MXFP4 concentrates representational burden into coarser E2M1 elements and therefore needs more aggressive stabilization (Taghian et al., 9 Apr 2026). The HiF4 inference paper emphasizes a different trade-off: compared with NVFP4, HiF4 uses a larger 64-element group and more fixed-point-like matrix multiplication, which it associates with lower area and power overhead while maintaining or improving average accuracy (Luo et al., 11 Feb 2026). The Wan2.2 W4A4 challenge paper, by contrast, treats MXFP4 primarily as a format with finer-grained exponent control by design, but reports that block-wise HiF4 with SmoothQuant and MixQ already achieves most of the attainable quality in that setting (Chen et al., 28 Jun 2026).

Several limitations are consistently reported. In Wan2.2, the remaining gap to FP16 is concentrated in aesthetic and subject-consistency metrics under one pipeline, and in subject consistency under the tail-aware PTQ variant (Chen et al., 28 Jun 2026, Feng et al., 26 May 2026). Both Wan2.2 papers also report sensitivity to calibration coverage and hyperparameters: the structured W4A4 pipeline identifies $s_0$6 and the outlier budget $s_0$7 as important tunables, while the PTQ method notes sensitivity to the percentile statistic and calibration prompts (Chen et al., 28 Jun 2026, Feng et al., 26 May 2026). In Ascend training, RLHF/GRPO, long-context, and multimodal robustness are identified as open regimes not covered by the present results (Taghian et al., 9 Apr 2026).

A broader conceptual limitation is that “HiFloat4” does not denote a single immutable 4-bit representation across the literature. Some works specify a uniform S1P2-centered hierarchical block format, some use block-floating shared exponents, and some leave the internal codebook entirely to an external toolkit (Luo et al., 11 Feb 2026, Chen et al., 28 Jun 2026, Feng et al., 26 May 2026). A plausible implication is that future discussions of HiFloat4 will need to disambiguate whether they refer to the Ascend format, a challenge-runtime abstraction, or a generic FP4 deployment pattern. The most technically consistent direction emerging from the current papers is a combination of hierarchical or blockwise scaling, explicit outlier management, and, for training, uniform or symmetric 4-bit grids that avoid shrinkage-inducing geometric asymmetry (Zhao et al., 18 Jun 2026).