HiFloat4 (HiF4): 4-Bit Quantized FP Format

• HiFloat4 is a 4-bit block-floating point format that uses shared hierarchical scaling metadata to extend dynamic range in quantized neural networks.
• It employs a multi-level scaling strategy over 64-element groups to balance dynamic range and precision, thereby enhancing robustness against activation outliers.
• HiFloat4 demonstrates competitive performance in LLM inference and diffusion models, offering improved accuracy and efficiency over traditional low-bit formats.

Searching arXiv for papers on HiFloat4 / HiF4 to ground the article. I’m unable to directly invoke the arXiv search tool in this interface, so I will rely on the provided arXiv records as the grounded source set for the article. HiFloat4, commonly abbreviated HiF4, denotes a set of low-bit numerical representations used in quantized machine learning, especially for 4-bit inference and training in LLMs and diffusion-transformer systems. In the dominant contemporary usage, HiF4 is a block floating-point design in which very small per-element payloads are combined with shared scaling metadata, typically across 64-element groups, to obtain wider dynamic range than uniform INT4 while preserving a highly regular GEMM datapath (Luo et al., 11 Feb 2026, Taghian et al., 9 Apr 2026). The term is not fully standardized across the literature: some papers define HiF4 through a three-level scaling hierarchy with E6M2 and 1-bit micro-exponents, some Ascend-oriented evaluations describe a UE6M2/E1/E1 + INT3 realization, and Wan2.2 challenge reports use either a toolkit-defined black-box HiFloat4 operator or a simplified block-exponent formulation for W4A4 video generation (Zhao et al., 13 Feb 2026, Feng et al., 26 May 2026, Chen et al., 28 Jun 2026). An older and unrelated usage also applies the name to exact 32-bit compression of selected IEEE-754 double values via table lookup (Neal, 2015).

1. Terminological scope and historical usage

In current low-precision deep-learning research, HiF4 is primarily associated with 4-bit floating-point or block-floating computation for LLM inference, post-training quantization, and FP4 training on specialized accelerators (Luo et al., 11 Feb 2026, Taghian et al., 9 Apr 2026). The central motivation is to combine the storage and throughput advantages of 4-bit arithmetic with stronger tolerance to activation and weight outliers than uniform integer quantization.

The nomenclature is, however, not singular. Luo et al. define HiF4 as a 64-element block with 32 bits of hierarchical shared scaling metadata, averaging 4.5 bits per value (Luo et al., 11 Feb 2026). An Ascend evaluation paper describes HiF4 as a three-level hierarchy with block-level scale $S_1$ in unsigned E6M2, sub-block scale $S_2$ in E1, micro-block scale $S_3$ in E1, and element quantization in 3-bit two’s-complement, again yielding 4.5 bits per element (Zhao et al., 13 Feb 2026). Wan2.2 challenge reports sometimes treat HiFloat4 as a fixed toolkit-defined mini-float and sometimes present an explicit block-exponent encoding for W4A4 inference (Feng et al., 26 May 2026, Chen et al., 28 Jun 2026).

A further source of ambiguity is historical. In a separate, non-neural-compression context, HiFloat4 is used for a 32-bit code consisting of the sign, exponent, and high-order mantissa bits of a double, with the missing low 32 mantissa bits restored by table lookup so that selected decimal-oriented subsets decode exactly to 64-bit IEEE values (Neal, 2015). This suggests that “HiFloat4” functions less as a single immutable standard than as a reused label for compact numerical encodings.

2. Numerical organizations in the modern FP4 literature

The most detailed modern specification appears in the language-model inference paper by Luo et al. A HiF4 block contains 64 values, each stored in 4 bits, plus 32 bits of shared metadata: 8 bits for a global base scale in E6M2, 8 bits for eight 1-bit level-2 micro-exponents, and 16 bits for sixteen 1-bit level-3 micro-exponents, for a total of 288 bits per block and 4.5 bits/value on average (Luo et al., 11 Feb 2026). In that formulation, each element uses sign-magnitude S1P2, equivalently an FP-style E1M2 payload, and reconstruction is

$$V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.$$

SharQ’s HiF4 summary adopts the same 64-element organization and makes the element set explicit as $$\{\pm 0.00, \pm 0.25, \pm 0.50, \pm 0.75, \pm 1.00, \pm 1.25, \pm 1.50, \pm 1.75\}$$, with E6M2 as the global scale and two binary micro-exponent levels (Meng et al., 25 Jun 2026). The Ascend pre-training paper is closely aligned, describing a global E6M2 scale with bias = 48, plus E1_8 and E1_16, and an S1P2 payload equivalent to E1M2 (Taghian et al., 9 Apr 2026).

By contrast, the Ascend evaluation paper presents a different low-level realization: UE6M2/E1/E1 + INT3, with sign implicit in the INT3 two’s-complement code and dequantization written as

$$X_{h_i} = \mathrm{sign}(x_i)\cdot S_1 \cdot S_2 \cdot S_3 \cdot (q_i / 2^3),$$

where $q_i \in \{-4,\dots,+3\}$ (Zhao et al., 13 Feb 2026). This is numerically similar in spirit—hierarchical shared scaling plus tiny element payloads—but not identical in payload semantics to the S1P2/E1M2 formulation.

Wan2.2 challenge work introduces yet another representation layer. One report treats HiF4 purely as a challenge-specified black-box quantize/dequantize operator (Feng et al., 26 May 2026). Another presents a block-exponent variant in which each value carries a 4-bit payload with 1 sign bit and 3-bit unsigned significand, together with one 4-bit exponent $E_b$ per block, using

$$\hat x = (-1)^s \times \bigl(m/2^3\bigr)\times 2^{E_b-B},$$

with bias $B=7$ and blocks reaching up to $S_2$0 weights (Chen et al., 28 Jun 2026).

Source line	Organization	Distinguishing feature
(Luo et al., 11 Feb 2026)	64 elements + 32 metadata bits	E6M2 + E1_8 + E1_16 + S1P2
(Zhao et al., 13 Feb 2026)	64-element hierarchy	UE6M2/E1/E1 + INT3
(Chen et al., 28 Jun 2026)	Per-value 4-bit payload + shared block exponent	Simplified block-exponent HiF4 for Wan2.2 W4A4

3. Quantization, dequantization, and range management

In Luo et al., quantization of a 64-value block proceeds by three-level peak reduction: first compute 16 local maxima over consecutive 4-element groups, reduce them to 8 maxima over 2×2 groups, and then to a global peak $S_2$1 (Luo et al., 11 Feb 2026). The peak is normalized to HiF4’s representable upper bound 7.0, quantized into E6M2, and its reciprocal is used to derive the level-2 and level-3 binary exponents. The level-2 bit for an 8-element subgroup is set when the subgroup peak times the reciprocal is $S_2$2; the level-3 bit for a 4-element group is set when the level-2-adjusted peak is $S_2$3. Each element is then rounded to S1P2, with clamping if necessary (Luo et al., 11 Feb 2026).

This max-based hierarchical-scale selection reappears in SharQ’s HiF4 extension. There, a block maximum $S_2$4 produces an ideal scale factor $S_2$5, which is rounded to the nearest representable E6M2 value; binary micro-exponents are then chosen at the 4 and 2 thresholds, and quantization is explicitly symmetric and zero-point-free (Meng et al., 25 Jun 2026). SharQ applies this both to the sparse backbone and the dense residual in its sparse–dense decomposition.

For post-training quantization, HiF4 is commonly paired with range-balancing transforms. In the Wan2.2 challenge summary, per-channel SmoothQuant-style folding computes

$S_2$6

with $S_2$7 in early FFNs, and rewrites the linear map as

$S_2$8

so that activation tails are reduced before 4-bit quantization (Chen et al., 28 Jun 2026). The same report then ranks channels by $S_2$9 and places the top-$S_3$0 in a MixQ-style outlier branch kept in FP16 (or BF16), while the bulk remains strict W4A4 (Chen et al., 28 Jun 2026).

A related Wan2.2 report uses activation-tail-aware percentile calibration rather than worst-case maxima. For each channel it replaces the activation range estimate by the empirical $S_3$1-th percentile,

$S_3$2

and constructs a balancing mask

$S_3$3

so that rare calibration outliers do not dominate the full 4-bit dynamic range (Feng et al., 26 May 2026).

4. Arithmetic structure and hardware mapping

A defining property of modern HiF4 is that its hierarchical metadata permits matrix products to be executed in a highly fixed-point manner. For two 64-element HiF4 blocks, the dot product in Luo et al. factors into a product of the two global E6M2 scales and an integer-weighted sum over element mantissas and micro-exponent shifts (Luo et al., 11 Feb 2026). In the hardware pipeline described there, level-3 shifts are absorbed into the mantissas, subgroup partial sums are shifted by the combined level-2 exponents, and only the final accumulated integer is multiplied by the product of the two global floating scales. Compared to NVFP4, this requires only one floating-point multiplication (E6M2×E6M2) and one final large int × int, whereas NVFP4 requires four small FP multiplies, four large int multiplies, and FP accumulation; for equivalent 64-length dot cores, HiF4 adds only $S_3$4 of the extra area of NVFP4 and saves $S_3$5 dynamic power (Luo et al., 11 Feb 2026).

On Ascend NPUs, HiF4 is explicitly co-designed with the datapath. The pre-training paper states that each Cube-Unit PE can load 256 bits at once—exactly 64 FP4 values—and perform a fused dot-product + accumulation in one cycle, so a HiF4 block maps one-to-one to the PE datapath (Taghian et al., 9 Apr 2026). The Ascend evaluation further notes that specialized instructions implement the three-level scale as integer additions of exponents plus bit shifts, avoiding floating-point multiplies in the inner loop and yielding a $S_3$6–$S_3$7 throughput boost vs. 8-bit modes; W4A4 HiF4 end-to-end latency is reported as ~60% of INT8 end-to-end on 8×Ascend 910 (Zhao et al., 13 Feb 2026).

In training, stabilization requirements differ by format. The Ascend pre-training study finds that HiF4 can maintain relative loss within $S_3$8 of BF16 using Random Hadamard Transform (RHT) on $S_3$9 and nearest rounding elsewhere, whereas MXFP4 typically requires stochastic rounding, RHT, and truncation-free scaling in combination (Taghian et al., 9 Apr 2026). On Blackwell GPUs, SharQ adds a different systems layer: a fused preparation kernel that combines normalization, 2:4 mask generation, sparse extraction, dense residual construction, and HiF4 quantization before launching a sparse HiF4 GEMM and a dense HiF4 GEMM (Meng et al., 25 Jun 2026).

5. Behavior in language-model inference and pre-training

Across LLM inference benchmarks, HiF4 is consistently presented as more robust than uniform 4-bit quantization and competitive with other low-bit floating formats. In Luo et al., for small models (7–14 B) over 8 tasks, the average accuracy drop versus BF16 is –1.81% for NVFP4 (direct cast), –1.68% for NVFP4 + PTS, –1.07% for HiF4 (direct cast), and –0.74% for HiF4 + HiGPTQ (Luo et al., 11 Feb 2026). For large models (560–671 B) over 10 tasks, HiF4 (direct) is reported as +0.48% on LongCat-560 B and +0.98% on DeepSeek-V3.1-671 B relative to BF16, while NVFP4 (direct) crashes on Mistral and LongCat (Luo et al., 11 Feb 2026). The paper’s stated interpretation is that HiF4 outperforms NVFP4 in all models and is more robust to outliers.

The Ascend evaluation reaches a similar conclusion from end-to-end W4A4 inference. On Qwen3-8B (2 K context, zero-shot), HiF4 reports PPL = 10.30 / 16.55 and acc = 96.5% (Δ3.5%), while SmoothQuant or SVDQuant improves the HiF4 gap to 2.0–2.2% versus BF16 (Zhao et al., 13 Feb 2026). On openPangu-7B, HiF4 gives Δ = 3.0% (RTN) and 2.7% (SmoothQuant/SVDQuant), and in KV-cache and attention quantization it remains usable where MXFP4 collapses on openPangu-7B with Δ>60% (Zhao et al., 13 Feb 2026).

In pre-training, HiF4 is used at much larger scale. On OpenPangu-1B, Llama3-8B, and Qwen3-MoE-30B, the Ascend study reports relative loss errors versus BF16 of 1.19%, 0.85%, and 0.88% for HiF4, compared with 1.79%, 1.44%, and 1.55% for MXFP4 (Taghian et al., 9 Apr 2026). The same work states that running all GEMMs in 4 bits yields up to 4× higher matrix-throughput and a 2.5× reduction in memory bandwidth compared to BF16, and that inactive experts in the MoE model stored in FP4 reduce footprint by ≈60% (Taghian et al., 9 Apr 2026).

SharQ shows that HiF4 also supports structured activation sparsity. On Qwen3-30B-A3B, HiF4 + RTN gives 76.38 on HellaSwag, 63.28 on Lambada, 70.17 on WinoGrande, 87.61 on BoolQ, and 9.08 on WikiText2, while SharQ + HiF4 improves these to 77.22, 64.08, 70.24, 88.07, and 8.92, respectively (Meng et al., 25 Jun 2026). SharQ’s broader results state that the method recovers 43–63% of the NVFP4-to-FP16 accuracy gap across formats and delivers 2.2–2.4$$V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.$$0 latency reduction over FP16 and 1.2–1.4$$V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.$$1 throughput improvement over FP8 in language-model serving (Meng et al., 25 Jun 2026).

Setting	Reported HiF4 outcome	Source
Small-model LLM inference	Direct cast: –1.07% avg. accuracy drop; HiGPTQ: –0.74%	(Luo et al., 11 Feb 2026)
Large-model LLM inference	Direct cast: +0.48% on LongCat, +0.98% on DeepSeek	(Luo et al., 11 Feb 2026)
Ascend pre-training	Relative loss error 1.19%, 0.85%, 0.88% on OpenPangu-1B, Llama3-8B, Qwen3-MoE-30B	(Taghian et al., 9 Apr 2026)
Ascend W4A4 inference	Qwen3-8B acc 96.5%; SmoothQuant/SVDQuant gap improves to 2.0–2.2%	(Zhao et al., 13 Feb 2026)
SharQ + HiF4	HellaSwag 77.22; WikiText2 8.92	(Meng et al., 25 Jun 2026)

6. Diffusion transformers, Wan2.2, and video-generation pipelines

HiF4 has also been adopted in diffusion-transformer quantization. In the Wan2.2 post-training report “Tail-Aware HiFloat4”, the target system is Wan2.2-I2V-A14B with two diffusion-transformer modules; in each transformer, all attention and feed-forward linear projections (400 layers per transformer) are quantized to W4A4 HiFloat4, while boundary layers remain in BF16 (Feng et al., 26 May 2026). Calibration is run on 16 prompts, activation percentiles are used to form the mask-only PTQ state, and the saved state stores only masks $$V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.$$2 and HiF4’s metadata, not an entire 4-bit weight copy (Feng et al., 26 May 2026). The reported official video metrics compare BF16 with HiF4 W4A4 as follows: Imaging quality 0.7027 → 0.6507, Aesthetic quality 0.5456 → 0.5458, Overall consistency 0.2263 → 0.2308, Subject consistency 0.9331 → 0.5324, Motion smoothness 0.9923 → 0.9803, and Unweighted mean 0.6800 → 0.5880 (Feng et al., 26 May 2026). The largest drop is therefore in subject consistency.

A separate Wan2.2 challenge submission combines three ingredients: SmoothQuant-style per-channel smoothing, MixQ-style mixed precision for sparse activation outliers, and block-wise HiF4 packing for feed-forward linear layers (Chen et al., 28 Jun 2026). After smoothing, each FFN weight matrix is packed into 32×32 tiles or 128×1 stripes, and a dual-branch GEMM preserves a tiny outlier set in FP16 (or BF16) while the bulk of channels use strict W4A4 (Chen et al., 28 Jun 2026). On official VBench I2V metrics, the paper reports FP16 baseline scores of 0.5445, 0.9626, 0.7086, 0.9730, and 0.9199 for Aesthetic, I2V subject, Imaging, Motion, and Subject consistency, respectively; Native HiF4 W4A4 drops by about 5.0% on all five axes, whereas Ours (W4A4) yields 0.5274 (−3.1%), 0.9375 (−2.6%), 0.6936 (−2.1%), 0.9769 (+0.4%), and 0.8875 (−3.5%) (Chen et al., 28 Jun 2026). The paper’s summary is that the pipeline stays within 2–3.5 percent of FP16 on most quality axes and improves motion smoothness.

A broader diffusion-transformer connection appears in HQ-DiT, which describes a HiFloat4-style 4-bit floating-point quantization rather than the 64-element hierarchical HiF4 specification. There, FP4 uses 1 sign bit and a tensor-dependent exponent/mantissa split, activations are fixed to E2M1, and a random orthogonal Hadamard matrix is used as a universal identity transform for outlier mitigation (Liu et al., 2024). The reported result is that HQ-DiT is the first instance where both weights and activations in DiTs are quantized to just 4 bits, with only a 0.12 increase in sFID on ImageNet (Liu et al., 2024). This suggests a broader FP4 design ecosystem in which HiF4-style ideas circulate beyond a single binary encoding.

7. Limitations, misconceptions, and comparative position

A common misconception is that HiF4 refers to one universally fixed 4-bit format. The literature does not support that reading. Some papers specify E6M2 + E1_8 + E1_16 + S1P2, others describe UE6M2/E1/E1 + INT3, and some challenge reports intentionally abstract HiFloat4 as a toolkit-defined operator (Luo et al., 11 Feb 2026, Zhao et al., 13 Feb 2026, Feng et al., 26 May 2026). A plausible implication is that HiF4 should be read as a format family or implementation family, not merely as one canonical bit layout.

The principal technical trade-off is range versus granularity. The 64-element grouping and hierarchical metadata deliver wide effective dynamic range, but they also make local scaling coarser than smaller-block alternatives. SharQ states this directly: HiF4’s 64-element group gives very wide dynamic range vs. a smaller local scale granularity, and in practice one sees slightly larger residuals than NVFP4 (group-32) on some layers (Meng et al., 25 Jun 2026). The same paper notes that because E6M2 supports only normal modes, very small peaks (< $$V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.$$3) underflow to zero, although this is described as rare in LLM activations (Meng et al., 25 Jun 2026).

Another recurrent issue is outlier sensitivity. Modern HiF4 pipelines nearly always add explicit countermeasures—SmoothQuant folding, percentile clipping, MixQ-style higher-precision branches, Random Hadamard Transform, or SharQ’s sparse–dense decomposition—rather than relying on naive round-to-nearest alone (Chen et al., 28 Jun 2026, Feng et al., 26 May 2026, Taghian et al., 9 Apr 2026, Meng et al., 25 Jun 2026). This does not imply that the format is ineffective; rather, it indicates that HiF4 is typically deployed as part of a larger quantization system.

Relative to adjacent formats, the papers position HiF4 as a middle ground between extremely cheap but fragile integer quantization and more overhead-heavy low-bit floating formats. The Ascend evaluation states that INT4 collapses, that hierarchical formats (NVFP4, HiF4) are essential in the 4-bit regime, and that HiF4’s 64/8/4 element hierarchy balances dynamic range and local precision, maintaining ≥96.5% accuracy on Qwen3-8B and ≥97.0% on openPangu-7B in W4A4 inference (Zhao et al., 13 Feb 2026). Luo et al. go further and report that HiF4 achieves higher average accuracy than the state-of-the-art NVFP4 format across multiple models and tasks (Luo et al., 11 Feb 2026). The overall comparative picture is therefore favorable, but strongly conditioned on calibration, outlier handling, and hardware support.