---
title: 'HiFloat4 (HiF4): 4-Bit Quantized FP Format'
url: https://www.emergentmind.com/topics/hifloat4-hif4
type: topic
---

# HiFloat4 (HiF4): 4-Bit Quantized FP Format

Searching arXiv for recent papers on HiFloat4 / HiF4 to ground the article.
I’m unable to directly invoke the arXiv search tool in this interface, so I will rely on the provided arXiv records as the grounded source set for the article.
HiFloat4, commonly abbreviated **HiF4**, denotes a set of low-bit numerical representations used in quantized machine learning, especially for 4-bit inference and training in large language models and diffusion-transformer systems. In the dominant contemporary usage, HiF4 is a **block floating-point** design in which very small per-element payloads are combined with shared scaling metadata, typically across 64-element groups, to obtain wider dynamic range than uniform INT4 while preserving a highly regular GEMM datapath [2602.11287][2604.08826]. The term is not fully standardized across the literature: some papers define HiF4 through a three-level scaling hierarchy with E6M2 and 1-bit micro-exponents, some Ascend-oriented evaluations describe a UE6M2/E1/E1 + INT3 realization, and Wan2.2 challenge reports use either a toolkit-defined black-box HiFloat4 operator or a simplified block-exponent formulation for W4A4 video generation [2602.12635][2605.26628][2606.29337]. An older and unrelated usage also applies the name to exact 32-bit compression of selected IEEE-754 double values via table lookup [1504.02914].

## 1. Terminological scope and historical usage

In current low-precision deep-learning research, HiF4 is primarily associated with **4-bit floating-point or block-floating computation** for LLM inference, post-training quantization, and FP4 training on specialized accelerators [2602.11287][2604.08826]. The central motivation is to combine the storage and throughput advantages of 4-bit arithmetic with stronger tolerance to activation and weight outliers than uniform integer quantization.

The nomenclature is, however, not singular. Luo et al. define HiF4 as a 64-element block with 32 bits of hierarchical shared scaling metadata, averaging **4.5 bits per value** [2602.11287]. An Ascend evaluation paper describes HiF4 as a three-level hierarchy with **block-level scale $S_1$ in unsigned E6M2, sub-block scale $S_2$ in E1, micro-block scale $S_3$ in E1, and element quantization in 3-bit two’s-complement**, again yielding **4.5 bits** per element [2602.12635]. Wan2.2 challenge reports sometimes treat HiFloat4 as a fixed toolkit-defined mini-float and sometimes present an explicit block-exponent encoding for W4A4 inference [2605.26628][2606.29337].

A further source of ambiguity is historical. In a separate, non-neural-compression context, HiFloat4 is used for a **32-bit code consisting of the sign, exponent, and high-order mantissa bits of a double**, with the missing low 32 mantissa bits restored by table lookup so that selected decimal-oriented subsets decode **exactly** to 64-bit IEEE values [1504.02914]. This suggests that “HiFloat4” functions less as a single immutable standard than as a reused label for compact numerical encodings.

## 2. Numerical organizations in the modern FP4 literature

The most detailed modern specification appears in the language-model inference paper by Luo et al. A HiF4 block contains **64 values**, each stored in **4 bits**, plus **32 bits of shared metadata**: **8 bits** for a global base scale in **E6M2**, **8 bits** for eight **1-bit** level-2 micro-exponents, and **16 bits** for sixteen **1-bit** level-3 micro-exponents, for a total of **288 bits per block** and **4.5 bits/value** on average [2602.11287]. In that formulation, each element uses **sign-magnitude S1P2**, equivalently an FP-style **E1M2** payload, and reconstruction is

$$
V_i = E6M2 \times 2^{\,E1\_8[\lceil i/8\rceil] + E1\_16[\lceil i/4\rceil]} \times m_i.
$$

SharQ’s HiF4 summary adopts the same 64-element organization and makes the element set explicit as **$\{\pm 0.00, \pm 0.25, \pm 0.50, \pm 0.75, \pm 1.00, \pm 1.25, \pm 1.50, \pm 1.75\}$**, with **E6M2** as the global scale and two binary micro-exponent levels [2606.26587]. The Ascend pre-training paper is closely aligned, describing a global **E6M2** scale with **bias = 48**, plus **E1\_8** and **E1\_16**, and an S1P2 payload equivalent to **E1M2** [2604.08826].

By contrast, the Ascend evaluation paper presents a different low-level realization: **UE6M2/E1/E1 + INT3**, with sign implicit in the **INT3 two’s-complement** code and dequantization written as

$$
X_{h_i} = \mathrm{sign}(x_i)\cdot S_1 \cdot S_2 \cdot S_3 \cdot (q_i / 2^3),
$$

where $q_i \in \{-4,\dots,+3\}$ [2602.12635]. This is numerically similar in spirit—hierarchical shared scaling plus tiny element payloads—but not identical in payload semantics to the S1P2/E1M2 formulation.

Wan2.2 challenge work introduces yet another representation layer. One report treats HiF4 purely as a challenge-specified **black-box quantize/dequantize operator** [2605.26628]. Another presents a block-exponent variant in which each value carries a **4-bit payload** with **1 sign bit** and **3-bit unsigned significand**, together with one **4-bit exponent $E_b$ per block**, using

$$
\hat x = (-1)^s \times \bigl(m/2^3\bigr)\times 2^{E_b-B},
$$

with **bias $B=7$** and blocks reaching **up to $32 \times 32$ weights** [2606.29337].

| Source line | Organization | Distinguishing feature |
|---|---|---|
| [2602.11287] | 64 elements + 32 metadata bits | E6M2 + E1\_8 + E1\_16 + S1P2 |
| [2602.12635] | 64-element hierarchy | UE6M2/E1/E1 + INT3 |
| [2606.29337] | Per-value 4-bit payload + shared block exponent | Simplified block-exponent HiF4 for Wan2.2 W4A4 |

## 3. Quantization, dequantization, and range management

In Luo et al., quantization of a 64-value block proceeds by **three-level peak reduction**: first compute **16 local maxima** over consecutive 4-element groups, reduce them to **8 maxima** over 2×2 groups, and then to a global peak $V_{\max}$ [2602.11287]. The peak is normalized to HiF4’s representable upper bound **7.0**, quantized into **E6M2**, and its reciprocal is used to derive the level-2 and level-3 binary exponents. The level-2 bit for an 8-element subgroup is set when the subgroup peak times the reciprocal is **$\ge 4$**; the level-3 bit for a 4-element group is set when the level-2-adjusted peak is **$\ge 2$**. Each element is then rounded to **S1P2**, with clamping if necessary [2602.11287].

This max-based hierarchical-scale selection reappears in SharQ’s HiF4 extension. There, a block maximum $v_{\max}$ produces an ideal scale factor **$\mathit{SF}=v_{\max}/7$**, which is rounded to the nearest representable **E6M2** value; binary micro-exponents are then chosen at the **4** and **2** thresholds, and quantization is explicitly **symmetric** and **zero-point-free** [2606.26587]. SharQ applies this both to the sparse backbone and the dense residual in its sparse–dense decomposition.

For post-training quantization, HiF4 is commonly paired with **range-balancing transforms**. In the Wan2.2 challenge summary, per-channel **SmoothQuant-style** folding computes

$$
s_j = \mathit{act\_scale}_j^{\alpha}\,\mathit{weight\_scale}_j^{1-\alpha},
$$

with **$\alpha \approx 0.5\!-\!0.7$ in early FFNs**, and rewrites the linear map as

$$
Y = XW = (X \oslash s)(s \odot W),
$$

so that activation tails are reduced before 4-bit quantization [2606.29337]. The same report then ranks channels by $\max_t|X_{t,j}|$ and places the top-$k$ in a **MixQ-style outlier branch** kept in **FP16 (or BF16)**, while the bulk remains strict W4A4 [2606.29337].

A related Wan2.2 report uses **activation-tail-aware percentile calibration** rather than worst-case maxima. For each channel it replaces the activation range estimate by the empirical **$p$-th percentile**,

$$
a_i^{(p)} = Q_p(\{|x_{j,i}|\}_j),
$$

and constructs a balancing mask

$$
m_i = \frac{w_i^{\,\alpha}}{(a_i+\epsilon)^{\,1-\alpha}},
$$

so that rare calibration outliers do not dominate the full 4-bit dynamic range [2605.26628].

## 4. Arithmetic structure and hardware mapping

A defining property of modern HiF4 is that its hierarchical metadata permits matrix products to be executed in a highly fixed-point manner. For two 64-element HiF4 blocks, the dot product in Luo et al. factors into a product of the two global **E6M2** scales and an integer-weighted sum over element mantissas and micro-exponent shifts [2602.11287]. In the hardware pipeline described there, level-3 shifts are absorbed into the mantissas, subgroup partial sums are shifted by the combined level-2 exponents, and only the final accumulated integer is multiplied by the product of the two global floating scales. Compared to **NVFP4**, this requires **only one floating-point multiplication (E6M2×E6M2) and one final large int × int**, whereas NVFP4 requires **four small FP multiplies**, **four large int multiplies**, and FP accumulation; for equivalent 64-length dot cores, HiF4 adds **only $\sim 1/3$ of the extra area of NVFP4** and saves **$\sim 10\%$ dynamic power** [2602.11287].

On **Ascend NPUs**, HiF4 is explicitly co-designed with the datapath. The pre-training paper states that each Cube-Unit PE can load **256 bits at once—exactly 64 FP4 values—and perform a fused dot-product + accumulation in one cycle**, so a HiF4 block maps **one-to-one** to the PE datapath [2604.08826]. The Ascend evaluation further notes that specialized instructions implement the three-level scale as **integer additions of exponents plus bit shifts**, avoiding floating-point multiplies in the inner loop and yielding a **$\sim 1.8$–$2\times$ throughput boost vs. 8-bit modes**; **W4A4 HiF4 end-to-end latency** is reported as **~60% of INT8 end-to-end** on **8×Ascend 910** [2602.12635].

In training, stabilization requirements differ by format. The Ascend pre-training study finds that HiF4 can maintain relative loss within **$\lesssim 1\%$** of BF16 using **Random Hadamard Transform (RHT)** on **$dW$** and **nearest rounding** elsewhere, whereas **MXFP4** typically requires **stochastic rounding**, **RHT**, and **truncation-free scaling** in combination [2604.08826]. On **Blackwell GPUs**, SharQ adds a different systems layer: a fused preparation kernel that combines normalization, **2:4** mask generation, sparse extraction, dense residual construction, and HiF4 quantization before launching a sparse HiF4 GEMM and a dense HiF4 GEMM [2606.26587].

## 5. Behavior in language-model inference and pre-training

Across LLM inference benchmarks, HiF4 is consistently presented as more robust than uniform 4-bit quantization and competitive with other low-bit floating formats. In Luo et al., for **small models (7–14 B)** over **8 tasks**, the average accuracy drop versus BF16 is **–1.81%** for **NVFP4 (direct cast)**, **–1.68%** for **NVFP4 + PTS**, **–1.07%** for **HiF4 (direct cast)**, and **–0.74%** for **HiF4 + HiGPTQ** [2602.11287]. For **large models (560–671 B)** over **10 tasks**, **HiF4 (direct)** is reported as **+0.48%** on **LongCat-560 B** and **+0.98%** on **DeepSeek-V3.1-671 B** relative to BF16, while **NVFP4 (direct)** crashes on **Mistral** and **LongCat** [2602.11287]. The paper’s stated interpretation is that HiF4 outperforms NVFP4 in all models and is more robust to outliers.

The Ascend evaluation reaches a similar conclusion from end-to-end W4A4 inference. On **Qwen3-8B (2 K context, zero-shot)**, HiF4 reports **PPL = 10.30 / 16.55** and **acc = 96.5% (Δ3.5%)**, while **SmoothQuant or SVDQuant** improves the HiF4 gap to **2.0–2.2%** versus BF16 [2602.12635]. On **openPangu-7B**, HiF4 gives **Δ = 3.0% (RTN)** and **2.7% (SmoothQuant/SVDQuant)**, and in KV-cache and attention quantization it remains usable where **MXFP4** collapses on openPangu-7B with **Δ>60%** [2602.12635].

In pre-training, HiF4 is used at much larger scale. On **OpenPangu-1B**, **Llama3-8B**, and **Qwen3-MoE-30B**, the Ascend study reports relative loss errors versus BF16 of **1.19%**, **0.85%**, and **0.88%** for HiF4, compared with **1.79%**, **1.44%**, and **1.55%** for MXFP4 [2604.08826]. The same work states that running all GEMMs in 4 bits yields **up to 4× higher matrix-throughput** and a **2.5× reduction in memory bandwidth** compared to BF16, and that inactive experts in the MoE model stored in FP4 reduce footprint by **≈60%** [2604.08826].

SharQ shows that HiF4 also supports structured activation sparsity. On **Qwen3-30B-A3B**, **HiF4 + RTN** gives **76.38** on HellaSwag, **63.28** on Lambada, **70.17** on WinoGrande, **87.61** on BoolQ, and **9.08** on WikiText2, while **SharQ + HiF4** improves these to **77.22**, **64.08**, **70.24**, **88.07**, and **8.92**, respectively [2606.26587]. SharQ’s broader results state that the method recovers **43–63% of the NVFP4-to-FP16 accuracy gap** across formats and delivers **2.2–2.4$\times$ latency reduction over FP16** and **1.2–1.4$\times$ throughput improvement over FP8** in language-model serving [2606.26587].

| Setting | Reported HiF4 outcome | Source |
|---|---|---|
| Small-model LLM inference | Direct cast: –1.07% avg. accuracy drop; HiGPTQ: –0.74% | [2602.11287] |
| Large-model LLM inference | Direct cast: +0.48% on LongCat, +0.98% on DeepSeek | [2602.11287] |
| Ascend pre-training | Relative loss error 1.19%, 0.85%, 0.88% on OpenPangu-1B, Llama3-8B, Qwen3-MoE-30B | [2604.08826] |
| Ascend W4A4 inference | Qwen3-8B acc 96.5%; SmoothQuant/SVDQuant gap improves to 2.0–2.2% | [2602.12635] |
| SharQ + HiF4 | HellaSwag 77.22; WikiText2 8.92 | [2606.26587] |

## 6. Diffusion transformers, Wan2.2, and video-generation pipelines

HiF4 has also been adopted in diffusion-transformer quantization. In the Wan2.2 post-training report **“Tail-Aware HiFloat4”**, the target system is **Wan2.2-I2V-A14B** with **two diffusion-transformer modules**; in each transformer, **all attention and feed-forward linear projections (400 layers per transformer)** are quantized to **W4A4 HiFloat4**, while **boundary layers** remain in **BF16** [2605.26628]. Calibration is run on **16 prompts**, activation percentiles are used to form the mask-only PTQ state, and the saved state stores **only masks $\{m_i\}$ and HiF4’s metadata**, not an entire 4-bit weight copy [2605.26628]. The reported official video metrics compare **BF16** with **HiF4 W4A4** as follows: **Imaging quality 0.7027 → 0.6507**, **Aesthetic quality 0.5456 → 0.5458**, **Overall consistency 0.2263 → 0.2308**, **Subject consistency 0.9331 → 0.5324**, **Motion smoothness 0.9923 → 0.9803**, and **Unweighted mean 0.6800 → 0.5880** [2605.26628]. The largest drop is therefore in **subject consistency**.

A separate Wan2.2 challenge submission combines three ingredients: **SmoothQuant-style per-channel smoothing**, **MixQ-style mixed precision for sparse activation outliers**, and **block-wise HiF4 packing** for feed-forward linear layers [2606.29337]. After smoothing, each FFN weight matrix is packed into **32×32 tiles** or **128×1 stripes**, and a **dual-branch GEMM** preserves a tiny outlier set in **FP16 (or BF16)** while the bulk of channels use strict W4A4 [2606.29337]. On official **VBench I2V** metrics, the paper reports **FP16 baseline** scores of **0.5445**, **0.9626**, **0.7086**, **0.9730**, and **0.9199** for **Aesthetic**, **I2V subject**, **Imaging**, **Motion**, and **Subject consistency**, respectively; **Native HiF4 W4A4** drops by about **5.0%** on all five axes, whereas **Ours (W4A4)** yields **0.5274 (−3.1%)**, **0.9375 (−2.6%)**, **0.6936 (−2.1%)**, **0.9769 (+0.4%)**, and **0.8875 (−3.5%)** [2606.29337]. The paper’s summary is that the pipeline stays within **2–3.5 percent** of FP16 on most quality axes and improves **motion smoothness**.

A broader diffusion-transformer connection appears in **HQ-DiT**, which describes a **HiFloat4-style 4-bit floating-point quantization** rather than the 64-element hierarchical HiF4 specification. There, FP4 uses **1 sign bit** and a tensor-dependent exponent/mantissa split, activations are fixed to **E2M1**, and a **random orthogonal Hadamard matrix** is used as a universal identity transform for outlier mitigation [2405.19751]. The reported result is that HQ-DiT is the **first instance where both weights and activations in DiTs are quantized to just 4 bits**, with **only a 0.12 increase in sFID on ImageNet** [2405.19751]. This suggests a broader FP4 design ecosystem in which HiF4-style ideas circulate beyond a single binary encoding.

## 7. Limitations, misconceptions, and comparative position

A common misconception is that HiF4 refers to one universally fixed 4-bit format. The literature does not support that reading. Some papers specify **E6M2 + E1\_8 + E1\_16 + S1P2**, others describe **UE6M2/E1/E1 + INT3**, and some challenge reports intentionally abstract HiFloat4 as a toolkit-defined operator [2602.11287][2602.12635][2605.26628]. A plausible implication is that HiF4 should be read as a **format family** or **implementation family**, not merely as one canonical bit layout.

The principal technical trade-off is **range versus granularity**. The 64-element grouping and hierarchical metadata deliver wide effective dynamic range, but they also make local scaling coarser than smaller-block alternatives. SharQ states this directly: HiF4’s **64-element group gives very wide dynamic range vs. a smaller local scale granularity**, and in practice one sees **slightly larger residuals than NVFP4 (group-32) on some layers** [2606.26587]. The same paper notes that because **E6M2 supports only normal modes**, very small peaks **(< $2^{-48}$)** underflow to zero, although this is described as rare in LLM activations [2606.26587].

Another recurrent issue is **outlier sensitivity**. Modern HiF4 pipelines nearly always add explicit countermeasures—**SmoothQuant folding**, **percentile clipping**, **MixQ-style higher-precision branches**, **Random Hadamard Transform**, or **SharQ’s sparse–dense decomposition**—rather than relying on naive round-to-nearest alone [2606.29337][2605.26628][2604.08826][2606.26587]. This does not imply that the format is ineffective; rather, it indicates that HiF4 is typically deployed as part of a larger quantization system.

Relative to adjacent formats, the papers position HiF4 as a middle ground between extremely cheap but fragile integer quantization and more overhead-heavy low-bit floating formats. The Ascend evaluation states that **INT4 collapses**, that **hierarchical formats (NVFP4, HiF4) are essential** in the 4-bit regime, and that HiF4’s **64/8/4 element hierarchy balances dynamic range and local precision**, maintaining **≥96.5% accuracy on Qwen3-8B** and **≥97.0% on openPangu-7B** in W4A4 inference [2602.12635]. Luo et al. go further and report that HiF4 achieves **higher average accuracy than the state-of-the-art NVFP4 format** across multiple models and tasks [2602.11287]. The overall comparative picture is therefore favorable, but strongly conditioned on calibration, outlier handling, and hardware support.

Source: https://www.emergentmind.com/topics/hifloat4-hif4