InfiR2: Comprehensive FP8 Training Recipe

• InfiR2 is an FP8 training methodology that integrates continual pre-training and supervised fine-tuning for reasoning-enhanced language models.
• It employs a hybrid quantization strategy with block-wise weight and token-wise activation schemes using IEEE-standard E4M3 and E5M2 formats.
• Empirical results demonstrate performance parity with BF16 along with reduced training time, lower memory usage, and increased throughput on reasoning tasks.

Searching arXiv for InfiR2 and related FP8 training literature. InfiR2 is a training methodology for reasoning-enhanced LLMs that presents an end-to-end FP8 recipe spanning continual pre-training and supervised fine-tuning. It is described as “A Comprehensive FP8 Training Recipe for Reasoning-Enhanced LLMs” and is positioned around the claim that FP8 training had lacked a comprehensive, open-source recipe for full large-language-model pipelines before this work (Wang et al., 26 Sep 2025). The method centers on a hybrid-granularity quantization strategy, uses IEEE-standardized FP8 formats E4M3 and E5M2, retains FP32 master weights and optimizer states, and reports performance parity with a BF16 baseline together with reductions in training time and peak memory usage and an increase in throughput on reasoning-oriented evaluations (Wang et al., 26 Sep 2025).

1. Definition and scope

InfiR2 denotes an FP8 training recipe rather than a new transformer architecture. Its stated objective is to make FP8 usable across the full training lifecycle of reasoning-enhanced LLMs, specifically combining continual pre-training and supervised fine-tuning within one numerically stable pipeline (Wang et al., 26 Sep 2025). In the paper’s framing, the central problem is that the computational and memory demands of LLM training remain a major barrier, while the practical adoption of FP8 has been slowed by the absence of a stable, open, end-to-end procedure (Wang et al., 26 Sep 2025).

The work is explicitly motivated by hardware support for FP8 in NVIDIA Hopper and Blackwell architectures and by the theoretical efficiency advantages of moving from BF16 to 8-bit floating-point representations (Wang et al., 26 Sep 2025). A plausible implication is that InfiR2 should be understood as a systems-and-numerics contribution aimed at reproducible large-scale training rather than as a model-family proposal in the usual sense.

2. Numerical design and quantization scheme

The technical core of InfiR2 is a fine-grained, hybrid-granularity quantization strategy intended to preserve numerical fidelity while improving efficiency (Wang et al., 26 Sep 2025). The method distinguishes between weights and activations.

For weights, InfiR2 uses block-wise quantization. The description states that block-wise quantization partitions weight tensors into blocks, with “e.g., 16x16” given as an example, and computes a scaling factor per block (Wang et al., 26 Sep 2025). For activations, it uses token-wise quantization, assigning each token activation its own scaling factor to better accommodate dynamic range and activation outliers in transformer workloads (Wang et al., 26 Sep 2025).

The paper specifies two IEEE-standardized FP8 formats. E4M3 is used for weights, characterized in the summary as having higher precision and narrower dynamic range, while E5M2 is used for activations, characterized as having wider dynamic range (Wang et al., 26 Sep 2025). The scale computation is given as

$S = \frac{\max(|X|)}{V_{\text{max}}}$

and quantization of an element $x$ is written as

$Q(x) = \text{round}\left(\frac{x}{S}\right).$

The scaling factor is further handled using UE8M0-style power-of-2 rounding, described as rounding up to the nearest power of 2 for representability and stability (Wang et al., 26 Sep 2025). The summary emphasizes that this design choice is important for performance, especially in larger models and advanced supervised fine-tuning stages (Wang et al., 26 Sep 2025).

A further stabilizing choice is that master weights, gradients, and optimizer states remain in FP32 even though forward and backward flows operate in FP8 (Wang et al., 26 Sep 2025). This is presented as a safeguard against catastrophic error accumulation and the loss of small updates during optimization.

3. Integration into the training pipeline

InfiR2 is presented as fully compatible with standard large-language-model training workflows and is applied in both continual pre-training and supervised fine-tuning (Wang et al., 26 Sep 2025). The continual pre-training setup continues from Qwen2.5-1.5B-base and Qwen2.5-7B-base checkpoints for 160B tokens (Wang et al., 26 Sep 2025). The data mixture is described in two stages: 140B tokens from public sources, specifically FineWeb, Nemotron, Stack-Edu, and Kaggle Notebooks, followed by 20B tokens incorporating high-quality, reasoning-oriented datasets AM-DeepSeek-R1 and AM-Qwen3 (Wang et al., 26 Sep 2025).

The reported continual pre-training configuration includes batch size 128, a learning-rate schedule from $1 \times 10^{-4}$ to $1 \times 10^{-5}$, and context length up to 32k tokens (Wang et al., 26 Sep 2025). For supervised fine-tuning, the paper uses a two-stage SFT process with InfiAlign-SFT-92k and InfiAlign-SFT-165k, running 5 epochs in each stage, with batch size 64 and learning rate $5 \times 10^{-5}$ (Wang et al., 26 Sep 2025). Experiments are reported for both 1.5B and 7B model scales, and the FP8 pipeline is applied in both the pre-training and SFT stages (Wang et al., 26 Sep 2025).

The following table summarizes the training setup stated in the paper.

Component	Reported configuration
Base models	Qwen2.5-1.5B-base, Qwen2.5-7B-base
Continual pre-training corpus	160B tokens
Public-data portion	140B tokens from FineWeb, Nemotron, Stack-Edu, Kaggle Notebooks
Reasoning-oriented portion	20B tokens from AM-DeepSeek-R1, AM-Qwen3
Pre-training batch size	128
Pre-training learning rate	$1 \times 10^{-4}$ to $1 \times 10^{-5}$
Context length	Up to 32k tokens
SFT datasets	InfiAlign-SFT-92k, InfiAlign-SFT-165k
SFT epochs	5 in each stage
SFT batch size	64
SFT learning rate	$5 \times 10^{-5}$

This integration across both continual pre-training and SFT is one of the main reasons the work is described as an end-to-end recipe rather than a narrowly scoped low-precision kernel or isolated systems optimization (Wang et al., 26 Sep 2025).

4. Empirical results

The paper reports that FP8 training is “remarkably stable” and “essentially lossless,” with performance on par with the BF16 baseline across a suite of reasoning benchmarks (Wang et al., 26 Sep 2025). The evaluation benchmarks explicitly listed are AIME24, AIME25, GPQA, and LiveCodeBench v5 (Wang et al., 26 Sep 2025). The summary states that models are evaluated in the same precision in which they were trained, described as an “apples-to-apples” comparison between FP8 and BF16 (Wang et al., 26 Sep 2025).

For convergence behavior, the paper states that the loss curves for FP8 and BF16 are “virtually identical” during 160B-token pre-training (Wang et al., 26 Sep 2025). It also reports example benchmark values. InfiR2-7B-FP8 achieves 55.73 on AIME24, which is stated as $+12.71$ over Qwen2.5-7B-base with the same SFT (Wang et al., 26 Sep 2025). For 1.5B models, FP8 is reported to sometimes outperform BF16 after SFT; one example given is GPQA, with 27.78 for FP8 versus 24.48 for BF16 (Wang et al., 26 Sep 2025). An average performance difference of within 1–2 points between FP8 and BF16 is also reported (Wang et al., 26 Sep 2025).

The summary includes one comparison table with the following BF16 and FP8 benchmark values:

Precision	AIME25	AIME24	GPQA	LiveCodeBench v5
BF16	17.91	17.50	31.94	16.41
FP8	18.45	17.39	29.48	17.10

The efficiency claims are equally central. InfiR2 reports up to a 22% reduction in training time, up to a 14% decrease in peak memory usage, and up to a 19% increase in throughput (Wang et al., 26 Sep 2025). Specific examples given in the summary include a 1.5B model at 8k context length, where total time changes from 2030 ms to 1590 ms, peak memory from 68.1 GB to 58.3 GB, and throughput from 340 TFlops to 376 TFlops; and a 7B model at 32k context length, where total time changes from 9590 ms to 8360 ms, peak memory from 78.1 GB to 67.4 GB, and throughput from 409 TFlops to 461 TFlops (Wang et al., 26 Sep 2025).

These numbers support the paper’s central claim that FP8 can function as a practical alternative to BF16 for reasoning-oriented large-scale training when the quantization and scaling design are chosen carefully (Wang et al., 26 Sep 2025).

5. Relationship to adjacent FP8 and efficient-training work

InfiR2 situates itself in a broader line of work that had already shown the feasibility of FP8 “in pieces,” with DeepGEMM, DeepSeek-V3, and COAT named as relevant antecedents in the summary (Wang et al., 26 Sep 2025). The distinction drawn by InfiR2 is that these earlier efforts did not provide what the paper calls a comprehensive, open-source, stable FP8 recipe for the full pipeline of continual pre-training plus supervised fine-tuning (Wang et al., 26 Sep 2025).

Within that framing, InfiR2 can be viewed as a recipe-level synthesis: it combines low-precision representation choices, hybrid-granularity quantization, scale management via UE8M0-style power-of-2 rounding, and end-to-end training validation on reasoning-oriented tasks (Wang et al., 26 Sep 2025). This suggests a division of labor in the FP8 literature between component-level optimizations and full-pipeline operationalization. InfiR2’s contribution falls primarily into the latter category.

A common misconception is to equate low-precision training with aggressive state compression across the entire optimization process. InfiR2 does not do that. The summary explicitly states that master weights, gradients, and optimizer states are stored in FP32, even though FP8 is used in forward and backward flows (Wang et al., 26 Sep 2025). Another common misconception is that FP8 recipes are only credible for short runs or partial fine-tuning; the paper’s 160B-token continual pre-training plus two-stage SFT setup is presented precisely to argue against that assumption (Wang et al., 26 Sep 2025).

6. Release, intended use, and broader significance

The paper states that the authors will release the code and key artifacts, including logs, intermediate checkpoints, and the InfiR2-7B-FP8 model (Wang et al., 26 Sep 2025). It further characterizes the effort as the first comprehensive, open-source FP8 pipeline for full LLM training (Wang et al., 26 Sep 2025). Because this claim appears in the provided material, it is best read as the paper’s self-positioning within the open FP8 training ecosystem.

The broader significance assigned to InfiR2 is accessibility and cost reduction. The summary states that the reduced compute and memory cost lowers the entry barrier for academics, smaller labs, startups, and resource-constrained researchers, while also aligning with sustainability goals through reduced hardware and energy usage (Wang et al., 26 Sep 2025). It further argues that FP8 is demonstrated as a “drop-in” replacement for BF16 in real-world, reasoning-heavy applications rather than in toy settings or partial workflows (Wang et al., 26 Sep 2025).

A plausible implication is that InfiR2 is relevant not only for benchmark-driven model development but also for experimental methodology. If the reported equivalence in convergence and reasoning performance generalizes, then FP8 may change the practical envelope for ablation studies, long-context experiments, and multi-stage post-training by reducing the marginal cost of each training run. The paper’s own evidence for that claim is limited to the reported reductions in time and memory and the benchmark parity it documents (Wang et al., 26 Sep 2025).

7. Limitations and interpretive cautions

The paper’s strongest claims are empirical rather than universal. It demonstrates stability and near-parity with BF16 under the reported settings, on the stated model sizes, datasets, and benchmarks (Wang et al., 26 Sep 2025). It does not establish, at least in the provided summary, that every architecture, optimizer configuration, or data regime will behave identically under the same FP8 recipe.

Another point requiring precision is the use of the term “lossless.” In the paper summary, “lossless” is explicitly qualified as meaning negligible difference in convergence, generalization, and reasoning benchmarks, not literal bitwise equivalence to BF16 training (Wang et al., 26 Sep 2025). Likewise, the reported efficiency gains are given as “up to” values and therefore should not be interpreted as guaranteed across all contexts (Wang et al., 26 Sep 2025).

Finally, the work concerns training methodology, not inference optimization. This distinction matters because neighboring research on efficient LLM deployment often targets KV-cache management, offloading, or near-storage execution rather than training numerics; INF$x$0, for example, addresses generative inference throughput via computational storage devices and attention-near storage rather than FP8 training (Jang et al., 14 Feb 2025). The similarity in naming can obscure the fact that InfiR2 and INF$x$1 operate at different stages of the model lifecycle and solve different systems problems.

Taken together, InfiR2 is best understood as a recipe for making FP8 training operational at scale for reasoning-enhanced LLMs: block-wise quantization for weights, token-wise quantization for activations, E4M3 and E5M2 format specialization, UE8M0-style power-of-2 scale handling, FP32 optimizer-state retention, and end-to-end validation through continual pre-training and two-stage SFT (Wang et al., 26 Sep 2025). Its main significance lies in turning low-precision training from an isolated hardware capability into a documented workflow with concrete evidence for stability, efficiency, and reasoning-task competitiveness.