---
title: 'InfiR2: Comprehensive FP8 Training Recipe'
url: https://www.emergentmind.com/topics/infir2
type: topic
---

# InfiR2: Comprehensive FP8 Training Recipe

Searching arXiv for InfiR2 and related FP8 training literature.
InfiR2 is a training methodology for reasoning-enhanced language models that presents an end-to-end FP8 recipe spanning continual pre-training and supervised fine-tuning. It is described as “A Comprehensive FP8 Training Recipe for Reasoning-Enhanced Language Models” and is positioned around the claim that FP8 training had lacked a comprehensive, open-source recipe for full large-language-model pipelines before this work [2509.22536]. The method centers on a hybrid-granularity quantization strategy, uses IEEE-standardized FP8 formats E4M3 and E5M2, retains FP32 master weights and optimizer states, and reports performance parity with a BF16 baseline together with reductions in training time and peak memory usage and an increase in throughput on reasoning-oriented evaluations [2509.22536].

## 1. Definition and scope

InfiR2 denotes an FP8 training recipe rather than a new transformer architecture. Its stated objective is to make FP8 usable across the full training lifecycle of reasoning-enhanced language models, specifically combining continual pre-training and supervised fine-tuning within one numerically stable pipeline [2509.22536]. In the paper’s framing, the central problem is that the computational and memory demands of large language model training remain a major barrier, while the practical adoption of FP8 has been slowed by the absence of a stable, open, end-to-end procedure [2509.22536].

The work is explicitly motivated by hardware support for FP8 in NVIDIA Hopper and Blackwell architectures and by the theoretical efficiency advantages of moving from BF16 to 8-bit floating-point representations [2509.22536]. A plausible implication is that InfiR2 should be understood as a systems-and-numerics contribution aimed at reproducible large-scale training rather than as a model-family proposal in the usual sense.

## 2. Numerical design and quantization scheme

The technical core of InfiR2 is a fine-grained, hybrid-granularity quantization strategy intended to preserve numerical fidelity while improving efficiency [2509.22536]. The method distinguishes between weights and activations.

For weights, InfiR2 uses block-wise quantization. The description states that block-wise quantization partitions weight tensors into blocks, with “e.g., 16x16” given as an example, and computes a scaling factor per block [2509.22536]. For activations, it uses token-wise quantization, assigning each token activation its own scaling factor to better accommodate dynamic range and activation outliers in transformer workloads [2509.22536].

The paper specifies two IEEE-standardized FP8 formats. E4M3 is used for weights, characterized in the summary as having higher precision and narrower dynamic range, while E5M2 is used for activations, characterized as having wider dynamic range [2509.22536]. The scale computation is given as

\[
S = \frac{\max(|X|)}{V_{\text{max}}}
\]

and quantization of an element \(x\) is written as

\[
Q(x) = \text{round}\left(\frac{x}{S}\right).
\]

The scaling factor is further handled using UE8M0-style power-of-2 rounding, described as rounding up to the nearest power of 2 for representability and stability [2509.22536]. The summary emphasizes that this design choice is important for performance, especially in larger models and advanced supervised fine-tuning stages [2509.22536].

A further stabilizing choice is that master weights, gradients, and optimizer states remain in FP32 even though forward and backward flows operate in FP8 [2509.22536]. This is presented as a safeguard against catastrophic error accumulation and the loss of small updates during optimization.

## 3. Integration into the training pipeline

InfiR2 is presented as fully compatible with standard large-language-model training workflows and is applied in both continual pre-training and supervised fine-tuning [2509.22536]. The continual pre-training setup continues from Qwen2.5-1.5B-base and Qwen2.5-7B-base checkpoints for 160B tokens [2509.22536]. The data mixture is described in two stages: 140B tokens from public sources, specifically FineWeb, Nemotron, Stack-Edu, and Kaggle Notebooks, followed by 20B tokens incorporating high-quality, reasoning-oriented datasets AM-DeepSeek-R1 and AM-Qwen3 [2509.22536].

The reported continual pre-training configuration includes batch size 128, a learning-rate schedule from \(1 \times 10^{-4}\) to \(1 \times 10^{-5}\), and context length up to 32k tokens [2509.22536]. For supervised fine-tuning, the paper uses a two-stage SFT process with InfiAlign-SFT-92k and InfiAlign-SFT-165k, running 5 epochs in each stage, with batch size 64 and learning rate \(5 \times 10^{-5}\) [2509.22536]. Experiments are reported for both 1.5B and 7B model scales, and the FP8 pipeline is applied in both the pre-training and SFT stages [2509.22536].

The following table summarizes the training setup stated in the paper.

| Component | Reported configuration |
|---|---|
| Base models | Qwen2.5-1.5B-base, Qwen2.5-7B-base |
| Continual pre-training corpus | 160B tokens |
| Public-data portion | 140B tokens from FineWeb, Nemotron, Stack-Edu, Kaggle Notebooks |
| Reasoning-oriented portion | 20B tokens from AM-DeepSeek-R1, AM-Qwen3 |
| Pre-training batch size | 128 |
| Pre-training learning rate | \(1 \times 10^{-4}\) to \(1 \times 10^{-5}\) |
| Context length | Up to 32k tokens |
| SFT datasets | InfiAlign-SFT-92k, InfiAlign-SFT-165k |
| SFT epochs | 5 in each stage |
| SFT batch size | 64 |
| SFT learning rate | \(5 \times 10^{-5}\) |

This integration across both continual pre-training and SFT is one of the main reasons the work is described as an end-to-end recipe rather than a narrowly scoped low-precision kernel or isolated systems optimization [2509.22536].

## 4. Empirical results

The paper reports that FP8 training is “remarkably stable” and “essentially lossless,” with performance on par with the BF16 baseline across a suite of reasoning benchmarks [2509.22536]. The evaluation benchmarks explicitly listed are AIME24, AIME25, GPQA, and LiveCodeBench v5 [2509.22536]. The summary states that models are evaluated in the same precision in which they were trained, described as an “apples-to-apples” comparison between FP8 and BF16 [2509.22536].

For convergence behavior, the paper states that the loss curves for FP8 and BF16 are “virtually identical” during 160B-token pre-training [2509.22536]. It also reports example benchmark values. InfiR2-7B-FP8 achieves 55.73 on AIME24, which is stated as \(+12.71\) over Qwen2.5-7B-base with the same SFT [2509.22536]. For 1.5B models, FP8 is reported to sometimes outperform BF16 after SFT; one example given is GPQA, with 27.78 for FP8 versus 24.48 for BF16 [2509.22536]. An average performance difference of within 1–2 points between FP8 and BF16 is also reported [2509.22536].

The summary includes one comparison table with the following BF16 and FP8 benchmark values:

| Precision | AIME25 | AIME24 | GPQA | LiveCodeBench v5 |
|---|---:|---:|---:|---:|
| BF16 | 17.91 | 17.50 | 31.94 | 16.41 |
| FP8 | 18.45 | 17.39 | 29.48 | 17.10 |

The efficiency claims are equally central. InfiR2 reports up to a 22% reduction in training time, up to a 14% decrease in peak memory usage, and up to a 19% increase in throughput [2509.22536]. Specific examples given in the summary include a 1.5B model at 8k context length, where total time changes from 2030 ms to 1590 ms, peak memory from 68.1 GB to 58.3 GB, and throughput from 340 TFlops to 376 TFlops; and a 7B model at 32k context length, where total time changes from 9590 ms to 8360 ms, peak memory from 78.1 GB to 67.4 GB, and throughput from 409 TFlops to 461 TFlops [2509.22536].

These numbers support the paper’s central claim that FP8 can function as a practical alternative to BF16 for reasoning-oriented large-scale training when the quantization and scaling design are chosen carefully [2509.22536].

## 5. Relationship to adjacent FP8 and efficient-training work

InfiR2 situates itself in a broader line of work that had already shown the feasibility of FP8 “in pieces,” with DeepGEMM, DeepSeek-V3, and COAT named as relevant antecedents in the summary [2509.22536]. The distinction drawn by InfiR2 is that these earlier efforts did not provide what the paper calls a comprehensive, open-source, stable FP8 recipe for the full pipeline of continual pre-training plus supervised fine-tuning [2509.22536].

Within that framing, InfiR2 can be viewed as a recipe-level synthesis: it combines low-precision representation choices, hybrid-granularity quantization, scale management via UE8M0-style power-of-2 rounding, and end-to-end training validation on reasoning-oriented tasks [2509.22536]. This suggests a division of labor in the FP8 literature between component-level optimizations and full-pipeline operationalization. InfiR2’s contribution falls primarily into the latter category.

A common misconception is to equate low-precision training with aggressive state compression across the entire optimization process. InfiR2 does not do that. The summary explicitly states that master weights, gradients, and optimizer states are stored in FP32, even though FP8 is used in forward and backward flows [2509.22536]. Another common misconception is that FP8 recipes are only credible for short runs or partial fine-tuning; the paper’s 160B-token continual pre-training plus two-stage SFT setup is presented precisely to argue against that assumption [2509.22536].

## 6. Release, intended use, and broader significance

The paper states that the authors will release the code and key artifacts, including logs, intermediate checkpoints, and the InfiR2-7B-FP8 model [2509.22536]. It further characterizes the effort as the first comprehensive, open-source FP8 pipeline for full LLM training [2509.22536]. Because this claim appears in the provided material, it is best read as the paper’s self-positioning within the open FP8 training ecosystem.

The broader significance assigned to InfiR2 is accessibility and cost reduction. The summary states that the reduced compute and memory cost lowers the entry barrier for academics, smaller labs, startups, and resource-constrained researchers, while also aligning with sustainability goals through reduced hardware and energy usage [2509.22536]. It further argues that FP8 is demonstrated as a “drop-in” replacement for BF16 in real-world, reasoning-heavy applications rather than in toy settings or partial workflows [2509.22536].

A plausible implication is that InfiR2 is relevant not only for benchmark-driven model development but also for experimental methodology. If the reported equivalence in convergence and reasoning performance generalizes, then FP8 may change the practical envelope for ablation studies, long-context experiments, and multi-stage post-training by reducing the marginal cost of each training run. The paper’s own evidence for that claim is limited to the reported reductions in time and memory and the benchmark parity it documents [2509.22536].

## 7. Limitations and interpretive cautions

The paper’s strongest claims are empirical rather than universal. It demonstrates stability and near-parity with BF16 under the reported settings, on the stated model sizes, datasets, and benchmarks [2509.22536]. It does not establish, at least in the provided summary, that every architecture, optimizer configuration, or data regime will behave identically under the same FP8 recipe.

Another point requiring precision is the use of the term “lossless.” In the paper summary, “lossless” is explicitly qualified as meaning negligible difference in convergence, generalization, and reasoning benchmarks, not literal bitwise equivalence to BF16 training [2509.22536]. Likewise, the reported efficiency gains are given as “up to” values and therefore should not be interpreted as guaranteed across all contexts [2509.22536].

Finally, the work concerns training methodology, not inference optimization. This distinction matters because neighboring research on efficient LLM deployment often targets KV-cache management, offloading, or near-storage execution rather than training numerics; INF\(^2\), for example, addresses generative inference throughput via computational storage devices and attention-near storage rather than FP8 training [2502.09921]. The similarity in naming can obscure the fact that InfiR2 and INF\(^2\) operate at different stages of the model lifecycle and solve different systems problems.

Taken together, InfiR2 is best understood as a recipe for making FP8 training operational at scale for reasoning-enhanced language models: block-wise quantization for weights, token-wise quantization for activations, E4M3 and E5M2 format specialization, UE8M0-style power-of-2 scale handling, FP32 optimizer-state retention, and end-to-end validation through continual pre-training and two-stage SFT [2509.22536]. Its main significance lies in turning low-precision training from an isolated hardware capability into a documented workflow with concrete evidence for stability, efficiency, and reasoning-task competitiveness.

Source: https://www.emergentmind.com/topics/infir2