Ozaki Scheme in Matrix Multiplication

• The Ozaki scheme is a method that rewrites high-precision DGEMM into a sum of low-precision GEMM calls using structured slicing or CRT-based techniques.
• Ozaki-I decomposes matrices into significance slices for error-free low-precision multiplication, while Ozaki-II employs integer scaling and CRT for precise INT8 operations.
• Both formulations address the gap between fast low-precision accelerators and slow FP64 pipelines, making high-accuracy results feasible on modern hardware.

to=arxiv_search.search വിവരം 天天中彩票nbajson {"query":"Ozaki scheme matrix multiplication Ozaki Scheme II FP8 INT8 DGEMM", "max_results": 10} to=arxiv_search.search വിവരം եցjson {"query":"Ozaki close-to-convex functions Ozaki scheme geometric function theory", "max_results": 10} The Ozaki scheme is a family of matrix-multiplication emulation methods that rewrites a high-precision product, especially FP64 DGEMM, into a structured sum of lower-precision matrix multiplications executed on highly optimized GEMM or tensor-core hardware, followed by higher-precision reconstruction. In contemporary usage, the term covers two closely related lines: the classical slice-based method, now often called Ozaki-I, and the CRT-based modular method, Ozaki-II. Both are motivated by the widening architectural gap between very fast low-precision matrix engines and comparatively slow native FP64 pipelines, and both aim to recover FP64-level or higher accuracy from low-precision substrates such as FP16, FP8, or INT8 (Ozaki et al., 10 Apr 2025, Mukunoki, 1 Aug 2025, Uchino et al., 9 Dec 2025).

1. Historical setting and computational objective

The modern matrix-multiplication meaning of the Ozaki scheme is rooted in a long line of “high precision from lower precision” techniques, but its distinctive feature is that it delegates the expensive dense-multiply stage to existing optimized GEMM kernels rather than reimplementing multiplication entirely in software arithmetic. One paper describes the Ozaki scheme as “an accurate matrix multiplication method proposed by Ozaki et al. in 2012” that enables FP64 matrix multiplication using low-precision floating-point operations such as FP16, and later notes that the method was extended to utilize integer arithmetic (Mukunoki, 1 Aug 2025). Another paper formulates the same idea more generally as a GEMM-oriented emulation method: a high-accuracy product is reduced to multiple calls to highly optimized matrix-multiplication kernels, with the dominant part of the work still carried by GEMM (Ozaki et al., 10 Apr 2025).

This objective became substantially more important as AI-oriented processors shifted silicon budget toward low-precision throughput. Several recent studies explicitly frame the Ozaki scheme as a way to obtain DGEMM-quality results on hardware where FP64 arithmetic is weak but INT8, FP16, or FP8 tensor-core performance is abundant (Ootomo et al., 2023, Uchino et al., 2024, Mukunoki, 1 Aug 2025). A recurring theme in the recent literature is that the method is not primarily about beating vendor DGEMM on processors with strong FP64 units; rather, it is about making scientific-style FP64 matrix multiplication practical on accelerators whose peak performance is concentrated in low-precision matrix engines (Mukunoki, 1 Aug 2025, Schwarz et al., 16 Nov 2025).

2. Classical slice-based formulation

In its classical form, the Ozaki scheme decomposes each operand into slices that capture successively smaller significance blocks. One canonical formulation writes

$$A = A_1 + A_2 + \dots + A_{k-1} + \underline{A}_k,\qquad B = B_1 + B_2 + \dots + B_{k-1} + \underline{B}_k,$$

and reorganizes the product as

$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$

The key design requirement is that the dominant slice products satisfy

$\mathtt{fl}(A_iB_j)=A_iB_j,$

so that the main work is performed without rounding error inside the chosen low-precision compute path (Ozaki et al., 10 Apr 2025).

This slice-based viewpoint also appears in inner-product form. A recent FP8-oriented exposition describes the method as an error-free transformation for matrix multiplication: vectors are recursively sliced into scaled lower-precision pieces,

$$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$

so that

$$\bm{x}^{T}\bm{y} = \sum_{p=1}^{s_x} \sum_{q=1}^{s_y} 2^{c_x^{(p)}+c_y^{(q)}}\, \underline{\bm{x}^{(p)}}^{T}\underline{\bm{y}^{(q)}},$$

with each low-precision pairwise product arranged to be evaluable in the accumulation format without rounding error (Mukunoki, 1 Aug 2025). The same paper emphasizes the essential cost model: the number of low-precision GEMMs scales like the square of the slice count, so slice selection is the central accuracy–performance dial (Mukunoki, 1 Aug 2025).

For tensor-core realization, the classical scheme is often written in a diagonally scaled form,

$$\tilde C := \mathtt{fl}\left( \sum_{i+j\le k+1} D_i^{-1}\big((D_iA_i)(B_jE_j)\big)E_j^{-1}\right),$$

where powers-of-two diagonal scalings make each sliced operand representable in the low-precision input format. In this conventional Ozaki-I setting, a $k$-slice decomposition requires

$\frac{k(k+1)}{2}$

matrix multiplications, which is why reducing slice count is so consequential (Ozaki et al., 10 Apr 2025).

3. Ozaki-I and Ozaki-II

A major development in the recent literature is the explicit separation between the classical slice-based scheme and a newer CRT-based variant, Ozaki-II. The two methods solve the same broad problem—high-accuracy matrix multiplication via fast low-precision kernels—but they do so through different decompositions (Ozaki et al., 10 Apr 2025, Uchino et al., 9 Dec 2025).

Aspect	Ozaki-I	Ozaki-II
Decomposition	Floating-point slices $A_i,B_j$	Scaled integers and modular residues
Reconstruction	Sum of slice products	CRT / Garner reconstruction
Low-precision work count	$\frac{k(k+1)}{2}$ GEMMs	One GEMM per modulus
Natural substrate	FP16 / FP8 / INT8 tensor cores	Especially INT8 modular GEMM

Ozaki-II begins by scaling and truncating floating-point inputs into integer matrices,

$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$0

then reducing those integers modulo pairwise coprime moduli $$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$1, computing modular products

$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$2

and reconstructing the integer product through CRT. In one standard formulation,

$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$3

with uniqueness ensured by the sufficient condition

$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$4

The floating-point result is then recovered by inverse scaling (Uchino et al., 9 Dec 2025, Uchino et al., 30 Jan 2026).

The significance of Ozaki-II is twofold. First, it provides direct control over the number of low-precision matrix multiplications through the number of moduli $$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$5, rather than through a triangular slice count. Second, it aligns naturally with INT8 matrix engines, because centered residues for suitably chosen moduli fit directly into signed 8-bit inputs with INT32 accumulation (Ozaki et al., 10 Apr 2025, Uchino et al., 30 Jan 2026). This explains why several later papers treat Ozaki-I and Ozaki-II as distinct branches of a common emulation framework rather than minor variants of the same algorithm (Uchino et al., 9 Dec 2025, Uchino et al., 11 Mar 2026).

4. Hardware mappings, optimization, and automation

A large part of the recent Ozaki-scheme literature is concerned with mapping these decompositions efficiently onto real hardware. On NVIDIA consumer GPUs, one study of DGEMM on integer matrix multiplication units reported that double-precision matrix multiplication could be computed faster than cuBLAS and faster than an existing FP16-Tensor-Core Ozaki implementation, and it demonstrated acceleration of a quantum circuit simulation by up to $$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$6 while maintaining FP64 accuracy (Ootomo et al., 2023). Another INT8-focused paper introduced two specific optimizations—rounding-to-nearest splitting and group-wise error-free accumulation—and reported that the FP64 accumulation ratio fell from about $$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$7–$$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$8 in the original implementation to about $$AB = \sum_{i+j \le k} A_i B_j + \sum_{i=1}^{k-1} A_i \underline{B}_{k+1-i} + \underline{A}_k B.$$9–$\mathtt{fl}(A_iB_j)=A_iB_j,$0 in the improved variants (Uchino et al., 2024).

Recent work also shows that the optimal low-precision substrate is changing. A Blackwell-era study revisited the floating-point path and argued that FP8 and FP6 tensor cores are replacing INT8 as favored low-precision hardware. It reported that FP8 Tensor Core Ozaki outperforms FP16 Tensor Core Ozaki for sufficiently large matrices, specifically for $\mathtt{fl}(A_iB_j)=A_iB_j,$1, and that replacing all hardware FP64 instructions in slicing and accumulation with integer-based FP64 emulation did not degrade overall performance on the tested RTX 5060 Ti GPU (Mukunoki, 1 Aug 2025). In parallel, the CRT-based line has been extended to FP8: a 2026 paper on FP8 Ozaki-II reported that a hybrid square-modulus and Karatsuba construction reduced the FP8 matrix-multiplication count to $\mathtt{fl}(A_iB_j)=A_iB_j,$2 for $\mathtt{fl}(A_iB_j)=A_iB_j,$3 moduli, compared with $\mathtt{fl}(A_iB_j)=A_iB_j,$4 for the minimum accurate FP8 Ozaki-I configuration (Uchino et al., 11 Mar 2026).

Another strand of work addresses usability rather than raw kernel design. “Guaranteed DGEMM Accuracy While Using Reduced Precision Tensor Cores Through Extensions of the Ozaki Scheme” introduced Exponent Span Capacity (ESC) and Automatic Dynamic Precision (ADP), a fully GPU-resident framework that chooses slice counts conservatively, handles Inf/NaN inputs, and falls back to native FP64 when emulation is slower or unsafe. In the reported 55-bit mantissa setting, this framework incurred less than $\mathtt{fl}(A_iB_j)=A_iB_j,$5 runtime overhead and achieved up to $\mathtt{fl}(A_iB_j)=A_iB_j,$6 and $\mathtt{fl}(A_iB_j)=A_iB_j,$7 speedups over native FP64 GEMM on NVIDIA Blackwell GB200 and RTX Pro 6000 Blackwell Server Edition, respectively; the same work also reported up to $\mathtt{fl}(A_iB_j)=A_iB_j,$8 end-to-end speedup in QR factorization (Schwarz et al., 16 Nov 2025). For Ozaki-II fast mode, later work identified a lack of scale invariance in the existing scaling rule and proposed a revised formula derived from the CRT uniqueness condition via the Cauchy–Schwarz inequality, with experiments on NVIDIA GH200 showing accuracy comparable to accurate mode while throughput remained comparable to fast mode (Kawakami et al., 28 Jun 2026).

5. Generalizations and applications

Although dense real GEMM remains the canonical target, the Ozaki scheme has been generalized in several directions. For multiple-precision linear algebra on CPUs, “Acceleration of Multiple Precision Matrix Multiplication using Ozaki scheme” showed that DGEMM-backed Ozaki decomposition can accelerate DD, TD, and QD matrix multiplication and can outperform Strassen-based MPFR multiplication up to moderate precision ranges (Utsugiri et al., 2023). For arbitrary-precision complex arithmetic, another study embedded Ozaki-scheme real GEMMs inside 3M and 4M complex decompositions and reported that the 3M CGEMM method based on the Ozaki scheme with $\mathtt{fl}(A_iB_j)=A_iB_j,$9 was the fastest for 256-bit calculations, whereas Strassen multiplication was the fastest for 768-bit calculations (Kouya, 2023).

The CRT-based line has also been extended to complex arithmetic. “Emulation of Complex Matrix Multiplication based on the Chinese Remainder Theorem” formulated complex Ozaki-II through a Karatsuba-based reduction to three real matrix products and reported speedups of $$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$0–$$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$1 and $$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$2–$$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$3 over native single- and double-precision complex matrix multiplication routines on NVIDIA B200 for sufficiently large problem sizes (Uchino et al., 9 Dec 2025). A later FP8-focused extension adapted Ozaki-II to FP8 MMA units for DGEMM emulation and argued that this route is increasingly relevant on architectures where INT8 capability is reduced while FP8 remains strong (Uchino et al., 11 Mar 2026).

Beyond direct matrix multiplication, the scheme has been adapted to other numerical kernels. One eigenvalue-decomposition paper used an Ozaki-based accurate product $$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$4 inside a forward-error-oriented iterative refinement method for eigenvectors of a real symmetric matrix, and its central modification was to split only the matrix $$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$5 fully while truncating the approximate eigenvector matrix to its leading slice, thereby reducing multiplication cost while still targeting a prescribed forward error (Terao et al., 22 Feb 2026). At a more speculative systems level, “FP8 is All You Need (Part 2)” extended Ozaki Scheme II to 3-D FFT through a Bailey six-step decomposition and proposed a tensor-core Garner reformulation together with a Kulisch fixed-point “escape route,” motivated by the regime $$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$6 in which CRT reconstruction latency dominates rather than amortizes (Matsuoka, 28 May 2026).

6. Terminological ambiguity and non-numerical uses of “Ozaki”

The expression Ozaki scheme belongs to numerical linear algebra and high-performance computing, but the name Ozaki also appears prominently in geometric function theory with a completely different meaning. Multiple papers explicitly note that they are not about an “Ozaki scheme” in the numerical-analysis sense; instead, they study Ozaki close-to-convex functions, where “Ozaki” refers to a class of normalized analytic functions introduced by S. Ozaki in 1941 (Kumar et al., 2023). In that literature, the defining condition is

$$\bm{x} = \sum_{p=1}^{s_x} 2^{c_x^{(p)}} \underline{\bm{x}^{(p)}},\qquad \bm{y} = \sum_{q=1}^{s_y} 2^{c_y^{(q)}} \underline{\bm{y}^{(q)}},$$7

or its generalizations, and the main topics are Schwarzian and pre-Schwarzian norm estimates, distortion and growth theorems, and Hankel determinants rather than computational emulation (Ahamed et al., 2024, Obradović et al., 2020, Rath et al., 2023).

This distinction matters because arXiv searches for “Ozaki” return two largely unrelated research programs. In numerical HPC, the Ozaki scheme denotes a precision-structuring method for emulating high-accuracy GEMM from low-precision kernels (Ozaki et al., 10 Apr 2025, Mukunoki, 1 Aug 2025). In geometric function theory, “Ozaki” denotes analytic function classes and associated coefficient problems (Kumar et al., 2023, Ahamed et al., 2024). The shared surname masks a substantial terminological divergence, and modern usage therefore often benefits from the explicit labels Ozaki-I, Ozaki-II, or Ozaki close-to-convex class.