Evaluation of Bfloat16, Posit, and Takum Arithmetics in Sparse Linear Solvers

Abstract: Solving sparse linear systems lies at the core of numerous computational applications. Consequently, understanding the performance of recently proposed alternatives to the established IEEE 754 floating-point numbers, such as bfloat16 and the tapered-precision posit and takum machine number formats, is of significant interest. This paper examines these formats in the context of widely used solvers, namely LU, QR, and GMRES, with incomplete LU preconditioning and mixed precision iterative refinement (MPIR). This contrasts with the prevailing emphasis on designing specialized algorithms tailored to new arithmetic formats. This paper presents an extensive and unprecedented evaluation based on the SuiteSparse Matrix Collection -- a dataset of real-world matrices with diverse sizes and condition numbers. A key contribution is the faithful reproduction of SuiteSparse's UMFPACK multifrontal LU factorization and SPQR multifrontal QR factorization for machine number formats beyond single and double-precision IEEE 754. Tapered-precision posit and takum formats show better accuracy in direct solvers and reduced iteration counts in indirect solvers. Takum arithmetic, in particular, exhibits exceptional stability, even at low precision.

• The paper demonstrates that takum and posit formats achieve improved accuracy and reduced iteration counts compared to bfloat16 in sparse linear solvers.
• The methodology uses diverse matrices and both direct and iterative solvers, highlighting the viability of ultra-low precision in MPIR techniques.
• The study reveals that the broader dynamic range of takum arithmetic effectively mitigates numerical stability issues in low-precision computations.

An Evaluation of Emerging Numeric Formats in Sparse Linear Solvers

In the ongoing diversification of numerical representation formats, this paper provides an analytical assessment of alternative number formats—namely, bfloat16, posit, and takum—in the context of sparse linear solvers. The research shifts focus from traditional IEEE 754 floating-point arithmetic to potentially more efficient representations, motivated by the recent advancements in low-precision computation and challenges such as the "memory wall" in high-performance computing.

The experimental framework developed in this study evaluates these numeric formats using a diverse collection of matrices from the SuiteSparse Matrix Collection. The core evaluation targets include direct solvers like LU and QR factorization and iterative methods such as GMRES, equipped with incomplete LU preconditioning and Mixed Precision Iterative Refinement (MPIR).

Key Findings and Numerical Results

The experimental results reveal several crucial performance insights:

1. Tapered Precision Advantages: Both posit and takum formats demonstrate enhanced accuracy and reduced iteration counts across all tested numerical solvers compared to IEEE 754 floating-point formats. Specifically, takum arithmetic displays exceptional stability in low precision, which is pivotal for ensuring computational accuracy without resorting to higher precision data types.
2. Comparison with bfloat16: The performance of takum arithmetic consistently surpasses that of bfloat16 across different solvers and precision levels, particularly in the context of direct solvers and iterative methods.
3. Mixed Precision Iterative Refinement: Introducing 8-bit posits and takums in MPIR demonstrates their viability in this domain, yielding convergent results with few iterations—highlighting potential pathways for energy-efficient and high-performance computations using ultra-low precision.
4. Dynamic Range Efficacy: The linear takum format, with a broader dynamic range than posits, effectively tackles numerical stability issues that often challenge low-precision formats, thereby suggesting substantive future benefits for applications demanding high dynamic range with variable precision.

Implications and Future Directions

This study contributes critical insights into the practicality of employing advanced numeric representations in sparse linear solvers, which underpin numerous scientific and engineering applications. The evidence supporting takum formats as a favorable alternative to bfloat16 can stimulate further adoption in fields where 16-bit precision is standard. Furthermore, this work hints at broader implications for novel five-precision level iterative refinement strategies, coupling the extended dynamic range with scalable precision requirements.

Future research directions could capitalize on these findings by exploring the integration of equilibrated matrices in MPIR processes, potentially further elevating computational efficiency. Additionally, advancing GMRES iterative refinement with more than three precision levels may offer new avenues to harness these numeric formats' advantages.

The study stands as a comprehensive evaluation, making a strong case for rethinking number formats in high-performance computations—an endeavor that could fundamentally alter the precision dynamics in numerically intensive applications.