Inexactness and Correction of Floating-Point Reciprocal, Division and Square Root (2404.00387v1)
Abstract: Floating-point arithmetic performance determines the overall performance of important applications, from graphics to AI. Meeting the IEEE-754 specification for floating-point requires that final results of addition, subtraction, multiplication, division, and square root are correctly rounded based on the user-selected rounding mode. A frustrating fact for implementers is that naive rounding methods will not produce correctly rounded results even when intermediate results with greater accuracy and precision are available. In contrast, our novel algorithm can correct approximations of reciprocal, division and square root, even ones with slightly lower than target precision. In this paper, we present a family of algorithms that can both increase the accuracy (and potentially the precision) of an estimate and correctly round it according to all binary IEEE-754 rounding modes. We explain how it may be efficiently implemented in hardware, and for completeness, we present proofs that it is not necessary to include equality tests associated with round-to-nearest-even mode for reciprocal, division and square root functions, because it is impossible for input(s) in a given precision to have exact answers exactly midway between representable floating-point numbers in that precision. In fact, our simpler proofs are sometimes stronger.
- 2019. IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2019 (Revision of IEEE 754-2008) (July 2019), 1–84. https://doi.org/10.1109/IEEESTD.2019.8766229
- DLFloat: A 16-b Floating Point Format Designed for Deep Learning Training and Inference. In 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). 92–95. https://doi.org/10.1109/ARITH.2019.00023
- Nicolas Brisebarre and Jean-Michel Muller. 2007. Correct Rounding of Algebraic Functions. RAIRO - Theoretical Informatics and Applications 41, 1 (Jan. 2007), 71–83. https://doi.org/10.1051/ita:2007002
- Correctness Proofs Outline for Newton-Raphson Based Floating-Point Divide and Square Root Algorithms. In Proceedings 14th IEEE Symposium on Computer Arithmetic (Cat. No.99CB36336). IEEE Comput. Soc, Adelaide, SA, Australia, 96–105. https://doi.org/10.1109/ARITH.1999.762834
- David Goldberg. 1991. What Every Computer Scientist Should Know about Floating-Point Arithmetic. Comput. Surveys 23, 1 (March 1991), 5–48. https://doi.org/10.1145/103162.103163
- An FPGA Implementation of Pipelined Multiplicative Division with IEEE Rounding. In 15th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM 2007). 185–196. https://doi.org/10.1109/FCCM.2007.59
- John Harrison. 1999. A Machine-Checked Theory of Floating Point Arithmetic. In Theorem Proving in Higher Order Logics, Yves Bertot, Gilles Dowek, Laurent Théry, André Hirschowitz, and Christine Paulin (Eds.). Vol. 1690. Springer Berlin Heidelberg, Berlin, Heidelberg, 113–130. https://doi.org/10.1007/3-540-48256-3_9
- John Harrison. 2000a. Formal Verification of Floating Point Trigonometric Functions. In Formal Methods in Computer-Aided Design (Lecture Notes in Computer Science), Warren A. Hunt and Steven D. Johnson (Eds.). Springer, Berlin, Heidelberg, 254–270. https://doi.org/10.1007/3-540-40922-X_14
- John Harrison. 2000b. Formal Verification of IA-64 Division Algorithms. In Theorem Proving in Higher Order Logics (Lecture Notes in Computer Science), Mark Aagaard and John Harrison (Eds.). Springer, Berlin, Heidelberg, 233–251. https://doi.org/10.1007/3-540-44659-1_15
- C. Iordache and D.W. Matula. 1999. On Infinitely Precise Rounding for Division, Square Root, Reciprocal and Square Root Reciprocal. In Proceedings 14th IEEE Symposium on Computer Arithmetic (Cat. No.99CB36336). IEEE Comput. Soc, Adelaide, SA, Australia, 233–240. https://doi.org/10.1109/ARITH.1999.762849
- Midpoints and Exact Points of Some Algebraic Functions in Floating-Point Arithmetic. IEEE Trans. Comput. 60 (March 2011). https://doi.org/10.1109/TC.2010.144
- W Kahan. 2004. A Logarithm Too Clever by Half. https://people.eecs.berkeley.edu/~wkahan/LOG10HAF.TXT.
- A Study of BFLOAT16 for Deep Learning Training. arXiv:cs, stat/1905.12322
- Peter Markstein. 2000. IA-64 and Elementary Functions: Speed and Precision. Prentice Hall PTR.
- P. W. Markstein. 1990. Computation of Elementary Functions on the IBM RISC System/6000 Processor. IBM Journal of Research and Development 34, 1 (Jan. 1990), 111–119. https://doi.org/10.1147/rd.341.0111
- Jean-Michel Muller. 2005. On the Definition of Ulp (x). Ph.D. Dissertation. INRIA, LIP.
- D. Price. 1995. Pentium FDIV Flaw-Lessons Learned. IEEE Micro 15, 2 (April 1995), 86–88. https://doi.org/10.1109/40.372360
- James E Robertson. 1958. A New Class of Digital Division Methods. IRE transactions on electronic computers 3 (1958), 218–222.
- Keith D Tocher. 1958. Techniques of Multiplication and Division for Automatic Binary Computers. The Quarterly Journal of Mechanics and Applied Mathematics 11, 3 (1958), 364–384.