--- title: Custom Floating-Point Formats url: https://www.emergentmind.com/topics/custom-floating-point-formats-d9c7642c-4114-4ea2-8d0c-782593a21f3c type: topic --- # Custom Floating-Point Formats Custom floating-point (FP) formats are user- or domain-defined representations for real numbers with parameters like exponent width, mantissa width, and encoding rules set according to application-specific constraints. These formats allow designers to optimize for dynamic range, precision, energy, area, or application-level error, and stand in contrast to rigid IEEE 754 standard formats. Research across deep learning, scientific computing, hardware design, and software vectorization demonstrates vibrant activity in custom FP, with rigorous methodologies for design, analysis, and hardware/software co-design. ## 1. Format Definitions and Parameterization A custom floating-point format typically splits a total bit-width $N$ into fields: - **Sign**: $1$ bit (optional for nonnegative-only signals) - **Exponent**: $E$ bits (controls dynamic range) - **Mantissa (Fraction)**: $M$ bits (controls precision) - **Bias**: Typically $2^{E-1}-1$ (but can be tunable) Normalized numbers are interpreted as: $$ x = (-1)^s \times 2^{e-b} \times \left(1 + \frac{f}{2^M}\right) $$ where $s$ is the sign bit, $e$ the stored exponent, $b$ the bias, $f$ the mantissa. Subnormals, NaN, $\pm\infty$, and IEEE-754 special value handling may be omitted for efficiency in custom variants, or, as with IEEE-style custom formats, retained for compatibility [1909.13271, 2007.01530, 2212.04184, 2207.03192]. Nonlinear and tapered-precision encodings such as posit and takum decouple dynamic range and precision, allowing regime/exponent fields of variable or fixed width to "taper" the mantissa precision as a function of magnitude [2503.14067, 1811.01721, 2504.21197, 2409.16626]. Some formats adapt field boundaries dynamically for entropy coding (EFloat) [2102.02705]. Parameter configuration may be per-tensor, per-layer, per-block, or reconfigurable at runtime, as in FFP8, AdaptivFloat, or run-time reconfigurable FPGA/ASIC multipliers [2104.07329, 1909.13271, 1909.13318]. ## 2. Dynamic Range, Precision, and Format Design Rules The tradeoff between exponent width (dynamic range) and mantissa width (precision) underpins custom FP design. - **Dynamic Range**: $2^{2^{E-1}-1}$ for normalized IEEE-like formats; subnormals extend this [2212.04184, 2007.01530]. - **Unit in the Last Place (ULP)**: At unit magnitude, $2^{-M}$; for numbers outside the [1,2) range, the ULP scales accordingly [2212.04184]. - **Rounding Error**: With round-to-nearest-even, max error per operation is $0.5$ ULP. - **Field Splitting Principles**: For a given word length $N=1+E+M$, increasing $E$ doubles range but halves mantissa precision. Applications with large value distributions need $E\ge5$ even for 8-bit formats. Application-specific error budgets dictate the required $M$, sometimes as low as $2$–$4$ [2212.04184, 2207.03192]. - **Layer/Tensor Specialization**: Modern DNNs and signal domains often benefit from per-layer or per-tensor FP format specialization, exploiting narrower dynamic range in intermediate tensors to increase mantissa bits [1909.13271, 2104.07329]. Formats such as AdaptivFloat explicitly recompute an exponent bias per layer to always maximize representable range and minimize quantization error—an approach that consistently outperforms block floating-point, posit, and uniform integer quantization under tight bitwidth budgets [1909.13271]. ## 3. Hardware, Algorithmic, and Software Implementations ### Hardware Implementations - **Customizable FPUs**: Open-source units like FPnew implement a datapath parameterized by $(1,E,M)$ fields, supporting fast switching between multiple custom or standard floating-point formats. Fine-grained SIMD capabilities are integrated for energy and throughput proportionality down to 8 bits. Going from 64 to 8 bits yields over $10\times$ energy and throughput improvement in the FPnew silicon measurements [2007.01530]. - **Dynamic and Run-Time Reconfiguration**: Some designs allow dynamic selection among several exponent/mantissa split modes (fixed at $8$, $16$, $23$, $36$, $52$ bits mantissa) per operand, facilitating adaptation to the accuracy/performance tradeoff at runtime [1909.13318]. FFP8 and AdaptivFloat also use per-layer coefficients or runtime configuration to optimize quantization windows [2104.07329, 1909.13271]. - **Specialized Hardware**: Designs may omit denormals, reduce logic for rounding, and use compact mantissa multipliers or hybrid algorithms (e.g., Karatsuba + Vedic multipliers) for area and energy gains, especially in FPGA deployments [1909.13318, 2409.05837]. ### Software and Algorithmic Implementations - **Bitslice Vectorization**: Bit-level parallelism allows software emulation of arbitrary-precision FP arithmetic using regular integer SIMD units, circumventing fixed hardware width and offering efficient FP for low $n$ ($n<16$). This approach excels when vector width is large and precision is small [1602.04716, 2007.06563]. - **Domain-Specific Libraries and DSLs**: Libraries such as FlexFloat [1711.10374] and custom DSLs [2409.05837] facilitate rapid prototyping of circuits and algorithms using user-defined precision, with support for vectorization and automatic pipeline register balancing. - **Mixed-Precision Toolchains**: Tools iterate over call-sites or tensor-variables to minimize $(E,M)$ per site, subject to an overall error budget defined using application metrics (e.g., SQNR, ULP error) [2005.02732, 1711.10374]. ## 4. Empirical Results, Comparative Analyses, and Application Domains Custom FP research corroborates improvements in inference/training accuracy, resource utilization, and energy efficiency. - **Deep Learning**: AdaptivFloat delivers up to $+0.3$ BLEU and $-0.75$ WER over FP32 at 8-bit and 6-bit quantization, outperforming 8-bit integer, block floating-point, and posit variants on Transformers, LSTMs, and CNNs [1909.13271]. FFP8 achieves Top-1 accuracy within $0.11\%$ of FP32 with per-layer tuning, with negligible hardware cost [2104.07329]. - **Large Language Models**: Microscaling (MXInt/BFP) formats allow LLM inference at average mantissa $\approx4$ bits and $<0.5\%$ loss vs. FP32, with near-int8 area/energy densities [2307.15517]. EFloat (entropy-coded floats) reclaims $\sim4.3$ bits from exponent into mantissa per value, with EF16 yielding $17\times$–$35\times$ lower RMSE vs. BF16 [2102.02705]. - **Scientific Computing**: Custom FPUs (FlexFloat-based) achieve an $18\%$ energy reduction and $12\%$ runtime reduction compared to all-32-bit, with most variables mapped to 8- or 16-bit formats [1711.10374]. - **Signal Processing/FFT**: In FFT-based spectral algorithms, posit8 and takum8 outperform E4M3/E5M2 and bfloat16, which are unstable due to limited range. Takum16 is specifically recommended for moderate-precision FFT/PDE tasks because of stable, high SNR [2504.21197]. - **Approximate Search/Compression**: Custom 8-bit unsigned formats, optimized for target value distribution (e4m4, e5m3), reduce memory bank conflicts and latency in GPU-based ANN search, with recall loss $<1\%$ [2301.06672]. - **RDBMS and Reproducibility**: Superaccumulator-based associative custom FP structures make floating-point aggregation reproducible at a cost of approximately $2\times$ in end-to-end wall-clock time, enabling consistent numerical results in high-cardinality group-by queries [1802.09883]. ## 5. Trade-Offs, Design Guidance, and Methodologies Designers face multidimensional trade-offs governed by format parameters, hardware constraints, and application-level correctness or quality targets: - **Energy/Area/Throughput vs. Precision/Range**: Lower bit-widths reduce silicon area and energy linearly (for adders) or sublinearly (for multipliers), but too little exponent width induces overflows/underflows; too little mantissa impairs numerical accuracy [2212.04184]. - **Application Mapping**: For inference and training of DNNs, 8–10 bit floats (with $E=4$–$5$) typically dominate fixed-point in $n\leq8$; for linear DSP, 12–16 bit fixed-point can outperform FP in accuracy/energy [2212.04184]. - **Layer/Block/Cluster Adaptation**: Layer-specific tuning (AdaptivFloat) and block-wise exponent sharing (MXInt/BFP) enable near-FP32 accuracy at substantially reduced bitwidth and resource [2307.15517, 1909.13271]. - **Mixed-Precision and Dynamic Policies**: Profiling tools (VPREC-libm, FlexFloat, MASE) enable per-call/per-tensor $(E,M)$ allocation under error, energy, or area constraints [2005.02732, 2307.15517]. Suggested methodology for format selection [2212.04184, 2005.02732]: 1. Profile value distributions to determine required dynamic range. 2. Choose exponent width so that all important data avoid overflow. 3. Allocate remaining bits to the mantissa to meet target error. 4. Validate via simulated or hardware-in-the-loop evaluation. 5. Where possible, tune formats per layer/tensor/call site to exploit local dynamic range/precision needs. ## 6. Novel Encodings and Alternatives to IEEE 754 Several lines of research pursue alternatives to static IEEE 754-style field splits to further optimize resource-accuracy trade-offs: - **Tapered Precision (Posit, Takum, HiFloat8)**: Regime-based encodings (posit, takum) concentrate precision near $|x|\approx1$, with reduced precision for extreme magnitudes, improving overall representational efficiency [2503.14067, 1811.01721, 2409.16626]. - **Entropy Coding (EFloat)**: Adaptive-length exponent codes reclaim bits for significand, leveraging nonuniform exponent distributions in data (notably in embeddings), providing lossless ±FP32 range at much lower total bitwidth [2102.02705]. - **MICROSCALE FORMATS/BLOCK FLOAT**: Block floating-point with shared exponents (MXInt, BFP) and per-block adaptation is effective in compressing weights/activations with nonnegative impact on LLM accuracy, dramatically reducing area and arithmetic cost [2307.15517]. - **Hardware Optimization via Bitslice**: Bitslice FP arithmetic, mapping arbitrary custom $(E,M)$ splits to wide integer SIMD instructions, provides high-throughput prototype evaluation and directly supports arbitrary-precision FP in software and hardware accelerators [1602.04716, 2007.06563]. | Format or Approach | Exponent Config | Mantissa Config | Notable Features | |--------------------|-----------------|-----------------|-------------------------------------------------------------------| | AdaptivFloat [1909.13271] | Per-layer variable bias $E$ | Fixed $M$ | Layer-specific dynamic range max/min, optimal clipping/rounding | | FPnew [2007.01530] | Param. $k$ | Param. $p$ | Multi-format FPU, scalar+SIMD, full IEEE-754 compliance | | EFloat [2102.02705]| Huffman code (δ bits avg) | $N-1-\delta$ | Exponent entropy coding, average 4-bit exponent, maximized mantissa| | HOBFLOPS [2007.06563] | Any $E$ | Any $M$ | Bitslice vectorization, software/hardware symmetry | | FFP8 [2104.07329] | $y$ (per-tensor/layer) | $z$ (per-tensor/layer) | Per-tensor format, tunable sign, exponent bias, zero retraining | | MXInt/BFP [2307.15517] | Block-shared $e$ | Per-elem $m$ | High dynamic range, mixed-precision search, dataflow accelerator | | Posit/Takum [2503.14067, 2504.21197] | Regime + exp | Tapered with magnitude | Nonlinear dynamic range/precision curve, class-universal encoding | ## 7. Limitations, Future Directions, and Ongoing Developments While custom FP formats provide compelling leverage for performance and energy, limitations persist: - **Hardware Complexity**: Nonstandard encodings (posits, takum, HiFloat8) require regime/scale decoding and sometimes result in irregular pipeline paths or more complicated arithmetic units [2503.14067, 1811.01721]. - **Conversion Overhead**: In software implementations, bitslice and custom FP representations may incur packing/unpacking costs, particularly above 16 bits or with irregular memory access [1602.04716]. - **Mixed-Precision Fragmentation**: Excessively granular or dynamic allocation of formats risks conversion overheads and poor vectorization [2005.02732, 1711.10374]. - **Application-Specific Trade-offs**: Some algorithms (large FFTs, PDE solvers) are exceptionally sensitive to dynamic range; others demand uniform precision. Custom FP adoption requires systematic benchmarking within target domains [2504.21197, 2212.04184]. - **Standardization vs. Specialization**: AVX10.2 and other ISAs now support multiple low-precision FP formats (E4M3/E5M2/bfloat16), but integrating a single universal, tapered-precision number format (e.g., takum) is proposed as a way to remove complexity [2503.14067]. Ongoing research is focused on unified encoding/decoding pipelines, extending compiler toolchains for pervasive mixed-precision inference/training, leveraging entropy and data-aware field mapping (as in EFloat), and exploring design-space exploration tools for automated hardware-software co-design at the system level. --- References: - AdaptivFloat: [1909.13271] - FPnew: [2007.01530] - Bitslice Vector Types: [1602.04716] - MXInt/MASE: [2307.15517] - MiniFloat-NN/ExSdotp: [2207.03192] - FP/FxP Cost/Precision Analysis: [2212.04184] - RDBMS Reproducibility: [1802.09883] - EFloat: [2102.02705] - FFP8: [2104.07329] - Precision Library/Profiling: [2005.02732] - Run-time Reconfigurable FP Multiplier: [1909.13318] - Takum SIMD/ISA: [2503.14067] - GPU-Oriented 8-bit ANNS: [2301.06672] - HiFloat8 Tapered FP8: [2409.16626] - Custom Spatial Filters/DSL for FPGAs: [2409.05837] - HOBFLOPS: [2007.06563] - FlexFloat/Transprecision: [1711.10374] - Log-float/Posit-Tapered: [1811.01721] - Spectral algorithms/FFTs in emerging formats: [2504.21197] Source: https://www.emergentmind.com/topics/custom-floating-point-formats-d9c7642c-4114-4ea2-8d0c-782593a21f3c