Mixed Multi-Precision Dataflow Strategy

• Mixed multi-precision dataflow is a strategy that assigns variable numeric precision in pipelines to reduce bandwidth and energy consumption while closely approximating full-precision results.
• Architectural implementations use heterogeneous mapping, compute-in-memory arrays, and fine-grained scheduling to optimize the allocation and switching of numeric precisions.
• Empirical evaluations show that these strategies enhance throughput, minimize latency, and preserve accuracy across deep learning, HPC, and numerical solver applications.

A mixed multi-precision dataflow strategy orchestrates the movement and processing of data streams in computational pipelines where numeric precision varies across operands, layers, spatial regions, or temporal phases. This class of dataflow is now central in high-throughput deep learning accelerators, scientific solvers, and high-performance numerical kernels, as it enables aggressive reductions in bandwidth, memory footprint, and computational energy while preserving, or closely approximating, full-precision task accuracy. Architectures and algorithms have evolved to exploit both static and dynamic precision assignment, support rapid precision switching, and optimize the mapping of multi-precision operations onto modern hardware primitives.

1. Principles and Motivations

Mixed multi-precision dataflow arises from the observation that not all computations demand uniform high precision. In CNNs, early layers are often over-provisioned in bit-width; in scientific solvers, iterative updates can tolerate low precision steps, provided that occasional high-precision corrections restore global convergence. The main driving principles are:

• Bandwith and energy minimization: Lower bit-widths cut DRAM/BRAM traffic and dynamic power, crucial for memory-bound applications and edge inferencing.
• Throughput maximization: Hardware units (FPGAs, GPUs, custom accelerators) achieve higher MAXs/s at reduced precision, sometimes by an order of magnitude.
• Precision-adaptive convergence: In multistage algorithms, critical operations (e.g., residual correction) are computed in higher precision to ensure numerical stability, while bulk operations proceed at low precision.
• Dynamic adaptation: Run-time metrics can drive precision switches, maximizing speedup during robust training phases and elevating precision when gradients become information-starved.

These principles have been concretized in diverse domains, including FPGA-based CNN inference (Latotzke et al., 2022), quantized fixed-point deep learning training (Rajagopal et al., 2020), conjugate-gradient solvers for LQCD (Jang et al., 2011, Clark et al., 2023), mixed-precision iterative refinement in dense linear systems (Oktay et al., 2021), patch-based neural inference on MCUs (Tao et al., 2024), tile-centric HPC matrix multiply (Zhang et al., 20 Aug 2025), FPGA compute-in-BRAM DNN engines (Chen et al., 2023), and hardware-aware quantized GEMM for DL inference (Martínez et al., 13 Jun 2025).

2. Architectural and Hardware Design Patterns

Architectures implementing mixed multi-precision dataflow expose several common patterns:

• Heterogeneous precision mapping: Processing elements (PEs) dynamically select the active subset of multiplier slices or reconfigure the operand wordlength on a per-operation, per-layer, or per-tile basis. E.g., BP-ST-1D MACs on FPGAs connect only ⌈w_Q(ℓ)/k⌉ partial product generators (PPGs) per layer/channel of the CNN, with k fixed at design time and w_Q programmable (Latotzke et al., 2022).
• Systolic or compute-in-memory arrays: Mixed-precision is achieved by re-grouping primitive low-bit multipliers (e.g., in SPEED’s RISC-V vector SAU of 16×4b units grouped to form 1×16b, 4×8b, or 16×4b MACs) (Wang et al., 2024), or by partitioning on-chip BRAMs so that each PE supports multi-bit-precision weight/activation loading (Chen et al., 2023).
• Fine-grained dataflow scheduling: Dataflow engines select between feature-first and channel-first tiling per layer as in SPEED, maximizing data reuse and compute-to-communication ratio according to kernel size and precision assignment (Wang et al., 2024). FPGA accelerators may split the network into streaming and time-multiplexed blocks with group-specific quantization and buffering to optimize BRAM and DRAM bandwidth (Nguyen et al., 2020).
• Multi-format storage and in-flight conversion: Tensors are stored in bit-packed, custom or standardized formats (e.g., IEEE-fp16/32/64, int20, int30, or block floating-point), with upward or downward conversion at load/unpack boundaries or at register-file ingress (Clark et al., 2023, Zhang et al., 20 Aug 2025, Zee et al., 2019).

Table: Representative Mixed Multi-Precision Architectural Elements

Hardware Domain	Precision Control Mechanism	Dataflow Adaptation
FPGA CNN (Latotzke et al., 2022)	LUT-programmed per-layer/channel wordlength, clock gating	PE slicing, tile size per precision
RISC-V DNN (Wang et al., 2024)	Custom VSACFG instruction for 4/8/16b, per-layer dataflow	Feature-first or channel-first tiling
Compute-in-BRAM (Chen et al., 2023)	Duplication shuffler, per-port per-bit packing	Tile-wise assignment with activation-sharing
HPC GEMM (Zhang et al., 20 Aug 2025)	Per-tile (block) precision metadata, in-flight conversion	DAG-based PaRSEC scheduling
BLIS GEMM (Zee et al., 2019)	Packing/casting to computation precision per operand	Microkernel abstraction

3. Dataflow Scheduling and Precision Switching Strategies

Mixed multi-precision dataflow encompasses both static and dynamic schemes:

• Static (compile-time/fixed schedule): Bit-width assignment is determined per layer, per channel, per tile, or per patch branch, typically through profiling, Bayesian optimization, or heuristic rules. FPGAs leverage Boolean parameter tables or small LUTs to reconfigure PE participation on the fly (Latotzke et al., 2022, Nguyen et al., 2020, Tao et al., 2024); static tiling in GEMM is orchestrated with a global or numerically-aware tile mapping (Zhang et al., 20 Aug 2025).
• Dynamic (run-time/adaptive switching): Precision is escalated at run time in response to monitored metrics, such as inter-epoch gradient diversity in neural network training (MuPPET) (Rajagopal et al., 2020) or correction-stagnation in iterative refinement (Oktay et al., 2021). These policies compare online metrics to decaying thresholds, triggering quantization regime transitions to avoid stagnation or accuracy loss.
• Iterative multi-stage approaches: Multi-stage refinement strategies (as in MSIR) escalate from bare-bones low-precision solves, through more robust GMRES-based correction, to full high-precision refactorization only when convergence criteria so dictate (Oktay et al., 2021).
• Hierarchical DSE: Pareto-optimal designs are explored via layer-wise, PE-array-level, and system-level dataflow/precision sweeps, balancing resource use, accuracy, and throughput (Latotzke et al., 2022).

4. Algorithmic and Implementation Techniques

Algorithmic constructs for mixed multi-precision dataflow include:

• Partial sum management: Activations typically maintain higher-bit accumulators (e.g., INT32 or FP32) even as input operands are stored or processed at 1–8 bits, avoiding overflow/precision loss (Latotzke et al., 2022, Martínez et al., 13 Jun 2025). Quantized GEMM solves perform micro-tile accumulation in the accumulator precision, then rescale/dequantize at the output (Martínez et al., 13 Jun 2025).
• Packing/unpacking and typecasting: High-performance frameworks (BLIS, HPC-GEMM) push all casting into O(1)-per-element packing/unpacking routines, compiling a dataflow that minimizes convert-instruction overhead and preserves hot microkernel simplicity (Zee et al., 2019, Zhang et al., 20 Aug 2025).
• Dual-function memory/compute units: Compute-in-BRAM engines (e.g., M4BRAM) enable concurrent memory access and MAC in block storage, integrating broadcast/duplication to saturate hardware utilization under varying precision constraints (Chen et al., 2023).
• Custom number formats: Int-B (int20, int30, bit-packed float, block floating point) deliver more effective mantissa bits per word than conventional IEEE formats and permit greater control over accuracy-bandwidth trade-offs (Clark et al., 2023).
• Error and stability control: Reliable updates, gradient re-projection, and numerically motivated tolerance setting ensure that mixed-precision execution does not sacrifice global convergence or stability, especially in iterative scientific solvers (Jang et al., 2011, Oktay et al., 2021, Clark et al., 2023).

5. Performance, Accuracy, and Trade-offs

Empirical results across domains consistently demonstrate the value of mixed multi-precision dataflow:

• Throughput and Area Efficiency: FPGA CNN accelerators achieve 1.13 TOps/s (ResNet-152, 127 MHz) with a 9.4× parameter footprint reduction versus floating-point baselines; RISC-V vector engines (SPEED) yield up to 287 GOPS, a 6× area-efficiency uplift over prior RVV engines in 4-bit mode (Latotzke et al., 2022, Wang et al., 2024).
• Bandwidth and Latency Wins: Compute-in-BRAM M4BRAM achieves 2.2× speedup in DNN inference, reducing off-chip traffic and retaining full memory functionality; mixed-precision DNN deployment on MCUs achieves ≈2.2× BitOPs reduction with ≤1% accuracy loss (Chen et al., 2023, Tao et al., 2024).
• Accuracy Preservation: Parameter quantization to aggressive bit-widths with Bayesian optimization (e.g., 1.1–1.4 bits/weight) enables model size reductions of 22–29×, while mAP and Top-1 accuracy drops remain under 1% compared to full-precision (Nguyen et al., 2020).
• Mixed Precision in Matrix Operations: Fine-grained tile-level assignment in hardware-aware GEMM (Zhang et al., 20 Aug 2025) and dot-product-centric MIP-GEMM (Martínez et al., 13 Jun 2025) delivers near-linear throughput gains (2–4×) as INT8/SP fraction increases, maintaining relative Frobenius errors within 10^-12 of DP references and <1% accuracy loss on DNN benchmarks.
• Energy and Communication Reduction: INT8 inference yields up to 18.5× less energy per MAC than FP32 (Rajagopal et al., 2020); communication volume in mixed-precision finite difference solvers scales with bit-width, achieving up to 3× reduction over DP (Siklósi et al., 27 May 2025).
• Resilience and Scalability: Multi-GPU solvers (Jang et al., 2011) and mixed-tile GEMM (Zhang et al., 20 Aug 2025) display strong scaling, error resilience, and parallel efficiency up to thousands of nodes by virtue of the adaptive, quantized dataflow.

6. Applications and Methodological Significance

Mixed multi-precision dataflow strategies have become foundational in:

• Deep Neural Network Inference and Training: Layer/channel/tile-wise quantization, dynamic precision switching during training (e.g., MuPPET), and patch-based inference optimizations for MCUs (Latotzke et al., 2022, Rajagopal et al., 2020, Tao et al., 2024).
• Scientific HPC and Lattice QCD: Krylov solvers with dynamic register/DRAM format duality, bit-packed representations, and periodic accuracy correction (Jang et al., 2011, Clark et al., 2023).
• Matrix Multiplication and Linear Algebra: Tile-centric mixed-precision DAGs orchestrated by runtime systems (PaRSEC), and GEMM microkernels exploiting mixed-precision ISA extensions (Zhang et al., 20 Aug 2025, Martínez et al., 13 Jun 2025, Zee et al., 2019).
• Explicit Finite-Difference PDE Solutions: Runge–Kutta pipelines assigning high-precision only to accumulators, using lower-precision for local temporaries and fluxes (Siklósi et al., 27 May 2025).

The pervasive adoption of mixed multi-precision dataflows is enabled by algorithmic, hardware, and software co-design, with key challenges including dynamic scheduling, error control, resource mapping, and the development of policies that remain robust across diverse architectures and application domains.

7. Design Guidelines and Outlook

The collective results motivate several design desiderata:

1. Keep accumulators and global state in higher precision; lower-precision is reserved for inputs, intermediate temporaries, or inner iterations (Siklósi et al., 27 May 2025, Latotzke et al., 2022).
2. Exploit hardware: Use native rethinkable ISA instructions (e.g., RISC-V custom, SIMD DOT, in-BRAM MAC), packing, and flexible controller logic (Wang et al., 2024, Chen et al., 2023, Martínez et al., 13 Jun 2025).
3. Balance design parameters: Choose tile size, PE slicing, and dynamic precision thresholds to Pareto-optimize throughput, logic, power, and accuracy (Latotzke et al., 2022, Zhang et al., 20 Aug 2025).
4. Validating accuracy: Mixed-precision schemes must be empirically benchmarked against discretization error and convergence criteria to ensure reliability (Oktay et al., 2021, Rajagopal et al., 2020, Siklósi et al., 27 May 2025).
5. Software frameworks: Leverage templated code (OPS, OpenSBLI, BLIS) and runtime systems (PaRSEC) to manage complexity and portability (Zhang et al., 20 Aug 2025, Zee et al., 2019, Siklósi et al., 27 May 2025).

Mixed multi-precision dataflow strategies now constitute a mature and essential paradigm in both specialized accelerators and portable HPC workflows, offering a scalable response to the shifting balance of algorithmic and hardware constraints in contemporary computational science and machine learning.