SEADA: An efficient methodology for optimizing mixed-precision DNNs on multi-precision spatial architectures

Abstract: Mixed-precision computation has been introduced in deep neural networks (DNNs) as an effective approach to reduce latency, energy consumption, and memory footprint. However, efficiently mapping mixed-precision networks onto multi-precision spatial architectures poses several challenges. These include determining the appropriate precision for each layer, balancing layer-wise accuracy sensitivity to quantization against architectural heterogeneity and system-level constraints, and accurately estimating the system-level cost of heterogeneous precision assignments. This work presents SEADA, an efficient methodology designed to address these challenges. SEADA comprises: (i) a configurable system-level analytical cost model of a multi-precision spatial accelerator architecture; (ii) a fast mapping tool that identifies near-optimal mappings of DNN workloads onto the target integer accelerator; (iii) analytical models for floating-point layers to estimate the overall benefits of mixed-precision execution; and (iv) a per-layer precision selection methodology based on bit-level entropy, enabling efficient assignment across multiple numerical precisions. SEADA's efficiency provides designers with a robust framework for the design-space exploration of multi-precision architectures.

• The paper demonstrates SEADA's integrated workflow using entropy-based metrics for per-layer precision assignment in mixed-precision DNNs to optimize energy and performance.
• It introduces an analytical cost model and a Knapsack-based mapping tool that accelerates design space exploration without the need for full model retraining.
• Experimental evaluations on BERT-base and ResNet-50 reveal significant energy, EDP, and time reductions while maintaining accuracy within constrained energy budgets.

SEADA: An Efficient Methodology for Optimizing Mixed-Precision DNNs on Multi-Precision Spatial Architectures

Introduction and Motivation

The exponential growth in computational demand and energy consumption for deep neural networks (DNNs) has catalyzed the exploration of mixed-precision computation as an optimization strategy. Mixed-precision DNNs exploit layer-wise heterogeneity in quantization sensitivity, assigning different bit-widths to different layers. However, efficiently mapping such networks onto multi-precision spatial hardware introduces intricate challenges, particularly in the selection and orchestration of per-layer precisions under tight system-level constraints. The SEADA methodology directly confronts these challenges through an integrated workflow and analytic modeling approach, targeting both hardware and workload co-design.

Methodological Framework

SEADA comprises four principal components: a configurable analytical cost model for system-level performance evaluation, an accelerated mapping tool supporting per-layer precision assignment, analytic models for the accurate estimation of floating-point layers, and an entropy-driven, layer-wise precision selection methodology.

The canonical DNN acceleration workflow and the SEADA framework are juxtaposed below, highlighting SEADA's streamlined and analytic approach to joint accuracy-performance optimization.

Figure 1: (a) State-of-the-art DNN mapping and quantization workflow. (b) Main SEADA methodology components for efficient, accurate mixed-precision design-space exploration.

SEADA adopts the Entropy Approximation Guided Layer selection (EAGL) paradigm for fast, scalable per-layer bit-width allocation, using empirical bit-level entropy as a proxy for quantization sensitivity. Cost and benefit metrics for each configuration are precomputed into a mapping library, enabling rapid lookup-based evaluation over the design space. This facilitates efficient Knapsack-based assignment of bit-widths subject to system-level constraints, decoupling precision assignment from full retraining.

Target Architecture and Modeling

The target hardware model is a generic multi-precision spatial accelerator supporting both fixed-point and floating-point pipelines within each processing element (PE). Functional units dynamically support INT8/INT4/INT2 and FP32/FP16/FP8/FP4 instructions, with the architecture exposing flexible dataflow and on-chip memory hierarchy parameters.

Figure 2: Spatial accelerator architecture template with multi-precision functional units, hierarchical memory, and configurable dataflow.

QuickFlow, an iterative local-search-based mapper, is extended to model the unique constraints of mixed-precision computation and data movement, including operand packing and partial-sum handling in the presence of heterogeneous precision assignments. Energy, delay, and EDP estimates thereby capture true system-level cost trade-offs induced by precision heterogeneity.

Analytical Floating-Point Layer Models

Not all DNN layers benefit from quantization; normalization and activation layers (LayerNorm, BatchNorm, GELU, and MHSA in Transformer architectures) require high-precision arithmetic to preserve accuracy. SEADA includes detailed analytic models for these layers, employing operation fusion (e.g., fusing dequantization and quantization with core computations) to minimize the incremental cost of precision conversion. Fast, polynomially approximated functional units are assumed for expensive operations such as exponentiation and reciprocals, consistent with modern DNN-optimized hardware.

Figure 3: (a) BERT-base and (b) ResNet-50 computational graphs highlighting the partitioning of low-precision and full-precision operations.

Precision Assignment Strategy

Layer-wise precision allocation is cast as a multi-choice Knapsack Problem, maximizing estimated accuracy subject to an energy (or other cost) budget. EAGL's empirical-entropy-based gain metric is extended to multi-bitwidth configurations (beyond binary choices). Precision assignments respect activation-sharing constraints across architectural groups, and mixed-precision configurations are instantiated in PyTorch via automated module rewriting.

Fine-tuning for hardware-constrained mixed-precision models is conducted using knowledge distillation from full-precision teacher networks to hasten convergence and robustly recover accuracy.

Experimental Evaluation

BERT-base Accelerator Mapping

Extensive evaluation is performed on a BERT-base model (fine-tuned on SQuAD 1.1), mapped to an Eyeriss-derived architecture with full analytical modeling of computation and data movement at the precision granularity level.

Significant energy and EDP improvements are observed as energy budgets are reduced by constraining more layers to lower bit-widths, with accuracy loss remaining below 1% for budgets above 45%.

Figure 4: (a) Fine-tuned BERT-base accuracy vs. energy budget; (b) Normalized EDP for various layer-precision distributions; (c) Normalized energy partitioned by layer type.

Comprehensive energy breakdowns clarify that once major compute kernels are quantized, non-accelerated floating-point layers (esp. MHSA) become dominant energy consumers, underscoring the necessity of holistic, end-to-end optimization strategies.

Figure 5: Detailed energy breakdown by architectural component and BERT-base block, normalized to QKV layers, highlighting the impact of precision assignment.

ResNet-50 Accelerator Mapping

Analogous analysis for ResNet-50 (ImageNet) further validates the generality of SEADA's approach:

• Accuracy loss remains within 4.5% for energy budgets above 55%.
• Mixed-precision enables up to 48% energy reduction relative to an 8-bit baseline, with EDP reductions as high as 66% at the lowest budget.

Figure 6: (a) ResNet-50 accuracy vs. energy budget; (b) Normalized EDP for different layer-wise precision allocations; (c) Energy by layer type.

Architectural energy dissection reveals the distinct contributions across the convolutional, memory, and normalization blocks.

Figure 7: Energy breakdown for ResNet-50 layers and architectural components, normalized to the initial convolution.

Holistic Design Space Exploration

A core outcome of SEADA is its integration within broad architectural exploration loops. The introduced Computational Bit Reduction (CBR) metric—measuring the effective reduction in overall bitwise MAC operations—demonstrates strong correlation with observed model accuracy after fine-tuning. This enables hardware-agnostic proxying of accuracy during architectural parameter sweeps, supporting rapid Pareto front identification for EDP and accuracy-coherent solutions.

Figure 8: Pareto frontier of accelerator design space, jointly minimizing EDP and CBR; the horizontal threshold reflects the validated CBR–accuracy relationship.

Numerical Results and Claims

• For BERT-base, SEADA achieves a 48% EDP reduction at 45% energy budget with <1% accuracy loss, and 97% EDP reduction over full-precision baselines.
• For ResNet-50, similar budgets yield 57% EDP and 39% energy reductions, with up to 4.5% accuracy degradation.
• Cost of fused quantization/dequantization is shown to be marginal for compute-bound workloads (BERT) but significant for memory-bound tasks (ResNet-50 BN layers), with up to 40% energy savings in the latter.
• The total computational design space for ResNet-18 is shown to be intractably large ($\mathcal{O}(10^{33})$) for exhaustive search.
• The efficiency of SEADA's analytic modeling and library-based lookup reduces mapping and precision allocation time to minutes, vastly outpacing alternatives reliant on retraining or brute-force methods.

Theoretical and Practical Implications

The research reconceptualizes mixed-precision optimization as a joint analytical, mapping, and selection problem, leveraging entropy-based sensitivity proxies and operation fusion. The demonstrated benefit is a rapid, unified methodology that aligns hardware and workload optimization, crucial for next-generation, resource-constrained DNN deployments.

The CBR metric's operational decoupling of mapping and accuracy estimation offers a practical surrogate for large-scale, cross-architectural exploration, with direct impact for both ASIC/FPGA DNN accelerator design and neural architecture search frameworks.

Future Directions

Potential future developments include extending EAGL-like entropy metrics to sub-layer or channel/group granularities, integrating QAT-specific improvements for extreme quantization scenarios, and broadening analytic modeling to arbitrary operator fusion and non-uniform, per-tensor precision assignments. As hardware specialization evolves, further expansion of SEADA's mapping and cost libraries to exotic PE architectures (e.g., in-memory computing, stochastic rounding units) is warranted.

Conclusion

SEADA embodies a rigorous, efficient framework for optimizing mixed-precision DNNs on multi-precision spatial hardware, uniting fast analytical estimation, mapping-aware cost modeling, and entropy-driven bit-width allocation. The approach exceeds limitations of prior work—grounded in both hardware and algorithmic realities—enabling tractable, accurate co-design across the algorithm-architecture spectrum (2606.27884).