Precision-Scalable Carry-Save Multiplier

• Precision-scalable CSM is a reconfigurable multiplier unit that supports multiple numeric formats (INT, FP, MX) to optimize area, power, and energy efficiency in MAC architectures.
• It employs dynamic operand partitioning and 2D sub-word sharing to reduce shifter count and enable parallel processing, resulting in significant resource savings.
• Integrated in-line exponent adjustment and flexible mode control ensure seamless dynamic precision scaling while maintaining computational throughput and sub-0.2% MAC error.

A precision-scalable carry-save multiplier (CSM) is a fundamental arithmetic unit designed for multiply-accumulate (MAC) architectures, capable of dynamically morphing its datapath to support a variety of data precisions and formats—including INT, floating-point (FP), and microscaling (MX) representations. Precision-scalable CSMs, as exemplified by the “Jack Unit,” replace the conventional fixed-width FP-significand multipliers with a reconfigurable structure. This approach exploits dynamic operand partitioning, 2D sub-word sharing, and integrated exponent-difference adjustment to achieve 1.17–2.01× area savings, 1.05–1.84× power reduction, and significant improvements in energy efficiency compared to standard CSMs, while maintaining full throughput and supporting dynamic format switching (Noh et al., 7 Jul 2025).

1. Architectural Principles

The essential innovation of the precision-scalable CSM is the replacement of a fixed-width (e.g., 8×8-bit) multiplier with a single datapath controllable by a “mode” signal. This datapath can be dynamically configured to operate as:

• A single 8×8-bit signed multiplier
• Four 4×4-bit signed multipliers in parallel
• Mixed 4×8 or 8×4 multipliers for specific MXFP/MXINT formats

Operands $X[7:0]$ and $W[7:0]$ are partitioned into “high” and “low” 4-bit sub-words:

$X = X_H \cdot 2^4 + X_L \ W = W_H \cdot 2^4 + W_L$

Partial products are then generated as:

• $P_0 = X_L \cdot W_L$ (weight $2^0$)
• $P_1 = X_L \cdot W_H$ (weight $2^4$)
• $P_2 = X_H \cdot W_L$ (weight $2^4$)
• $P_3 = X_H \cdot W_H$ (weight $2^8$)

A small carry-save reduction tree combines these appropriately shifted results to yield the complete multiplied value. In lower-precision modes, sub-multiplier outputs remain partitioned, while in full precision, outputs are fused. Operand delivery leverages a bypassable 8-wire bus with pipeline registers, permitting sequential delivery of wider operands (e.g., 16-bit over two cycles) without bandwidth loss.

2. Partial Product Generation and 2D Sub-word Parallelism

To maximize hardware efficiency, the precision-scalable CSM employs “2D sub-word parallelism.” Instead of assigning independent shifters to each CSM, multiple CSMs (typically four) are fused into a “super-CSM” structure. The grouping of identical shift amounts for partial products (e.g., all $P_3$ terms are shifted by $2^8$) enables shared shifters across columns and rows. For four CSMs ($4 \times 4 = 16$ 4×4 multipliers), only one shifter is needed per unique shift group—reducing the shifter count by 75%. In lower-precision or sparse modes, unused sub-multipliers and shifters are power-gated. The sharing principle persists, as all active subwords in a group experience the same shift.

3. Inline Exponent-Difference Significand Adjustment

Unlike traditional FP MACs, which handle exponent alignment outside the CSM, the precision-scalable CSM integrates this adjustment within the multiplier. For products with exponents $E_1$ and $E_2$, alignment requires shifting the significand with the smaller exponent by $\Delta E = |E_1 - E_2|$, so that

$$S_1 \cdot 2^{E_1} + S_2 \cdot 2^{E_2} \rightarrow S_1 \cdot 2^{E_{max}} + (S_2 \gg \Delta E) \cdot 2^{E_{max}}$$

Within the carry-save tree, each partial product $S_i$ is shifted right by $\Delta E_i = E_{max} - E_i$, permitting the final accumulation to proceed as a pure integer sum:

$$\sum_i \left(S_i \gg \Delta E_i \right) \times 2^{E_{max}}$$

Exponent extraction is implemented using a small comparator tree that determines $E_{max}$ in a single cycle. For FP and MXFP modes, sign bits are XORed in parallel to build the sign bundle driving downstream shifters.

4. Dynamic Precision Scaling and Control

A 3-bit mode signal selects the active data representation ({INT8, INT4, FP8, bfloat16, MXINT8, MXFP8, ...}) and modulates several operational aspects:

• Clock-gating of inactive sub-multipliers
• Operand lane-packing and pipeline control on the bypass bus
• Activation of the exponent extractor and normalization logic
• Insertion of a “bias” in MX modes (to enable shared exponents)

Within each 4×4 multiplier, gating logic latches a “live” signal, determined by the mode selection. Barrel shifters for exponent alignment are disabled whenever $\Delta E_i \equiv 0$ or in pure integer modes. Power gating extends to unused adder stages at the output. In MX configurations, only one exponent calculator operates per block (adjustable block size up to 32), further economizing on control logic.

5. Area, Power, and Delay Characteristics

A direct comparison against conventional CSMs demonstrates the efficiency of precision-scalable design. The table below summarizes the reported metrics for the major architectural improvements:

Improvement Step	Area Reduction	Power Reduction	Critical Path
Precision-scalable CSM (MAC② vs MAC①)	$1.37\times$		3.60 ns
+ In-CSM exponent adjust (MAC③ vs MAC②)	$-20.2\%$		3.40 ns
+ 2D sub-word sharing (Jack vs MAC③)	$-11.1\%$		3.30 ns
Total Jack CSM vs. four fixed CSMs	$2.01\times$ smaller	$1.84\times$ less

Additional results:

• In a full 32×32 MAC array at 400 MHz, the Jack Unit yields $1.93\times$ smaller MAC-array area and $1.42\times$ smaller wire area ($1.60\times$ total accelerator area reduction).
• Energy efficiency improves by $1.32$–$5.41\times$ across INT, FP, and MX formats.
• For accuracy, omission of intermediate FP rounding yields sub-0.2% MAC error on a ConvNeXt-T convolution layer.
• Overhead: ≈15% extra control logic (exponent extractor and barrel shifter network) versus a single-mode CSM.
• The added pipeline registers in the bypass link increase on-chip buffer latency by ≈69%.

6. Trade-offs, Current Limitations, and Potential Extensions

The precision-scalable CSM’s principal trade-offs are modest increases in control and buffer latency for substantial area and power savings. The architecture currently supports up to 8-bit significands, with cascading available for 16-bit support without changing the core partial-product grouping. Notably, omission of intermediate FP rounding introduces minor quantization error, although it remains below 0.2% for representative ML workloads; subnormal handling can be added if required for large exponent differences.

Extension points under consideration include:

• Integration of on-the-fly stochastic rounding for ML training
• In-CSM denormal/subnormal number detection
• Support for mixed 8:4-bit subword operations (e.g., bf16×f8 in a single pass)

A plausible implication is that these directions may further enhance applicability to diverse ML and HPC workloads.

7. Significance in Contemporary Arithmetic and Accelerator Design

The precision-scalable CSM is a foundational building block in the Jack Unit MAC, supporting heterogeneous dataflows for AI accelerators. It achieves area and energy efficiencies through flexible datapath reconfiguration, dynamic operand packing, 2D sub-word sharing, and internal exponent-alignment—all without sacrificing computational throughput or requiring multiple fixed-precision blocks. The design serves as a model for future precision-scalable arithmetic logic, facilitating efficient inference and training in hardware architectures targeting emerging ML and scientific computing workloads (Noh et al., 7 Jul 2025).