Jack Unit MAC Architecture

• Jack Unit is a versatile MAC block that integrates support for integer, floating point, and microscaling data formats using reconfigurable logic.
• It employs a precision-scalable carry-save multiplier with 2D sub-word parallelism to dynamically adjust bit-widths and reduce hardware duplication.
• Empirical evaluations show that its innovative design lowers silicon cost and power consumption, enhancing energy efficiency in AI accelerator prototypes.

The Jack Unit is a multiply-accumulate (MAC) hardware block engineered for area- and energy-efficient processing across multiple numeric data formats, including integer (INT), floating point (FP), and microscaling (MX) representations. Unlike conventional MAC designs that dedicate separate computational resources to each data format, the Jack Unit introduces architectural re-use and precision scaling, enabling AI accelerators to achieve high throughput at significantly reduced silicon cost and energy consumption (Noh et al., 7 Jul 2025). This unit meets the increasing demand within AI and machine learning for hardware that is simultaneously flexible, scalable, and efficient across diverse model and quantization regimes.

1. Architectural Overview

The Jack Unit is built around a precision-scalable carry-save multiplier (CSM), advanced internal alignment of significands for floating point operations, and a novel 2D sub-word parallelism scheme. The design pivots on several core ideas:

• Single reconfigurable CSM: The traditional fixed-size CSM in FP multipliers is replaced by a composite array of smaller sub-multipliers. These can be fused or split dynamically to realize various bit-widths, enabling the same hardware to process 4×4, 8×8, or mixed-precision operations.
• Integrated significance alignment: Adjustment for exponent differences in FP addition is performed inside the CSM. This innovation merges what is normally a distinct pre-addition step (right shift for alignment) into the multiplication path, reducing latency and logic overhead.
• 2D sub-word parallelism: Sub-multipliers are grouped so that those aligned in the same “column” share barrel shifters for left and right shifts, drastically cutting the number of shifters and their associated logic.

The cumulative effect is that the Jack Unit is functionally a “jack-of-all-trades” MAC block, able to process workloads ranging from high-precision FP16/bfloat16 to ultra-low precision INT4, and specialized formats such as MXINT and MXFP where groups share a common exponent.

2. Precision-Scalable Carry-Save Multiplier (CSM)

The standard carry-save multiplier in a floating-point unit computes the significand product (mantissa × mantissa). In the Jack Unit, this logic is divided into an array of smaller, configurable sub-multipliers. For example:

• 8-bit mode: Four 4×4-bit sub-multipliers are fused to support an 8×8-bit operation.
• 4-bit mode: Each sub-multiplier operates independently, effectively supporting four parallel 4×4-bit multiplications.

This architecture avoids the costly duplication of MAC units for each data precision, enabling both resource and energy savings.

Calculation schematic:

// Pseudocode (Verilog style) for (mode in [8x8, 4x4, ...]) { if (mode == 8x8) Fuse 4 sub-multipliers; else if (mode == 4x4) Map each input to a sub-multiplier independently; } // Dynamic configuration is achieved via mode control signals.

3. In-Multiplier Significand Alignment for Floating Point

A distinctive feature of the Jack Unit is the integration of exponent-difference alignment into the multiplier datapath. Conventional FP adders require a dedicated shifting stage (to align significands when exponents differ), but here:

• The exponent difference $\Delta e = |E_x - E_y|$ is calculated.
• The significand of the operand with the smaller exponent is shifted immediately, before accumulation:

$M_{\text{aligned}} = M \gg \Delta e$

This shift is performed within the CSM, enabling accumulation with a fixed-precision integer adder tree (INT adder), which is simpler and lower-cost than a full-precision FP adder tree.

This approach also benefits MX formats, where all values in a block share the same exponent, and thus blockwise dot products can avoid frequent alignment logic.

4. 2D Sub-Word Parallelism

Rather than deploying separate shifters for each sub-multiplier, the Jack Unit leverages a 2D arrangement to share shift logic among sub-multipliers occupying the same “column.” For multiple CSMs operating in parallel, only one shifter per group is required.

If, for instance, a cluster of 4 sub-multipliers would traditionally require 4 separate shifters, the Jack Unit reduces this to a single shared shifter—cutting shift hardware cost by up to 75%. This optimization translates directly to area and dynamic power reduction, since shifters represent a significant source of hardware resource in variable-precision MAC arrays.

Block diagram illustration:

+----------+ +----------+ +----------+ +----------+ | SubMult1 | | SubMult2 | | SubMult3 | | SubMult4 | +----------+ +----------+ +----------+ +----------+ | | | | +---------------+---------------+---------------+ | Shared Barrel Shifter | Accumulator Tree

5. Data Format Flexibility and Supported Modes

The Jack Unit supports:

• Integer arithmetic: INT8, INT4, and other common AI quantization formats.
• Floating point: bfloat16, FP8, and other IEEE and non-IEEE FP modes.
• Microscaling (MX) arithmetic: For vectors $\vec{x} = [\hat{x}_1, ..., \hat{x}_n]$ with a shared exponent $e_x$,

$$\vec{x} \cdot \vec{y} = (\hat{\vec{x}} \cdot \hat{\vec{y}}) \cdot 2^{e_x + e_y}$$

This breadth enables DNN accelerators to tailor precision/energy trade-offs per layer or operation, switching seamlessly between high-precision and highly quantized regimes.

Table: Example configurations

Data Format	Bit-Width	Sub-Multiplier Use	Typical Use Case
INT8	8	Fused 4×4 blocks	AI inference layers
FP8	8	Fused 4×4 blocks	Mixed-precision training
INT4	4	Parallel 4×4 units	Ultra-low-power AI
MXINT	4–8	Shared exponent	Efficient quantized ops

6. Area, Power, and System-Level Impact

Empirical evaluation of Jack Units synthesized and laid out for silicon demonstrates:

• Area reduction: 1.17×–2.01× smaller footprint compared to baseline designs with dedicated MACs per data type.
• Power savings: 1.05×–1.84× lower consumption.
• System impact: In AI accelerator prototypes (tested on five benchmarks including ConvNeXt-T, BERT, GPT2-Small, etc.), the Jack-Unit-based system delivers 1.32×–5.41× higher energy efficiency.

These gains primarily derive from:

• Avoiding duplicated MAC resources.
• Reduced shift and alignment logic (due to 2D sub-word parallelism).
• Use of integer adders for accumulation wherever feasible.

7. Significance and Context

The Jack Unit's capacity to handle a spectrum of data precisions and formats within a compact hardware budget is particularly impactful for AI chips in edge devices, data centers, and emerging neuromorphic or reconfigurable architectures. By moving exponent alignment into the multiplier, adopting fine-grained parallelism, and supporting block-level microscaling, the design addresses pressing demands for adaptable compute substrates in heterogeneous workloads.

These architectural principles are extensible to broader contexts in digital signal processing and low-precision arithmetic, and may inform the design of future general-purpose AI accelerators.

---

In summary, the Jack Unit exemplifies a hardware approach that unifies support for diverse numeric representations through fine-grained reconfigurable logic, judicious datapath sharing, and novel integration of arithmetic alignment, resulting in quantifiable improvements in area and energy efficiency for AI acceleration platforms (Noh et al., 7 Jul 2025).