XtraMAC: Adaptive Mixed-Precision MAC

• XtraMAC is a mixed-precision MAC architecture that unifies INT, FP, and mixed operations for LLM inference on FPGA.
• It uses a fixed four-stage pipeline with II=1 and dynamic operand packing to optimize DSP resource sharing and maintain constant latency.
• The design employs unified mantissa-product decomposition and decoupled INT/FP accumulation, yielding significant resource and energy efficiency improvements.

XtraMAC is a multiply–accumulate (MAC) architecture for mixed-precision LLM inference on FPGA that unifies integer, floating-point, and mixed-precision operations within a single, datatype-adaptive microarchitecture. It is designed for workloads in which MAC formats such as $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$, $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$, and $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$ can switch at runtime, including at cycle granularity. The design targets efficient DSP resource sharing, constant latency, and initiation interval of one across all supported datatypes, and is evaluated on an AMD Xilinx U55c FPGA (Yu et al., 7 May 2026).

1. Problem setting in mixed-precision LLM inference

Modern LLMs use mixed-precision quantization—e.g. INT4 for compute-heavy layers and BF16 or FP formats for numerically sensitive layers—to reduce memory and energy. Within one forward pass, MACs may appear in formats such as $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$, $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$, and $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$, and these formats can switch at runtime, as in projection layers versus attention layers. FPGA-based LLM accelerators therefore must support mixed-precision MAC with heterogeneous operand widths in $A\times B + C$ and runtime datatype switching between formats such as INT8, BF16, and FP8 at cycle granularity (Yu et al., 7 May 2026).

The architectural motivation is framed by three limitations in existing FPGA MAC designs. Up-casting-based designs promote all operands to the widest supported format, leaving most DSP bit-capacity unused; the paper gives the example that executing INT4$\times$BF16 on a BF16 MAC uses only approximately $32\%$ of the 45-bit multiplier. Spatial replication duplicates a separate MAC datapath per format and uses a multiplexer to select the active one, so inactive pipelines waste area and power, with effective DSP utilization of approximately $26.7\%$. Temporal sharing decomposes high-precision MACs into multiple cycles on a low-precision unit; the cited example is TATAA, which breaks BF16 into four INT8 micro-ops, yielding DSP utilization for BF16 of approximately $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$0. The stated conclusion is that these approaches do not match the fine-grained bit-width patterns of mixed-precision LLM workloads, resulting in under-utilized DSPs and limited parallelism.

A common misconception is that runtime datatype switching necessarily implies hardware reconfiguration. In XtraMAC, switching is purely via multiplexers, with all per-datatype mapping, packing, and reconstruction units existing in parallel. No hardware reconfiguration is required.

2. Four-stage pipeline and datapath organization

XtraMAC adopts a fixed four-stage pipeline with initiation interval $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$1 and deterministic latency, with default latency of 4 cycles. A per-cycle datatype-select signal, described as $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$2-way, is registered at entry and carried through all stages (Yu et al., 7 May 2026).

The four stages are:

1. Stage 1: Operand interpretation and DSP-port packing.
2. Stage 2: DSP multiplication and per-lane post-compute, including exponent, sign, and normalization.
3. Stage 3: Decoupled INT and FP accumulation.
4. Stage 4: Lane result concatenation and output select.

All stages are separated by registers. The DSP48E2 internal registers are bypassed so that multiplication is combinational between the Stage 1 and Stage 2 registers. The datatype-select signal and any operand $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$3 used for accumulation are delayed through matched register slices to align control and data. By default, each stage is one cycle, although intra-stage registers may be added, for example in the FP adder, to meet timing without changing $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$4 or overall pipeline length.

The separation of INT and FP accumulation is motivated by the different cost profiles of their adders. The paper states that floating-point addition incurs super-linear LUT cost in mantissa width, which justifies separate INT and FP adder paths. This suggests that the pipeline is not only datatype-adaptive at the multiplier level, but also explicitly structured around asymmetric post-multiply costs.

3. Unified arithmetic decomposition

A central property of XtraMAC is that, regardless of whether the operation is $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$5, $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$6, or $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$7, the core multiply reduces to an integer mantissa product performed by the DSP slice, while sign and exponent are handled outside the DSP (Yu et al., 7 May 2026).

For floating-point multiplication, the decomposition is stated as

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$8

and therefore

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$9

For integer$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$0floating-point multiplication, the integer operand is treated as $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$1 with exponent equal to $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$2, yielding

$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$3

In both cases, the DSP computes only the mantissa product $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$4 or $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$5. After multiplication, a leading-zero count $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$6 normalizes the product mantissa and updates the exponent according to

$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$7

or $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$8 for $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$9.

This decomposition is the arithmetic basis for datatype unification. The paper’s formulation implies that datatype heterogeneity is absorbed primarily by operand interpretation, sign logic, exponent bookkeeping, and normalization, while the DSP multiplier remains a shared integer-mantissa engine.

4. Dynamic operand packing and resource models

XtraMAC packs multiple low-precision MAC lanes bit-precisely into the DSP inputs $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$0 and $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$1 (Yu et al., 7 May 2026). The packing equations are

$$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$2

The DSP computes one wide multiplication,

$$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$3

and per-lane products are recovered by shift-and-mask:

$$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$4

where $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$5, i.e. lane bit-width plus one-bit guard.

The DSP utilization model is given in terms of effective multiplicand widths $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$6 and $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$7, with $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$8 for the DSP48E2 multiplier, corresponding to a 27-bit $$\mathrm{INT}\times \mathrm{INT}\rightarrow \mathrm{INT}$$9 port and 18-bit $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$0 port:

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$1

Packing $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$2 lanes increases aggregate utilization. The maximum lane parallelism per DSP is bounded by

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$3

The adder resource models distinguish integer and floating-point cost growth. Integer adder LUT cost is modeled as

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$4

while floating-point shifter cost for alignment and normalization is modeled as

$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$5

Per-operation resource usage is estimated by dividing total LUT, FF, and DSP by lane parallelism. The paper provides the example that an INT4$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$6BF16$$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$7BF16 MAC may pack $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$8 lanes, so one DSP slice delivers two MACs, with per-MAC usage of DSP $$\mathrm{INT}\times \mathrm{FP}\rightarrow \mathrm{FP}$$9, LUT $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$0, and FF $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$1.

5. Implementation and measured performance

The implementation target is the AMD Xilinx Alveo U55c FPGA (XCKU15P), using the DSP48E2 primitive with a 27$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$218 multiplier. The platform description includes 14.2 Mb UltraRAM, 1.6 Mb BRAM, and 32 HBM channels at 16 GB/s each. The tool flow uses Vivado 2022.2 and Vitis 2022.2. The stated clock target is 300 MHz, with 400+ MHz achievable for small instantiations. The repository is open-sourced at https://github.com/Xtra-Computing/XtraMAC, with hw/, scripts/, kernel/, and sw/ directories covering RTL, synthesis and place-and-route scripts, an HLS GEMV integration example, and simulation drivers for mixed-precision test vectors (Yu et al., 7 May 2026).

Against the AMD Xilinx Floating-Point Operator IP, with an open-source INT$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$3FP converter, the reported reductions are LUT $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$4–$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$5, FF $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$6–$$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$7, and DSP $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$8, with compute density improvement of $$\mathrm{FP}\times \mathrm{FP}\rightarrow \mathrm{FP}$$9–$A\times B + C$0. Against spatial replication using Vendor IP plus INT8 and against temporal sharing with TATAA, the per-BF16 MAC reductions are as follows:

Baseline	Resource change per BF16 MAC	DSP change
Spatial replication	LUT down 35.5%, FF down 58.7%	DSP down 75.0%
TATAA	LUT down 59.7%, FF down 72.5%	DSP down 93.8%

Measured per-MAC resource usage is normalized by lane parallelism. The overall reported result is compute density improvement of $A\times B + C$1–$A\times B + C$2, with LUT reduced by $A\times B + C$3–$A\times B + C$4, FF by $A\times B + C$5–$A\times B + C$6, and DSP by $A\times B + C$7.

For mixed-precision GEMV on the U55c with 30 active HBM channels of 512-bit width and 1920 cascaded XtraMAC instances, the sustained clock is 250–300 MHz. The reported GEMV latency for $A\times B + C$8 is 24.6 ms versus H100 CUTLASS at 29.4 ms, corresponding to $A\times B + C$9 speedup. For $\times$0, the latency is 74.3 ms versus H100 at 87.9 ms, also corresponding to $\times$1 speedup.

The energy-efficiency measurements report FPGA power of approximately 85 W and GPU power of approximately 135 W. Energy per GEMV is approximately 2.09 J for the FPGA versus approximately 3.97 J for the GPU, corresponding to $\times$2 higher efficiency.

6. End-to-end implications, scalability, and limitations

Using a spatial-LLM analytical model on Alveo V80 with 10,848 DSPs, 810 GB/s HBM, and 18 Mb BRAM, XtraMAC is reported to show minimal speedup at small batch size 1 because the workload is memory bound. At batch sizes 8–32, the workload becomes compute bound, and the simulated end-to-end speedup is $\times$3–$\times$4 across Qwen-3-8B-AWQ, Llama-3.1-8B-W8A8, Qwen3-8B-FP8, and GPT-oss-20B (Yu et al., 7 May 2026).

The paper states that the DSP-centric packing principle directly applies to devices with wider DSPs, such as 48$\times$548 multipliers, and multiple DSP clusters. Additional lanes per DSP would yield further compute density gains. Support for emerging low-bitwidth formats including FP8 variants (E4M3, E5M2), FP4, and MIX4 can be added by computing the appropriate shift stride $\times$6 and decoding fields in Stage 1; the four-stage pipeline and lane packing are described as generalizing to arbitrary sign, exponent, and mantissa widths.

The design also makes explicit accuracy–performance trade-offs. Lane packing increases DSP utilization at the cost of requiring exact bit arena partitioning, with no cross-lane carries. Users must choose guard-bit margin $\times$7 to absorb carry overflows. Flush-to-zero and DAZ semantics simplify handling subnormals but may affect numerical edge cases. The paper notes that one could extend the exception-flag pipeline to support IEEE-754 flush and denorm semantics. This suggests that XtraMAC’s efficiency derives partly from a deliberate restriction of certain floating-point edge behaviors rather than from multiplier packing alone.

In summary, XtraMAC is positioned as a datatype-adaptive FPGA MAC architecture whose defining features are unified mantissa-product decomposition, dynamic operand packing, decoupled INT and FP accumulation, and fixed four-stage execution with $\times$8 and constant latency. Within the reported evaluation, these properties are associated with higher compute density, lower per-operation resource cost, and improved throughput and energy efficiency for representative mixed-precision LLM workloads (Yu et al., 7 May 2026).