APTPU-Gen: Approximate TPU Generation Flow

• APTPU-Gen is an LLM-driven framework that generates approximate and exact TPUs using a unified, output-stationary systolic array design.
• It leverages retrieval-augmented generation with prompt templating and LoRA fine-tuning to assemble validated HDL modules from a curated RTL corpus.
• Experimental results show up to 96% power and 92% area reduction, demonstrating effective budget-constrained optimization in DNN inference and training.

Searching arXiv for the cited TPU-Gen paper and related terminology. APTPU-Gen is best understood as an informal label for the approximate TPU generation flow of TPU-Gen, an LLM-driven framework for automatic generation of custom tensor processing units focused on systolic array architectures. The formal term in the paper is TPU-Gen; “APTPU-Gen” is not used. In the reported formulation, approximate TPU generation is a first-class capability built on the APTPU lineage, reusing and generalizing an output-stationary systolic array template with Pre-Approximation Units (PAUs) and Approximate Processing Elements (APEs), and coupling that architecture to prompt templating, LoRA fine-tuning, Retrieval-Augmented Generation (RAG), and a verified PPA pipeline (Vungarala et al., 7 Mar 2025).

1. Terminology and lineage

The paper’s scope is broader than the shorthand “APTPU-Gen” suggests. TPU-Gen is presented as a unified framework that can generate both exact and approximate systolic-array TPUs. Approximate generation is not an auxiliary feature but a central design mode, supported by dedicated approximate MAC libraries, application-level accuracy constraints, and an automated search-and-validation loop. In practice, “APTPU-Gen” maps directly to TPU-Gen’s approximate TPU generation flow (Vungarala et al., 7 Mar 2025).

The approximate architecture inherits from the earlier APTPU: Approximate Computing Based Tensor Processing Unit, which introduced an output-stationary systolic array with PAUs and APEs. TPU-Gen reuses and generalizes that template. The paper therefore situates approximate TPU generation at the intersection of approximate arithmetic, systolic-array accelerator design, and LLM-assisted HDL synthesis. A common misconception is to treat APTPU-Gen as a standalone framework; the cited work instead formalizes a single generator whose approximate mode exposes the APTPU design lineage while sharing the same prompt, retrieval, verification, and PPA infrastructure as the exact flow (Vungarala et al., 7 Mar 2025).

2. Problem setting and design objectives

The motivating problem is the joint optimization of power, performance, and area (PPA) for DNN inference and training under workload-specific constraints. The paper frames approximate computing, especially approximate multiply-and-accumulate units in systolic arrays, as a way to trade bounded accuracy loss for improvements in energy, area, and latency. The challenge is not only the arithmetic approximation itself, but the combinatorial interaction among array size, dataflow, buffer sizing, MAC microarchitecture, and approximation hyper-parameters such as truncation depth, logarithmic-base behavior, or compressor configuration (Vungarala et al., 7 Mar 2025).

Three bottlenecks are emphasized. First, manual exploration of this joint space demands substantial hardware expertise and design time. Second, the domain lacks a large, public, structured corpus of TPU-like designs annotated with both PPA and error metrics. Third, free-form prompting of general LLMs is unreliable for HDL generation because it tends to produce missing modules, inconsistent interfaces, and un-compilable RTL. TPU-Gen addresses these bottlenecks by combining a curated hardware corpus with a guardrailed LLM pipeline in which the model emits top-level structure and headers, while RAG supplies dependent RTL modules from the indexed dataset (Vungarala et al., 7 Mar 2025).

3. Generation pipeline and approximate systolic architecture

The hardware generation pipeline begins from a high-level specification containing the target DNN or layers, PPA budgets, accuracy tolerance, array size $S$, data width (DW), weight width (WW), and an approximate versus exact choice. A Prompt Generator converts this request into a structured template aligned with the dataset schema. A fine-tuned code LLM, trained with LoRA using Adam lr=$1e^{-5}$, then generates the top-level Verilog and the associated headers. The RAG module uses those headers to retrieve all dependent modules—such as controllers, FIFOs, PAUs, APEs, multiplier and adder variants, buffers, and interconnect glue—from the TPU-Gen corpus, forming a complete and compilable project. The resulting RTL is passed through Yosys for logic synthesis and OpenROAD for floorplan, placement, route, and PPA extraction; functional simulation marks designs Valid/Invalid, and invalid outputs trigger iterative refinement with error feedback (Vungarala et al., 7 Mar 2025).

At the microarchitectural level, the framework targets an output-stationary (OS) systolic array. The declared components are weight memory, IFMap memory, controller, FIFOs, PAUs, and APEs. IFMaps and weights stream through the array, while partial sums remain inside the processing elements to minimize writes. PAUs are shared across rows or columns and dynamically truncate or lower operand precision before MAC execution. APEs implement MACs using a chosen approximate multiplier and approximate adder, inheriting parameters from the library. The interconnect is a nearest-neighbor systolic wavefront in which inputs propagate diagonally and PEs exchange data every cycle. Convolutional layers are lowered to matrix multiplication, for example by im2col, or scheduled directly in OS tiles; fully connected and GEMM layers map naturally to the same template (Vungarala et al., 7 Mar 2025).

The paper’s RAG strategy is specifically aimed at hallucination control. Rather than asking the LLM to synthesize an entire nontrivial RTL hierarchy in a single pass, TPU-Gen constrains the model to top-level composition and then uses the emitted headers as validated retrieval keys. This suggests that the architecture treats HDL generation less as pure sequence modeling than as schema-constrained project assembly, which is particularly important for approximate TPUs because module compatibility depends on parameter-consistent arithmetic blocks, PAU settings, and interface signatures.

4. Dataset, design space, and operator libraries

The TPU-Gen corpus is the central enabling artifact for approximate generation. It contains a parameterized RTL library of 13 approximate adders and 12 approximate multipliers, together with multi-level design records, synthesized PPA metrics, and error metrics. The paper states the following coverage (Vungarala et al., 7 Mar 2025):

Dimension	Stated coverage	Notes
Approximate operator library	13 approximate adders; 12 approximate multipliers	Parameterized RTL
Precision coverage	DW in $\{8,16,32\}$; WW in $[3,32]$ bits	Supports quantized to higher precision
Array sizes	$S \in \{4,8,16,32,64,128,192,256\}$	8 systolic sizes
Design volume	29,952 variations per $S$; 239,616 total possible; 25,000 current checkpoint	Includes PPA and error metrics

The dataset is organized at two levels. One level stores top-level Verilog bundles for whole TPU instances, which are useful for direct RAG composition. The second level stores an MG-Verilog-style module dataset with metadata and labels adapted from MG-Verilog for HDL generation. The metadata includes $$(S, DW, WW, \text{dataflow}=OS, \text{adder\_type}, \text{multiplier\_type}, \text{hyper-parameters}, \text{PAU truncation settings}, \text{interface signatures})$$, together with PPA and error metrics. Pyverilog parsers extract module I/O and hierarchy for indexing and retrieval (Vungarala et al., 7 Mar 2025).

The approximate MAC catalog spans multiple arithmetic families. The multiplier side includes Mitchell/logarithmic variants, DRALM, ALM variants such as LOA/MAA3/SOA accumulation, Broken-Array Multiplier (BAM) with VBL control, Alphabet Set Multiplier (ASM) with Nibble_Width, RoBA, and truncated Booth, Wallace, or array multipliers with approximate compressors. The adder side includes Lower-Part OR Adder (LOA), Optimized LOA, MAA3, SOA, carry speculative adders, reverse carry propagate adders, and other LSB-approximate designs. The principal knobs include total bit-width $W$, imprecise part size $M$, VBL, MULT_DW, ROUND_WIDTH, and PAU truncation settings (Vungarala et al., 7 Mar 2025).

5. Optimization criteria, workload mapping, and quantitative models

Approximate generation is described as a constrained search over architectural and arithmetic parameters. The search space includes $S$, DW/WW, OS dataflow, on-chip buffer depths, PAU truncation parameters, and the adder/multiplier class and hyper-parameters used inside the APEs. For each candidate, TPU-Gen generates the top level, populates the design by RAG, synthesizes it, computes PPA and error metrics, and checks whether the design satisfies constraints of the form power $1e^{-5}$0, area $1e^{-5}$1, latency $1e^{-5}$2, and accuracy loss $1e^{-5}$3. Pareto-feasible candidates are then retained (Vungarala et al., 7 Mar 2025).

The framework uses explicit throughput, latency, and energy models. For an $1e^{-5}$4 systolic array, the number of MACs is

$1e^{-5}$5

Throughput is modeled as

$1e^{-5}$6

where $1e^{-5}$7 is the post-synthesis clock frequency and $1e^{-5}$8 is utilization. For a GEMM tile $1e^{-5}$9 on an OS array, per-tile latency is approximated by

$\{8,16,32\}$0

ignoring DMA, and energy per MAC is

$\{8,16,32\}$1

These expressions are used together with synthesis and place-and-route results to rank candidates (Vungarala et al., 7 Mar 2025).

Error and quality are assessed at both the arithmetic-cell and array levels. The reported operator-level metrics are

$\{8,16,32\}$2

$\{8,16,32\}$3

$\{8,16,32\}$4

$\{8,16,32\}$5

and

$\{8,16,32\}$6

For an $\{8,16,32\}$7 tile computing $\{8,16,32\}$8 with outputs $\{8,16,32\}$9, tile-level aggregation includes

$[3,32]$0

with analogous aggregation for MAE, ER, and SNR. The paper states that PAU truncation and MAC errors are measured by automated simulation across operand distributions relevant to the selected DW/WW. At the DNN level, TPU-Gen validates designs on MNIST, CIFAR-10, SVHN, and CIFAR-100 using LeNet, VGG16, ResNet18, and ResNet56, but it does not publish explicit network-accuracy numbers; instead, the pipeline checks whether the design satisfies an application-specified tolerance $[3,32]$1 (Vungarala et al., 7 Mar 2025).

A concrete selection example is also given. For DW=8, WW=4, and $[3,32]$2, the flow may choose DRALM with MULT_DW=2, LOA with $[3,32]$3, and a PAU that truncates IFMaps by 1 LSB. The generated design is kept if it satisfies $[3,32]$4 mW, $[3,32]$5 mm$[3,32]$6, and $[3,32]$7 ms with acceptable array-level MSE/SNR; otherwise the flow adjusts MULT_DW, $[3,32]$8, or PAU truncation and retries. This illustrates the intended role of APTPU-Gen as a budget-constrained approximate hardware synthesis loop rather than a fixed approximate architecture (Vungarala et al., 7 Mar 2025).

6. Experimental results, baselines, and limitations

The experimental flow uses 45 nm FreePDK45, with Yosys for logic synthesis and OpenROAD for place-and-route and PPA extraction. Fine-tuning was performed on multiple 7–13B code LLMs using 5,000 Verilog headers from version 1 of the TPU-Gen dataset, with an 80:20 train/test split and 4×A100 (80 GB) GPUs. Benchmarks included LeNet (MNIST), VGG16 (CIFAR-10), ResNet18 (SVHN), and ResNet56 (CIFAR-100), and evaluation covered module generation, integration, RAG ablation, PPA against manually optimized designs, and representative RTL-to-GDSII results (Vungarala et al., 7 Mar 2025).

The paper reports several quantitative outcomes. Under prompting alone, commercial LLMs achieved approximately 72% pass@1 for module generation and approximately 50% for integration, while approximate TPUs required more human guidance without fine-tuning. With fine-tuning + prompt templating, single-shot success increased markedly; for example, CodeLlama-7B-hf achieved 27/30 passes with a structured template versus 1/30 with free-form human prompts, with similar trends for Mistral-7B, StarCoder2-7B, and CodeQwen1.5-7B. With RAG, complete-project pass rates across 30 designs were 100% for CodeLlama-7B-hf, 100% for Mistral-7B, 98% for StarCoder2-7B, and 95% for CodeQwen1.5-7B; without RAG, all models had 0% pass because of token limits, missing modules, and hallucinated variables. For approximate TPU generation, TPU-Gen achieved, on average, 92% area reduction and 96% power reduction from manual optimization reference values when searching for budget-feasible designs across the evaluated networks (Vungarala et al., 7 Mar 2025).

The principal baseline is a manual design flow in which expert engineers iteratively tune TPU templates under a one-day effort cap per network. The paper states that TPU-Gen consistently meets or betters those reference PPA budgets with less violation. Representative GDSII layouts are shown for an exact TPU, a prompting-only TPU, and a TPU-Gen approximate TPU. To the authors’ knowledge, this constitutes the first LLM-driven framework for automatically generating both exact and approximate TPU designs with an open dataset, RAG-based hallucination mitigation, and a verified PPA pipeline focused on systolic arrays (Vungarala et al., 7 Mar 2025).

The limitations are explicit. The release emphasizes systolic arrays with OS dataflow rather than non-systolic architectures or weight/input-stationary dataflows. The current dataset checkpoint of 25,000 designs is substantially smaller than the 239,616 possible combinations. Correctness depends heavily on corpus fidelity, so out-of-distribution requests may require manual curation. Validation is reported only at 45 nm, leaving portability to later technology nodes as future work. Finally, although the flow enforces application-level accuracy constraints, the paper does not publish detailed per-network accuracy deltas. A common overstatement is therefore to interpret APTPU-Gen as a fully general approximate-accelerator generator with complete accuracy characterization; the reported system is narrower, corpus-dependent, and centered on OS systolic arrays, but within that scope it provides an end-to-end approximate TPU generation pipeline grounded in explicit operator libraries, retrieval-backed HDL assembly, and signoff-oriented PPA evaluation (Vungarala et al., 7 Mar 2025).