---
title: 'GAP9: Ultra-Low-Power Edge AI SoC'
url: https://www.emergentmind.com/topics/gap9
type: topic
---

# GAP9: Ultra-Low-Power Edge AI SoC

GAP9 is an ultra-low-power PULP-based RISC-V edge System-on-Chip from GreenWaves Technologies that appears in the literature as an MCU-class platform for near-sensor DSP, neural inference, and increasingly on-device training. Across work on biosignals, embedded vision, nano-drones, visual odometry, and compiler design, it is described as a heterogeneous processor combining a controller core, a multicore compute cluster, a software-managed memory hierarchy, and, in many reports, the NE16 neural accelerator. This combination is used to support deployments ranging from quantized CNNs and tiny transformers to continual learning and full-network fine-tuning under wearable and battery-powered constraints [2307.01619][2410.08855][2603.08725].

## 1. Architectural identity and reported configurations

GAP9 is generally positioned as an ultra-low-power MCU/SoC for edge AI rather than as a host-class embedded processor. A 2026 review classifies it as a **“RISC-V Manycore”** device with **int8** precision, **1.51E-01 TOPS**, **0.0640 W**, and **2.36 TOPS/W**, and describes it as a platform with **configurable precision** and **1.6 MB of L2 SRAM** used mainly in vision and nano-drone applications [2603.08725]. Other papers instead describe it operationally as a **PULP** processor with a **Fabric Controller** domain and a compute cluster, emphasizing that the cluster is the main target for ML and DSP kernels [2402.10748][2407.13706].

The core-count description varies slightly by source, but the pattern is consistent. Several papers describe GAP9 as a **10-core** SoC with a controller plus a compute cluster [2307.01619][2407.13706]. The compute side is reported either as **“9 RISC-V cores”** [2605.15423], as **“one orchestration core + eight worker cores”** [2307.05999], or as a **9-core RISC-V compute cluster** accompanied by a **single-core RISC-V controller** [2407.13706]. This suggests that different authors report either the full cluster membership or the worker-core view used by parallel kernels.

The memory hierarchy is one of the most stable identifying features. Across papers, GAP9 is repeatedly described with **128 kB** shared **L1** cluster memory and **1.5 MB** or **1.6 MB** **L2** on-chip SRAM, plus **2 MB** of on-chip non-volatile memory or flash-like storage [2402.10748][2407.13706]. A number of system papers also attach large external memories, such as **32 MB external RAM** and **64 MB external Flash** for embedded vision nodes [2605.15423], or **64 MiB PSRAM** and **256 MiB NOR flash** in smart glasses [2606.07431]. In practice, however, many deployments are still shaped primarily by the on-chip L1/L2 limits.

The arithmetic model is heterogeneous. GAP9 is described as supporting **transprecision floating point** including **IEEE 32-bit**, **IEEE 16-bit**, and **bfloat16**, while also providing aggressive low-precision execution for ML [2407.13706]. Several papers additionally highlight **four shared FPUs** on the cluster [2605.15423][2509.10021]. When the NE16 accelerator is discussed explicitly, it is framed as a CNN-oriented engine for quantized workloads, with some papers stating support for convolutions and fully connected layers at the architectural level, even though deployed backend libraries may impose narrower practical support [2410.08855].

## 2. Memory hierarchy, data movement, and software toolchains

GAP9 is typically programmed as a scratchpad-centric system rather than as a cache-centric one. The common execution model stages weights, activations, and temporary buffers from **L2** into **L1**, runs compute from L1-resident tiles, and writes results back through DMA-managed transfers. MATCH, a TVM-based compiler framework, makes this execution model explicit by modeling GAP9 with asynchronous DMA and double buffering, using the latency relation  
$$
L = \max(L_{ops}, L_{mem,1,2})
$$  
to capture overlap between computation and L2–L1 transfers [2410.08855].

This memory model is visible in application papers as well. The tiny ECG transformer for arrhythmia detection was intentionally downsized so the 8-bit model footprint of **49 kB** and its working set target the **128 kB L1** cluster memory rather than only the larger L2 [2402.10748]. In pest detection, MobileNetV3-SSDLite was scheduled layer by layer with a **1.2 MB L2 buffer**, a **115.6 kB L1 working buffer**, **1.3 MB external RAM** for activations, and **3.44 MB external FLASH** for parameters, making the deployment a concrete example of explicit multi-level tiling on GAP9 [2408.15911]. In downfacing VIO, the fabric controller uses DMA to move camera and IMU data into L1 so that the compute cluster can process feature tracking and filtering locally, while SuperPoint becomes memory-bound because it does not fit fully in L2 and spills to L3 [2509.10021].

The software toolchains around GAP9 are similarly varied but all reflect the same memory-conscious execution style. Quantized inference papers frequently use **QuantLab** and **DORY** [2402.10748][2309.07135]. Vision deployments targeting NE16 commonly use **NNTool** or **GAPflow**, with GAPflow generating two binaries when separate full-resolution and low-resolution inference graphs are required [2605.15423][2408.15911]. On-device training papers use **AutoTiler**, **NNTOOL**, and the **GAP SDK** for inference-oriented components, then add custom training kernels or **PULP-TrainLib** for backpropagation and optimizer updates [2407.03644][2409.10654]. Full-network fine-tuning extends the stack further: BioTrain uses **PyTorch**, an enhanced **ONNX Runtime training API**, **Deeploy**, **PULPTrainLib**, and **PULP-NN** to compile complete forward, backward, and update graphs to bare-metal C for GAP9 [2604.13359].

Compiler research has treated GAP9 as a canonical heterogeneous target. MATCH defines GAP9 as a target with two execution modules, **Cluster** and **NE16**, and reports **2.15× lower average latency** than **DORY** on the MLPerf Tiny suite by assigning layers across the accelerator and the 8-core cluster rather than treating the SoC as a single execution resource [2410.08855]. This suggests that GAP9’s practical behavior depends as much on orchestration quality as on nominal hardware capability.

## 3. Numerical formats and execution modes

The literature on GAP9 is notable for spanning almost the full range of edge-AI numerical formats. At the architectural level, BioGAP describes GAP9 as supporting **multi-precision (from FP to aggressively quantized integer)**, and NE16 is presented there as a flexible accelerator supporting **16- or 8-bit features** and **weights from 8 to 2 bits** with **asymmetric, scaled quantization** [2307.01619]. Application papers then instantiate this flexibility in very different ways.

Many inference deployments are **int8** end to end. The arrhythmia transformer explicitly performs **8-bit integer inference throughout**, with all target computations restricted to integer type and the RR interval pair normalized to \([-2,2]\) to ease quantization before concatenation [2402.10748]. TinyissimoYOLO on GAP9 cluster execution uses **8-bit weights** and **8-bit activations** produced by **QuantLab QAT using TQT**, with integer-only inference and no reported accuracy drop relative to full precision for the cluster path [2307.05999]. The seizure detector EpiDeNet is likewise quantized to **INT-8** and deployed with **Quantlab + DORY**, with only approximately **\(\pm 0.1\%\)** variation in sensitivity and specificity relative to the floating-point model [2309.07135].

Other workloads deliberately stay in floating point. MR2-ByteTrack deploys CNN and Transformer-based video object detection on GAP9 using **FP16** for both weights and activations, explicitly describing the deployment as **lossless** rather than int8-quantized [2605.15423]. EEGNet-based on-device learning keeps the entire model in float, reporting that **no quantization or pruning is used** and that parameters remain **float type** for both online and offline training/testing [2409.00083]. Human-activity-recognition transfer learning on GAP9 uses **bfloat16** during on-device adaptation, while the STM32 comparison baseline uses **float32** [2407.03644]. BioTrain goes further and demonstrates full-network **FP32** backpropagation on GAP9 despite the platform’s 1.5 MB L2 limit [2604.13359].

A third class of deployments uses mixed precision by separating frozen inference from trainable adaptation. In wearable BMI and HAR personalization, the common pattern is an **int8 frozen backbone** plus a **fp32** or floating-point trainable head, so that heavy convolutional feature extraction fits the memory and energy envelope while the mutable classifier remains numerically convenient for updates [2409.10654][2409.00083]. This suggests that GAP9 is often used less as a single-format target than as a platform for workload-specific precision partitioning.

## 4. Demonstrated application domains

The range of GAP9 deployments is unusually broad for an MCU-class edge processor. The literature includes biosignal classification, EEG and ECG analysis, object detection, video object detection, pest monitoring, nano-drone perception, visual-inertial odometry, robotic grasp detection, and system-identification-based structural diagnostics [2402.10748][2309.07135][2408.15911][2509.10021][2507.13970][2504.04884].

| Domain | Configuration | Representative result |
|---|---|---|
| Arrhythmia classification | 8-bit tiny transformer on GAP9 | **4.28 ms**, **0.09 mJ**, **98.97%** clean accuracy, **98.36%** worst-case noisy accuracy [2402.10748] |
| Quantized object detection | TinyissimoYOLO on NE16 | **2.12 ms** and **149 \(\mu J\)** for `TY:3-3-88`; **3.46 ms** and **245 \(\mu J\)** for `TY:10-3-112` [2307.05999] |
| Transformer-based VOD | EffViT-Det MR2-ByteTrack \(P=1\) | **3.2 fps**, **17.1 ± 2.6 mJ/frame**, approximately **48.7 mAP** [2605.15423] |
| Downfacing VIO | ORB-based tracker on GAP9 | **118.5 FPS** max for ORB + template pipeline; PX4FLOW reaches **1216.3 FPS** [2509.10021] |
| End-to-end SysId damage detection | PARSY-VDD on GAP9 | **751 \(\mu s\)** at **370 MHz, 0.8 V** and **37 \(\mu J\)** at **240 MHz, 0.65 V** [2504.04884] |

In biosignal processing, GAP9 has been used both as a direct inference target and as the compute core of complete wearable systems. BioGAP, a **6 g** wearable platform built around GAP9, performs **eight FP FFTs of size 1024 in less than 0.425 ms**, achieving **16.7 Mflops/s/mW** on the FFT task and reducing wireless bandwidth by **97%** in its SSVEP BCI mode [2307.01619]. In seizure detection, EpiDeNet processes a **\(4 \times 2048\)** EEG window in **2.84 ms** at **17.89 mW**, for **0.051 mJ/inference** and **40.61 GMAC/s/W**, outperforming the best ARM Cortex-based solutions by approximately **160×** in energy efficiency [2309.07135].

In embedded vision, GAP9 has been used in both classical and learned pipelines. A GAP9-based codling-moth detector executes MobileNetV3-SSDLite on a **\(320\times240\)** image in **147 ms**, with **4.85 mJ** processing energy, making the CNN approximately energy-competitive with a tiled Viola–Jones detector on the same platform [2408.15911]. A separate pest-monitoring paper deploys **SSDLite-MobileNetV3** for three-class insect detection at **6.8 frame/s** within **33–34 mW**, with **mAP 0.79** and an **int8 + NE16** latency of **147 ms** [2407.00815]. GAP9Shield, a Crazyflie-compatible module, is built around GAP9 to support perception tasks such as **YOLO** object detection at **17 ms**, **MCL** at **23 mW** average, and **NanoSLAM** at **87.9 mW** and **< 250 ms** latency [2407.13706].

The platform has also hosted workloads that are structurally atypical for MCU-class inference. A robotic grasping paper partitions Heatmap-Guided Grasp Detection into **ResNet-MCU**, **AnchorNet-MCU**, **PointNet-MCU**, and **LocalNet-MCU** stages, obtaining a total **740.47 ± 0.0046 ms** inference time on GAP9, with **PointNet-MCU** accounting for about **90%** of runtime because batch-size-64 execution could not be realized in NNTool and had to be replaced by **64 passes through the network** [2507.13970]. This suggests that GAP9 can host nontrivial 6-DoF perception graphs, but with strong dependence on toolchain and operator structure.

## 5. On-device learning and adaptation

A recurrent theme in the GAP9 literature is that the device is not limited to static inference. Multiple papers use it for post-deployment adaptation under cross-subject or cross-session shift. The simplest and most common strategy is **classifier-only** adaptation. In human activity recognition, the backbone is frozen and only the final dense layer is updated on GAP9, with new labeled samples processed one at a time; this improves accuracy by **3.73%** on RecGym, **17.38%** on QVAR-Gesture, and **3.70%** on Ultrasound-Gesture in one paper [2508.15413], and by closely similar margins in an earlier ODTL implementation comparing GAP9 against STM32F756ZG [2407.03644].

The hardware-level costs of such shallow adaptation can be extremely small. EEGNet-based motor-imagery personalization reports **14.9 ms** and **0.76 mJ** per inference for the preferred **19-channel / 2-second** configuration, but only **20 \(\mu s\)** and **0.83 \(\mu J\)** per single online classifier update, with a total model footprint of **15.6 KByte** [2409.00083]. Wearable BMI continual learning with MI-BMInet updates only the final fully connected layer in **21.5–21.6 ms**, reports **0.45 mJ per inference** in the abstract, and estimates about **25 h** battery life on a **100 mAh, 3.7 V** battery when fine-tuning is performed once per second [2409.10654].

More recent work pushes beyond head-only adaptation. BioTrain argues that existing MCU-based wearable platforms support only shallow adaptation because conventional full-network backpropagation exceeds memory limits, then shows that GAP9 can support **full-network fine-tuning** for compact biosignal CNNs when training is reformulated around **gradient accumulation**, **Group Normalization**, static allocation, and compiler-managed tiling [2604.13359]. For the EEG model, conventional full fine-tuning would require **5.36 MB**, whereas the GAP9-feasible Edge-FT path reduces peak L2 memory to **0.67 MB**; for EOG, the reduction is from **2.24 MB** to **0.28 MB** [2604.13359]. System-level measurements report **17 samples/s** for EEG and **85 samples/s** for EOG at **below 50 mW**, making GAP9 one of the first MCU-class platforms in this literature to demonstrate on-chip full-network backpropagation rather than only last-layer updates [2604.13359].

This progression from classifier-only updates to full-network fine-tuning also clarifies a common misconception. GAP9 is often introduced as an inference-oriented edge SoC, but the training literature shows that it can host several levels of adaptation: online dense-layer updates [2407.03644], continual-learning heads with replay or distillation variants [2409.10654], and compiler-generated full-network FP32 training graphs [2604.13359]. A plausible implication is that the practical boundary on GAP9 is no longer “training versus inference” in the abstract, but the specific memory behavior of the chosen training scheme.

## 6. Performance regime, compiler implications, and limitations

The strongest performance results on GAP9 usually come from co-design rather than from raw accelerator use alone. TinyissimoYOLO shows this clearly: cluster-only execution already delivers **6.14×** and **6.77×** speedups over single-core GAP9 for two model variants, while NE16 raises computational efficiency to **41.22–42.84 MAC/cycle** and reduces latency to **2.12 ms** or **3.46 ms** depending on the model [2307.05999]. MATCH reaches its advantage over DORY not by replacing cluster execution with accelerator execution wholesale, but by mixing **Cluster** and **NE16** on a per-pattern basis and exploiting asynchronous DMA [2410.08855]. PARSY-VDD similarly derives its **90×** execution improvement over prior GAP9 SysId work from full-pipeline parallelization, not from a new accelerator block [2504.04884].

At the same time, several papers show that GAP9’s bottlenecks remain strongly memory- and toolchain-related. SuperPoint on GAP9 becomes **memory-bound** because the network does not fit into L2 and spills to L3, reducing throughput to **11.9 FPS** in the downfacing VIO pipeline and producing substantially worse odometry than ORB or PX4FLOW [2509.10021]. In grasp detection, deployment feasibility depends on four-way model partitioning, input-size reduction from **\(640 \times 360\)** to **\(320 \times 160\)**, and quantization; even then, total inference remains **740.47 ms** [2507.13970]. In early seizure-detection work, DORY-generated GAP8 code had to be **manually optimized** to ensure compatibility with GAP9 [2309.07135]. These cases suggest that GAP9’s limiting resource is frequently not compute density in isolation, but the interaction between scratchpad size, operator support, and generated dataflow.

The meaning of “real time” on GAP9 also depends on domain. The MR2-ByteTrack paper explicitly describes **3.2 frame/s** Transformer-based video object detection on GAP9 as the first **real-time Transformer-based VOD on an MCU-class embedded vision node**, using “real-time” in the embedded, causal, online sense rather than in the conventional **30 FPS** sense [2605.15423]. By contrast, QR- and PSD-based structural diagnostics can be sub-millisecond [2504.04884], and ORB-based VIO can exceed **100 FPS** [2509.10021]. This suggests that GAP9’s real-time operating point is workload-specific and cannot be summarized by a single universal frame-rate notion.

Finally, the broader comparative literature places GAP9 in a distinct middle ground. A 2026 review benchmarking a **336 million MAC** segmentation model reports **42.1 ms** latency and **20.8 MAC/cycle** on GAP9, concluding that it offers the best energy efficiency within **microcontroller-class power budgets**, while STM32N6 gives lower raw latency and Sony IMX500 achieves much higher utilization and lower energy-delay product through in-sensor compute [2603.08725]. This suggests that GAP9’s main significance is not that it dominates every metric, but that it occupies a particularly useful point in the design space: programmable, heterogeneous, and energy-efficient enough for always-on edge AI, yet flexible enough to support both DSP-heavy pipelines and increasingly sophisticated on-device learning.

Source: https://www.emergentmind.com/topics/gap9