TinyDEVO: Deep Event-based VO on ULP MCUs

• TinyDEVO is a deep learning-based event-driven visual odometry pipeline that processes event voxel grids with quantized convolutions on ultra-low-power MCUs.
• It employs aggressive channel pruning, architectural simplification, and mixed-precision quantization to deliver a 48–53× speedup and significant resource savings.
• TinyDEVO maintains competitive trajectory accuracy while drastically reducing memory and computational requirements, enabling practical on-device deployment.

TinyDEVO is a deep learning-based event-based visual odometry (VO) pipeline specifically engineered for deployment on resource-constrained, ultra-low-power (ULP) microcontroller units (MCUs). Distinct from conventional VO systems, which often rely on high-end GPUs, TinyDEVO demonstrates that state-of-the-art event-driven VO can operate end-to-end on a 9-core RISC-V MCU, at ~1.2 frames per second while consuming 86 mW average power, and delivering trajectory accuracy competitive with many full-scale VO algorithms. The design, optimizations, and performance metrics of TinyDEVO are documented precisely in (Marchei et al., 9 Apr 2026).

1. Model Architecture and Event Representation

TinyDEVO operates on the “event voxel grid” (EVG) representation, where asynchronous camera events $(x, y, t, p)$ are accumulated across five timestamped 2D bins, forming a 5 × $W$ × $H$ input tensor. No additional pre-processing—such as optical flow estimation—is imposed.

The end-to-end pipeline comprises four stages, inherited from the DEVO baseline:

1. Patchifier: A shallow convolutional network that extracts matching features ($MF$, output size: $\frac{W}{4} \times \frac{H}{4} \times Ch_{MF}$) and context features ($CF$, output size: $\frac{W}{4} \times \frac{H}{4} \times Ch_{CF}$). In TinyDEVO, feature channels are reduced to $Ch_{MF} = 64$ and $Ch_{CF} = 96$, the smallest feature pyramid is dropped, and bypass connections are omitted. All convolutions utilize int8 quantization and are accelerated by the GAP9’s NE16 unit.
2. Correlation Block: Extracts $N_{\mathrm{patches}}$ patches and forms a patch-graph $W$0 by computing pairwise dot-products (edges) over a temporal window $W$1 and patch lifetime $W$2:

$W$3

Resulting correlation features pass downstream.

1. Update Block: Aggregates patch correlations and contextual features. The original recurrent GRU structure is replaced in TinyDEVO by a layer-normalization and double fully connected (FC) sequence (LN+FC→ReLU→FC), with most FCs quantized to int8, except for the scatter softmax (FP16). Temporal convolutions and reduced softmax aggregations are retained for performance.
2. Bundle Adjustment (BA): Final pose refinement and global trajectory optimization across a sliding window, implemented in BF16. Negligible-motion graph edges are pruned using this differentiable optimizer.

2. Resource-Constrained Optimization Techniques

TinyDEVO achieves a marked reduction in computational and memory requirements through targeted interventions:

• Feature Channel Shrinking: Halving Patchifier output channels ($W$4, $W$5) yields 11.5× lower memory (63.8 MB vs. 733 MB) and 29.7× lower MACs per frame (5.2 G vs. 155 G).
• Architectural Pruning: Elimination of the small MF pyramid branch (saves 87.5 MB, 3.8 G MAC) and bypass connections (implementation simplification), as well as substituting GRUs with an LN+2FC block (saving 0.9 G MAC) contribute to streamlining the pipeline.
• Patch-Graph Hyperparameter Tuning: An extensive sweep over $W$6 (8–22), $W$7 (16–96), and $W$8 (8–13) identifies an optimal trade-off at $W$9. This configuration reduces graph edges, MACs, and memory by 65–75% compared to baselines, with only a modest increase in Average Trajectory Error (ATE).
• Quantization: Patchifier and update blocks are int8 (via NE16 acceleration), while correlation operates in FP16, and bundle adjustment in BF16. This facilitates full on-chip/external memory residency of network parameters and intermediates within a 63.8 MB envelope.

3. Deployment and Empirical Performance on Ultra-low-power MCU

TinyDEVO is deployed on a GAP9 SoC featuring a 1-core Fabric Controller and a 9-core compute cluster, each core supporting 4-way int8 SIMD and mixed-precision FPUs. The NE16 accelerator supports efficient $H$0 and $H$1 int8 convolutions.

• Processing Rates:
  • DEVO (fp16/int8) on GAP9: 44.9 s/frame (0.02 fps).
  • TinyDEVO: 0.85 s/frame (1.2 fps) for $H$2 inputs.
  • Stagewise speedups: PATCH (1.22×), CORR (12.6×), UPD (114×), BA (3.2×). Overall, a 48–53× speedup depending on input size.
• Power and Energy:
  • PATCH: 98 mW, CORR: 94 mW, UPD: 79 mW, BA: 39 mW. Average total power: 86 mW (including off-chip HyperRAM).
  • Energy per frame: $H$3.

4. Accuracy, Evaluation, and Benchmarking

TinyDEVO is benchmarked on MVSEC (indoor), HKU, and RPG datasets, using average trajectory error (ATE) after Umeyama alignment as the primary metric. The normalized ATE ($H$4) is also reported. Summary metrics:

Model	MVSEC (cm)	HKU (cm)	RPG (cm)	Memory (MB)	MACs/frame (G)
DEVO (full)	8.3	25.9	0.9	733	155
TinyDEVO	27.0	45.3	4.9	63.8	5.2

• Absolute error increases: $H$5 cm, $H$6 cm, $H$7 cm.
• Relative error increases: 2.25× (MVSEC), 0.75× (HKU), 4.44× (RPG).
• $H$8 rises from 0.42 to 0.79 from reduced to final TinyDEVO configurations.

Comparison with geometric VO:

• ORB-SLAM3 (RGB, 900 MB) on RPG: 2.97 cm ATE. TinyDEVO (64 MB): 2.2 cm ATE.
• EVO (event-based, 535 MB) on RPG: 10.1 cm ATE. TinyDEVO: 2.2 cm ATE.
• Memory savings: $H$9 to $MF$0 less; accuracy improvement: $MF$1 to $MF$2 on RPG.

5. Major Design Insights and Trade-offs

The TinyDEVO approach reveals several critical design principles for deep event-based VO on ULP hardware:

1. Aggressive Early Channel Pruning: Reducing feature-map channels in early convolutional stages yields significant resource savings with only a small impact on ATE.
2. Simplification of Temporal Modules: Replacing GRU recurrence by normalization and minimal FC layers in the update block balances cost and temporal modeling fidelity.
3. Mixed-Precision Quantization: Segmented allocation of int8 for convolutions and higher-precision (FP16/BF16) for graph-centric or aggregation operations maximizes both speed and accuracy.
4. Comprehensive Hyperparameter Sweeps: Careful selection of $MF$3, $MF$4, and $MF$5 identifies “Pareto knee” regions where further reduction significantly harms accuracy.
5. Heterogeneous Hardware Utilization: Effective overlap of data movement (DMA) and computing operations between accelerators and multicore CPU resources underpins real-time throughput.

TinyDEVO demonstrates that event-based deep VO algorithms can be compressed, quantized, and tuned to execute at near-real-time rates on microcontrollers while maintaining competitive VO accuracy, supporting deployment in on-device vision scenarios such as wearables and nano-robotics (Marchei et al., 9 Apr 2026).