---
title: 'TinyDEVO: Deep Event-based VO on ULP MCUs'
url: https://www.emergentmind.com/topics/tinydevo
type: topic
---

# TinyDEVO: Deep Event-based VO on ULP MCUs

TinyDEVO is a deep learning-based event-based visual odometry (VO) pipeline specifically engineered for deployment on resource-constrained, ultra-low-power (ULP) microcontroller units (MCUs). Distinct from conventional VO systems, which often rely on high-end GPUs, TinyDEVO demonstrates that state-of-the-art event-driven VO can operate end-to-end on a 9-core RISC-V MCU, at ~1.2 frames per second while consuming 86 mW average power, and delivering trajectory accuracy competitive with many full-scale VO algorithms. The design, optimizations, and performance metrics of TinyDEVO are documented precisely in [2604.08060].

## 1. Model Architecture and Event Representation

TinyDEVO operates on the “event voxel grid” (EVG) representation, where asynchronous camera events $(x, y, t, p)$ are accumulated across five timestamped 2D bins, forming a 5 × $W$ × $H$ input tensor. No additional pre-processing—such as optical flow estimation—is imposed.

The end-to-end pipeline comprises four stages, inherited from the DEVO baseline:
1. **Patchifier:** A shallow convolutional network that extracts matching features ($MF$, output size: $\frac{W}{4} \times \frac{H}{4} \times Ch_{MF}$) and context features ($CF$, output size: $\frac{W}{4} \times \frac{H}{4} \times Ch_{CF}$). In TinyDEVO, feature channels are reduced to $Ch_{MF} = 64$ and $Ch_{CF} = 96$, the smallest feature pyramid is dropped, and bypass connections are omitted. All convolutions utilize int8 quantization and are accelerated by the GAP9’s NE16 unit.
2. **Correlation Block:** Extracts $N_{\mathrm{patches}}$ patches and forms a patch-graph $\mathcal{P}_e$ by computing pairwise dot-products (edges) over a temporal window $R_w$ and patch lifetime $P_{LT}$:
   $$
   N_{\text{edges}} = N_{\text{patches}} \left( (R_w + 1)P_{LT} + \frac{m \bigl(2(R_w + 1) - 1 - m\bigr)}{2} \right), \quad m = \min \{ P_{LT} - 1, R_w \}
   $$
   Resulting correlation features pass downstream.
3. **Update Block:** Aggregates patch correlations and contextual features. The original recurrent GRU structure is replaced in TinyDEVO by a layer-normalization and double fully connected (FC) sequence (LN+FC→ReLU→FC), with most FCs quantized to int8, except for the scatter softmax (FP16). Temporal convolutions and reduced softmax aggregations are retained for performance.
4. **Bundle Adjustment (BA):** Final pose refinement and global trajectory optimization across a sliding window, implemented in BF16. Negligible-motion graph edges are pruned using this differentiable optimizer.

## 2. Resource-Constrained Optimization Techniques

TinyDEVO achieves a marked reduction in computational and memory requirements through targeted interventions:

- **Feature Channel Shrinking:** Halving Patchifier output channels ($Ch_{MF}$, $Ch_{CF}$) yields 11.5× lower memory (63.8 MB vs. 733 MB) and 29.7× lower MACs per frame (5.2 G vs. 155 G).
- **Architectural Pruning:** Elimination of the small MF pyramid branch (saves 87.5 MB, 3.8 G MAC) and bypass connections (implementation simplification), as well as substituting GRUs with an LN+2FC block (saving 0.9 G MAC) contribute to streamlining the pipeline.
- **Patch-Graph Hyperparameter Tuning:** An extensive sweep over $R_w$ (8–22), $N_{\text{patches}}$ (16–96), and $P_{LT}$ (8–13) identifies an optimal trade-off at $(R_w, P_{LT}, N_{\text{patches}}) = (12, 10, 24)$. This configuration reduces graph edges, MACs, and memory by 65–75% compared to baselines, with only a modest increase in Average Trajectory Error (ATE).
- **Quantization:** Patchifier and update blocks are int8 (via NE16 acceleration), while correlation operates in FP16, and bundle adjustment in BF16. This facilitates full on-chip/external memory residency of network parameters and intermediates within a 63.8 MB envelope.

## 3. Deployment and Empirical Performance on Ultra-low-power MCU

TinyDEVO is deployed on a GAP9 SoC featuring a 1-core Fabric Controller and a 9-core compute cluster, each core supporting 4-way int8 SIMD and mixed-precision FPUs. The NE16 accelerator supports efficient $3 \times 3$ and $1 \times 1$ int8 convolutions.

- **Processing Rates:**
  - DEVO (fp16/int8) on GAP9: 44.9 s/frame (0.02 fps).
  - TinyDEVO: 0.85 s/frame (1.2 fps) for $240 \times 180$ inputs.
  - Stagewise speedups: PATCH (1.22×), CORR (12.6×), UPD (114×), BA (3.2×). Overall, a 48–53× speedup depending on input size.
- **Power and Energy:**
  - PATCH: 98 mW, CORR: 94 mW, UPD: 79 mW, BA: 39 mW.   Average total power: 86 mW (including off-chip HyperRAM).
  - Energy per frame: $E_{\text{frame}} \approx \frac{86\,\mathrm{mW}}{1.1\,\mathrm{fps}} = 79\,\mathrm{mJ/frame}$.

## 4. Accuracy, Evaluation, and Benchmarking

TinyDEVO is benchmarked on MVSEC (indoor), HKU, and RPG datasets, using average trajectory error (ATE) after Umeyama alignment as the primary metric. The normalized ATE ($\overline{\mathrm{nATE}}$) is also reported. Summary metrics:

| Model         | MVSEC (cm) | HKU (cm) | RPG (cm) | Memory (MB) | MACs/frame (G) |
|---------------|------------|----------|----------|-------------|----------------|
| DEVO (full)   |    8.3     |  25.9    |   0.9    |    733      |    155         |
| TinyDEVO      |   27.0     |  45.3    |   4.9    |   63.8      |     5.2        |

- Absolute error increases: $\Delta_{\mathrm{MVSEC}} = 18.7$ cm, $\Delta_{\mathrm{HKU}} = 19.4$ cm, $\Delta_{\mathrm{RPG}} = 4.0$ cm.
- Relative error increases: 2.25× (MVSEC), 0.75× (HKU), 4.44× (RPG).
- $\overline{\mathrm{nATE}}$ rises from 0.42 to 0.79 from reduced to final TinyDEVO configurations.

**Comparison with geometric VO:**
- ORB-SLAM3 (RGB, 900 MB) on RPG: 2.97 cm ATE. TinyDEVO (64 MB): 2.2 cm ATE.
- EVO (event-based, 535 MB) on RPG: 10.1 cm ATE. TinyDEVO: 2.2 cm ATE.
- Memory savings: $8.4\times$ to $14\times$ less; accuracy improvement: $1.35\times$ to $4.5\times$ on RPG.

## 5. Major Design Insights and Trade-offs

The TinyDEVO approach reveals several critical design principles for deep event-based VO on ULP hardware:

1. **Aggressive Early Channel Pruning:** Reducing feature-map channels in early convolutional stages yields significant resource savings with only a small impact on ATE.
2. **Simplification of Temporal Modules:** Replacing GRU recurrence by normalization and minimal FC layers in the update block balances cost and temporal modeling fidelity.
3. **Mixed-Precision Quantization:** Segmented allocation of int8 for convolutions and higher-precision (FP16/BF16) for graph-centric or aggregation operations maximizes both speed and accuracy.
4. **Comprehensive Hyperparameter Sweeps:** Careful selection of $R_w$, $N_{\text{patches}}$, and $P_{LT}$ identifies “Pareto knee” regions where further reduction significantly harms accuracy.
5. **Heterogeneous Hardware Utilization:** Effective overlap of data movement (DMA) and computing operations between accelerators and multicore CPU resources underpins real-time throughput.

TinyDEVO demonstrates that event-based deep VO algorithms can be compressed, quantized, and tuned to execute at near-real-time rates on microcontrollers while maintaining competitive VO accuracy, supporting deployment in on-device vision scenarios such as wearables and nano-robotics [2604.08060].

Source: https://www.emergentmind.com/topics/tinydevo