TinyDEVO: Deep Event-based Visual Odometry on Ultra-low-power Multi-core Microcontrollers

Abstract: A key task in embedded vision is visual odometry (VO), which estimates camera motion from visual sensors, and it is a core component in many embedded power-constrained systems, from autonomous robots to augmented and virtual reality wearable devices. The newest class of VO systems combines deep learning models with bio-inspired event-based cameras, which are robust to motion blur and lighting conditions. However, state-of-the-art (SoA) event-based VO algorithms require significant memory and computation. For example, the leading approach DEVO requires 733 MB of memory and 155 billion multiply-accumulate (MAC) operations per frame. We present TinyDEVO, an event-based VO deep learning model designed for resource-constrained microcontroller units (MCUs). We deploy TinyDEVO on an ultra-low-power (ULP) 9-core RISC-V-based MCU, achieving a throughput of approximately 1.2 frames per second with an average power consumption of only 86 mW. Thanks to our neural network architectural optimizations and hyperparameter tuning, TinyDEVO reduces the memory footprint by 11.5x (to 63.8 MB) and the number of operations per frame by 29.7x (to 5.2 billion MACs per frame) compared to DEVO, while maintaining an average trajectory error of 27 cm, i.e., only 19 cm higher than DEVO, on three state-of-the-art datasets. Our work demonstrates, for the first time, the feasibility of an event-based VO pipeline on ultra-low-power devices.

• The paper presents a redesigned DEVO pipeline that reduces memory and compute by over 10×, enabling deep event-based visual odometry on ultra-low-power microcontrollers.
• The paper details aggressive architectural optimizations, pruning non-critical modules like GRU and softmax while preserving temporal convolutions to maintain competitive accuracy.
• The paper demonstrates practical embedded deployment on a RISC-V GAP9 SoC, achieving soft real-time inference at 1.2 fps under 100 mW power, suitable for nano-robotics and IoT wearables.

TinyDEVO: Deep Event-based Visual Odometry on Ultra-low-power Multi-core Microcontrollers

Introduction and Context

Visual odometry (VO) is an essential competency for embedded intelligent agents, enabling robust estimation of camera motion on platforms with strictly constrained power and compute resources. Recent advances leverage deep learning (DL) models with event-based cameras to achieve improved robustness to high-speed motion and challenging lighting, but state-of-the-art systems such as DEVO demand resources far exceeding those available on ultra-low-power (ULP) microcontroller units (MCUs). "TinyDEVO: Deep Event-based Visual Odometry on Ultra-low-power Multi-core Microcontrollers" (2604.08060) presents a complete reworking of the DEVO pipeline, specifically targeting drastic reductions in memory and computation to enable deployment on ULP multi-core RISC-V MCUs, closing the gap between high-performing event-based VO research and practical usage on embedded systems.

Figure 1: TinyDEVO is tailored for embedded event-based vision; shown are platforms including an IoT wearable and a miniaturized robot leveraging event cameras.

System Architecture and Optimization Methodology

TinyDEVO is derived from the high-performing DEVO framework—which combines event voxelized input with a patch-based CNN feature extractor and graph-based recurrent update modules—but subjects each architectural element to ablation- and constraint-driven optimizations. Key intervention points include the patchifier and update blocks, which dominate resource consumption.

Figure 2: High-level architectural blocks in DEVO optimized in TinyDEVO; optimizations target the patchifier (A) and update (B) stages.

Channel depths in feature representations are aggressively reduced (e.g., $Ch_{MF} = 64$, $Ch_{CF} = 96$), and non-critical pathways (such as bypass connections and secondary matching feature pyramids) are pruned. In the update block, elimination of computationally expensive softmax aggregation (SA) modules and GRU units is validated, while maintaining temporal convolutions (TC) as essential for preserving performance. Redundant repeated computation over patch graph edges is suppressed through careful parameterization of $N_{patches}$, $R_w$, and patch lifetime.

This systematic reduction pipeline leads to more than an order-of-magnitude reduction in both memory and compute without severe ATE degradation.

Hyperparameter and Trade-off Analysis

Parameter search across patch graph size, window lengths, and patch sampling density is performed to chart the full Pareto front between accuracy and throughput. The optimal configuration, found via knee-point analysis, utilizes $R_w=12$, $N_{patches}=24$, $P_{LT}=10$, yielding a model with $N_{edges} \approx 4.8$k.

Figure 3: Trade-off sweeps over $R_w$ and $N_{patches}$ versus number of edges, pipeline MACs, and ATE, identifying the knee of the Pareto frontier.

TinyDEVO achieves ATEs of $Ch_{CF} = 96$0 cm (MVSEC), $Ch_{CF} = 96$1 cm (HKU), and $Ch_{CF} = 96$2 cm (RPG) with only $Ch_{CF} = 96$3 G MACs/frame and $Ch_{CF} = 96$4 MB of RAM, contrasting with DEVO’s $Ch_{CF} = 96$5 MB and $Ch_{CF} = 96$6 G MACs/frame. The loss in accuracy is moderate: degradation in ATE is $Ch_{CF} = 96$7 $Ch_{CF} = 96$8 cm across datasets, while yielding $Ch_{CF} = 96$9 reduction in MACs and $N_{patches}$0 less RAM utilization.

Embedded Deployment and Empirical Evaluation

TinyDEVO is deployed on the GWT GAP9 SoC, featuring a heterogeneous cluster of RISC-V integer and mixed-precision floating-point cores with an NE16 int8-optimized accelerator. All DL operations are quantized (int8 for neural modules, fp16 for geometric), maintaining numerical reliability.

TinyDEVO executes at $N_{patches}$1 fps with a measured average power of $N_{patches}$2 mW—including all off-chip memory accesses. This marks a more than $N_{patches}$3 end-to-end latency improvement relative to unoptimized DEVO. Energy per inference is $N_{patches}$479 mJ.

Figure 4: Trajectory prediction samples from TinyDEVO across MVSEC, HKU, and RPG datasets.

Figure 5: Power traces of GAP9 EVK during TinyDEVO execution, showing per-module and aggregate power envelopes below 100 mW.

Quantitative Comparison Against Prior Art

On embedded settings, TinyDEVO outperforms state-of-the-art geometric monocular VO pipelines in both resource footprint and accuracy-per-memory. For example, ORB-SLAM3 yields ATE of $N_{patches}$5 cm at $N_{patches}$6 MB RAM, EVO yields $N_{patches}$7 cm at $N_{patches}$8 MB (measured), whereas TinyDEVO produces $N_{patches}$9 cm ATE at only $R_w$0 MB RAM—substantially better accuracy/memory efficiency (by factors of $R_w$1 and $R_w$2, respectively).

DL-based event-only approaches previously required server-class GPUs and high power, but TinyDEVO provides evidence of real-time, generalizable inference in form factors suitable for nano-robotics, XR wearables, and mobile IoT agents.

Theoretical and Practical Implications

This work demonstrates that competitive DL-based event-only VO is feasible within sub-100 mW MCUs. The model reduction methodology—combining architectural ablation and aggressive hyperparameter tuning—provides a template for bringing other high-complexity DL pipelines into the embedded domain. The approach also interrogates which features are critical for VO accuracy under severe constraints (e.g., the indispensable role of temporal convolutions).

TinyDEVO’s throughput is already sufficient for “soft” real-time closed-loop applications in robotics and AR/VR. The power envelope opens integration into long-duration battery-powered systems and creates opportunities for dense deployment of mobile perceptual agents. As microcontrollers and accelerators evolve, the presented reduction pipeline suggests avenues for attaining even more efficient hard-real-time solutions.

Conclusion

TinyDEVO delivers the first demonstration of high-accuracy, fully event-based DL VO on ULP MCUs, establishing an impactful benchmark for energy/performance trade-offs in embedded computer vision. The rigorous compression of the DEVO pipeline informs future reductions of DL models for edge AI, and sets the technical direction for future nano-robotic autonomy and wearables requiring onboard SLAM and self-localization.