Adaptive Deep Learning for Efficient Visual Pose Estimation aboard Ultra-low-power Nano-drones
Abstract: Sub-10cm diameter nano-drones are gaining momentum thanks to their applicability in scenarios prevented to bigger flying drones, such as in narrow environments and close to humans. However, their tiny form factor also brings their major drawback: ultra-constrained memory and processors for the onboard execution of their perception pipelines. Therefore, lightweight deep learning-based approaches are becoming increasingly popular, stressing how computational efficiency and energy-saving are paramount as they can make the difference between a fully working closed-loop system and a failing one. In this work, to maximize the exploitation of the ultra-limited resources aboard nano-drones, we present a novel adaptive deep learning-based mechanism for the efficient execution of a vision-based human pose estimation task. We leverage two State-of-the-Art (SoA) convolutional neural networks (CNNs) with different regression performance vs. computational costs trade-offs. By combining these CNNs with three novel adaptation strategies based on the output's temporal consistency and on auxiliary tasks to swap the CNN being executed proactively, we present six different systems. On a real-world dataset and the actual nano-drone hardware, our best-performing system, compared to executing only the bigger and most accurate SoA model, shows 28% latency reduction while keeping the same mean absolute error (MAE), 3% MAE reduction while being iso-latency, and the absolute peak performance, i.e., 6% better than SoA model.
- D. Palossi et al., “An open source and open hardware deep learning-powered visual navigation engine for autonomous nano-UAVs,” in International Conference on DCOSS, 2019.
- B. Varghese et al., “Challenges and opportunities in edge computing,” in IEEE International Conference on Smart Cloud, 2016.
- L. Lamberti et al., “Tiny-pulp-dronets: Squeezing neural networks for faster and lighter inference on multi-tasking autonomous nano-drones,” in IEEE International Conference on AICAS, 2022.
- D. Palossi et al., “Fully onboard ai-powered human-drone pose estimation on ultralow-power autonomous flying nano-uavs,” IEEE IOTJ, 2022.
- E. Cereda et al., “Deep neural network architecture search for accurate visual pose estimation aboard nano-uavs,” in IEEE ICRA, 2023.
- E. Park et al., “Big/little deep neural network for ultra low power inference,” in International Conference on CODES+ISSS, 2015.
- X. Sun et al., “Integral human pose regression,” in IEEE ECCV, 2018.
- D. C. Luvizon et al., “2d/3d pose estimation and action recognition using multitask deep learning,” in IEEE CVPR, 2018.
- R. A. Güler et al., “Densepose: Dense human pose estimation in the wild,” in IEEE CVPR, 2018.
- G. Shi et al., “Neural-swarm: Decentralized close-proximity multirotor control using learned interactions,” in IEEE ICRA, 2020.
- W. Zhao et al., “Learning-based bias correction for time difference of arrival ultra-wideband localization of resource-constrained mobile robots,” IEEE Robotics and Automation Letters, 2021.
- V. Sze et al., “Efficient Processing of Deep Neural Networks: A Tutorial and Survey,” Proceedings of the IEEE, 2017.
- D. J. Pagliari et al., “PLiNIO: A user-friendly library of gradient-based methods for complexity-aware dnn optimization,” in Forum on Specification & Design Languages (FDL), 2023.
- F. Daghero et al., “Energy-efficient deep learning inference on edge devices,” in Hardware Accelerator Systems for Artificial Intelligence and Machine Learning. Elsevier, 2021.
- H. Tann et al., “Runtime configurable deep neural networks for energy-accuracy trade-off,” in IEEE CODES, 2016.
- F. Daghero et al., “Human activity recognition on microcontrollers with quantized and adaptive deep neural networks,” ACM Trans. Embed. Comput. Syst., 2022.
- S. Teerapittayanon et al., “BranchyNet: Fast inference via early exiting from deep neural networks,” in ICPR, 2016.
- E. Flamand et al., “Gap-8: A risc-v soc for ai at the edge of the iot,” in IEEE Int. Conf. on ASAP, 2018.
- F. Conti et al., “An iot endpoint system-on-chip for secure and energy-efficient near-sensor analytics,” IEEE TCAS-I, 2017.
- A. Burrello et al., “Dory: Automatic end-to-end deployment of real-world dnns on low-cost iot mcus,” IEEE Trans. Comput., 2021.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.