StofNet: Super-resolution Time of Flight Network (2308.12009v2)
Abstract: Time of Flight (ToF) is a prevalent depth sensing technology in the fields of robotics, medical imaging, and non-destructive testing. Yet, ToF sensing faces challenges from complex ambient conditions making an inverse modelling from the sparse temporal information intractable. This paper highlights the potential of modern super-resolution techniques to learn varying surroundings for a reliable and accurate ToF detection. Unlike existing models, we tailor an architecture for sub-sample precise semi-global signal localization by combining super-resolution with an efficient residual contraction block to balance between fine signal details and large scale contextual information. We consolidate research on ToF by conducting a benchmark comparison against six state-of-the-art methods for which we employ two publicly available datasets. This includes the release of our SToF-Chirp dataset captured by an airborne ultrasound transducer. Results showcase the superior performance of our proposed StofNet in terms of precision, reliability and model complexity. Our code is available at https://github.com/hahnec/stofnet.
- “Multi-frame super-resolution for time-of-flight imaging,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 2327–2331.
- “Batvision: Learning to see 3d spatial layout with two ears,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), 2020, pp. 1581–1587.
- “Beyond image to depth: Improving depth prediction using echoes,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 8268–8277.
- “Co-attention-guided bilinear model for echo-based depth estimation,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022, pp. 4648–4652.
- “Extending GCC-PHAT using Shift Equivariant Neural Networks,” in Proc. Interspeech 2022, 2022, pp. 1791–1795.
- “CatChatter: Acoustic perception for mobile robots,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 7209–7216, 2021.
- Christopher Hahne, “3-dimensional sonic phase-invariant echo localization,” in 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023, pp. 4121–4127.
- “Deep learning based autonomous vehicle super resolution doa estimation for safety driving,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4301–4315, 2020.
- “Strategies for reliable automatic onset time picking of acoustic emissions and of ultrasound signals in concrete,” Ultrasonics, vol. 43, no. 7, pp. 538–546, 2005.
- “Improved acoustic emission source location during fatigue and impact events in metallic and composite structures,” Structural Health Monitoring, vol. 16, no. 4, pp. 382–399, 2017.
- “Deep learning approaches for robust time of arrival estimation in acoustic emission monitoring,” Sensors, vol. 22, no. 3, 2022.
- “Geometric ultrasound localization microscopy,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2023: 26th International Conference, Vancouver, Canada, October 8-12, 2023, Proceedings. Springer, 2023.
- “Extension of time-difference-of-arrival self calibration solutions using robust multilateration,” in 2021 29th European Signal Processing Conference (EUSIPCO), 2021, pp. 870–874.
- Christopher Hahne, “Multimodal exponentially modified gaussian oscillators,” in 2022 IEEE International Ultrasonics Symposium (IUS), 2022, pp. 1–4.
- “Acoustic signal detection through the cross-correlation method in experiments with different signal to noise ratio and reverberation conditions,” in Ad-hoc Networks and Wireless, Berlin, Heidelberg, 2015, pp. 66–79, Springer Berlin Heidelberg.
- “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 2015, pp. 234–241.
- “Audio super-resolution using neural nets,” in ICLR (Workshop Track), 2017.
- “Speaker recognition from raw waveform with sincnet,” in 2018 IEEE spoken language technology workshop (SLT). IEEE, 2018, pp. 1021–1028.
- “Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1874–1883.
- “Enhanced deep residual networks for single image super-resolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 136–144.
- “Deep learning for ultrasound localization microscopy,” IEEE transactions on medical imaging, vol. 39, no. 10, pp. 3064–3078, 2020.
- “Addressing the curse of imbalanced training sets: one-sided selection,” in Icml. Citeseer, 1997, vol. 97, p. 179.
- “Performance benchmarking of microbubble-localization algorithms for ultrasound localization microscopy,” Nature Biomedical Engineering, vol. 6, no. 5, pp. 605–616, 2022.