Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
103 tokens/sec
GPT-4o
11 tokens/sec
Gemini 2.5 Pro Pro
50 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
3 tokens/sec
DeepSeek R1 via Azure Pro
33 tokens/sec
2000 character limit reached

DART: Implicit Doppler Tomography for Radar Novel View Synthesis (2403.03896v1)

Published 6 Mar 2024 in cs.CV and cs.LG

Abstract: Simulation is an invaluable tool for radio-frequency system designers that enables rapid prototyping of various algorithms for imaging, target detection, classification, and tracking. However, simulating realistic radar scans is a challenging task that requires an accurate model of the scene, radio frequency material properties, and a corresponding radar synthesis function. Rather than specifying these models explicitly, we propose DART - Doppler Aided Radar Tomography, a Neural Radiance Field-inspired method which uses radar-specific physics to create a reflectance and transmittance-based rendering pipeline for range-Doppler images. We then evaluate DART by constructing a custom data collection platform and collecting a novel radar dataset together with accurate position and instantaneous velocity measurements from lidar-based localization. In comparison to state-of-the-art baselines, DART synthesizes superior radar range-Doppler images from novel views across all datasets and additionally can be used to generate high quality tomographic images.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (86)
  1. Constant false alarm rate (cfar) detection. Url: https://www.mathworks.com/help/phased/ug/constant-false-alarm-rate-cfar-detection.html; Accessed: 2023-11-15.
  2. Milli-rio: Ego-motion estimation with low-cost millimetre-wave radar. IEEE Sensors Journal, 21(3):3314–3323, 2020.
  3. Raysar - 3d sar simulator: Now open source. 2016.
  4. Pointillism: Accurate 3d bounding box estimation with multi-radars. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pages 340–353, 2020.
  5. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855–5864, 2021.
  6. M. S. Bartlett. Periodogram analysis and continuous spectra. Biometrika, 37(1/2):1–16, 1950.
  7. Nitin Bhatia et al. Survey of nearest neighbor techniques. arXiv preprint arXiv:1007.0085, 2010.
  8. JAX: composable transformations of Python+NumPy programs, 2018.
  9. Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 6045–6052. IEEE, 2018.
  10. A novel radar point cloud generation method for robot environment perception. IEEE Transactions on Robotics, 2022.
  11. C. J. Coleman. A ray tracing formulation and its application to some problems in over-the-horizon radar. Radio Science, 33(4):1187–1197, 1998.
  12. Radargrammetry and sar interferometry for dem generation: validation and data fusion. In SAR workshop: CEOS committee on earth observation satellites, page 367, 2000.
  13. Migration of ground-penetrating radar data method with a finite-element and dispersion. Geophysics, 69, 2004.
  14. Hidden gems: 4d radar scene flow learning using cross-modal supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9340–9349, 2023.
  15. Yaw aided radar inertial odometry uisng manhattan world assumptions. In 2021 28th Saint Petersburg International Conference on Integrated Navigation Systems (ICINS), pages 1–10, 2021.
  16. Full-waveform inversion of crosshole radar data based on 2-d finite-difference time-domain solutions of maxwell’s equations. IEEE Transactions on Geoscience and Remote Sensing, 45(9):2807–2828, 2007.
  17. Superrf: Enhanced 3d rf representation using stationary low-cost mmwave radar. In International Conference on Embedded Wireless Systems and Networks (EWSN)…, page 120. NIH Public Access, 2020.
  18. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501–5510, 2022.
  19. Improvements to the finite-difference time-domain method for calculating the radar cross section of a perfectly conducting target. IEEE Transactions on Microwave Theory and Techniques, 38(7):919–927, 1990.
  20. Mimo-sar: A hierarchical high-resolution imaging algorithm for mmwave fmcw radar in autonomous driving. IEEE Transactions on Vehicular Technology, 70(8):7322–7334, 2021.
  21. Radar super resolution using a deep convolutional neural network. Journal of Atmospheric and Oceanic Technology, 37(12):2197–2207, 2020.
  22. Through fog high-resolution imaging using millimeter wave radar. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020a.
  23. Through fog high-resolution imaging using millimeter wave radar. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11464–11473, 2020b.
  24. Real-time loop closure in 2d lidar slam. In 2016 IEEE international conference on robotics and automation (ICRA), pages 1271–1278. IEEE, 2016.
  25. A ray launching approach for modeling an fmcw radar system. In 2017 18th International Radar Symposium (IRS), pages 1–10, 2017.
  26. Franz Hlawatsch. Time-frequency analysis and synthesis of linear signal spaces: time-frequency filters, signal detection and estimation, and Range-Doppler estimation. Springer Science & Business Media, 1998.
  27. Neural lidar fields for novel view synthesis. arXiv preprint arXiv:2305.01643, 2023.
  28. The fundamentals of millimeter wave sensors. Texas Instruments, pages 1–8, 2017.
  29. Utilizing sar imagery in three-dimensional neural radiance fields-based applications. In Algorithms for Synthetic Aperture Radar Imagery XXX, page 1252002. SPIE, 2023.
  30. Millimeter wave systems for airports and short-range aviation communications: A survey of the current channel models at mmwave frequencies. In 2017 IEEE/AIAA 36th Digital Avionics Systems Conference (DASC), pages 1–8, 2017.
  31. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  32. mmwave-yolo: A mmwave imaging radar-based real-time multiclass object recognition system for adas applications. IEEE Transactions on Instrumentation and Measurement, 71:1–10, 2022.
  33. Multiview three-dimensional reconstruction by millimetre-wave portable camera. Scientific reports, 7(1):6479, 2017a.
  34. Multistatic millimeter-wave imaging by multiview portable camera. IEEE Access, 5:19259–19268, 2017b.
  35. Multiview mm-wave imaging with augmented depth camera information. IEEE Access, 6:16869–16877, 2018.
  36. Exploiting temporal relations on radar perception for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17071–17080, 2022a.
  37. Modality-agnostic learning for radar-lidar fusion in vehicle detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 918–927, 2022b.
  38. Azimuth super-resolution for fmcw radar in autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17504–17513, 2023.
  39. Soli: Ubiquitous gesture sensing with millimeter wave radar. ACM Transactions on Graphics (TOG), 35(4):1–19, 2016.
  40. Neural sparse voxel fields. Advances in Neural Information Processing Systems, 33:15651–15663, 2020.
  41. Radar-camera pixel depth association for depth completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12507–12516, 2021.
  42. See through smoke: robust indoor mapping with low-cost mmwave radar. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pages 14–27, 2020a.
  43. milliego: single-chip mmwave radar aided egomotion estimation via deep sensor fusion. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pages 109–122, 2020b.
  44. Learning neural acoustic fields. In Advances in Neural Information Processing Systems, pages 3165–3177. Curran Associates, Inc., 2022.
  45. Elmer finite element solver for multiphysics and multiscale problems. Forschungszentrum Juelich, 2013.
  46. 60 ghz synthetic aperture radar for short-range imaging: Theory and experiments. In 2014 48th Asilomar Conference on Signals, Systems and Computers, pages 553–558. IEEE, 2014.
  47. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. In CVPR, 2021.
  48. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
  49. CFAR: The principles of automatic radar detection in clutter. NASA STI/Recon Technical Report A, 90:23371, 1990.
  50. High-resolution radar dataset for semi-supervised learning of dynamic objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020.
  51. Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG), 41(4):1–15, 2022.
  52. Osprey: a mmwave approach to tire wear sensing. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pages 28–41, 2020.
  53. Exploring mmwave radar and camera fusion for high-resolution and long-range depth imaging. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3995–4002. IEEE, 2022.
  54. High resolution point clouds from mmwave radar. In 2023 IEEE International Conference on Robotics and Automation (ICRA), 2023.
  55. Neural implicit surface reconstruction using imaging sonar. arXiv preprint arXiv:2209.08221, 2022.
  56. 3d point cloud generation with millimeter-wave radar. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 4(4), 2020.
  57. Robust multimodal vehicle detection in foggy weather using complementary lidar and radar signals. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 444–453, 2021.
  58. Raw high-definition radar for multi-task learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17021–17030, 2022.
  59. Neural volumetric reconstruction for coherent synthetic aperture sonar. arXiv preprint arXiv:2306.09909, 2023.
  60. Principles of Modern Radar. Citeseer, 2010.
  61. Mark A Richards et al. Fundamentals of radar signal processing. Mcgraw-hill New York, 2005.
  62. Monte Carlo statistical methods. Springer, 1999.
  63. Hermann Rohling. Radar cfar thresholding in clutter and multiple target situations. IEEE Transactions on Aerospace and Electronic Systems, AES-19(4):608–621, 1983.
  64. Millimeter-wave radar sensor for automotive intelligent cruise control (icc). IEEE Transactions on microwave theory and techniques, 45(12):2444–2453, 1997.
  65. Seeing around street corners: Non-line-of-sight detection and tracking in-the-wild using doppler radar. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  66. Virtual radar: Real-time millimeter-wave radar sensor simulation for perception-driven robotics. IEEE Robotics and Automation Letters, 6(3):4704–4711, 2021.
  67. A realistic radar ray tracing simulator for large mimo-arrays in automotive environments. IEEE Journal of Microwaves, 1(4):962–974, 2021.
  68. Cylindrical millimeter-wave imaging technique for concealed weapon detection. In 26th AIPR Workshop: Exploiting New Image Sources and Sensors, pages 242–250. SPIE, 1998.
  69. millieye: A lightweight mmwave radar and camera fusion system for robust object detection. In Proceedings of the International Conference on Internet-of-Things Design and Implementation, pages 145–157, 2021.
  70. Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar. In Proceedings of the 3rd ACM Workshop on Millimeter-wave Networks and Sensing Systems, pages 51–56, 2019.
  71. Extending neural radiance fields (nerf) for synthetic aperture radar (sar) novel image generation. In Synthetic Data for Artificial Intelligence and Machine Learning: Tools, Techniques, and Applications, pages 268–276. SPIE, 2023.
  72. Deeppoint: A deep learning model for 3d reconstruction in point clouds via mmwave radar. arXiv preprint arXiv:2109.09188, 2021.
  73. Block-nerf: Scalable large scene neural view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8248–8258, 2022.
  74. Nerfstudio: A modular framework for neural radiance field development. arXiv preprint arXiv:2302.04264, 2023.
  75. Finite-difference time-domain simulation of ground penetrating radar on dispersive, inhomogeneous, and conductive soils. IEEE Transactions on Geoscience and Remote Sensing, 36(6):1928–1937, 1998.
  76. AWR1843AOP Single-chip 77- and 79-GHz FMCW mmWave Sensor Antennas-OnPackage (AOP). Texas Instruments, 2021.
  77. m-activity: Accurate and real-time human activity recognition via millimeter wave radar. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8298–8302. IEEE, 2021.
  78. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600–612, 2004.
  79. How noise radar technology brings together active sensing and modern electronic warfare techniques in a combined sensor concept. In 2019 Sensor Signal Processing for Defence Conference (SSPD), pages 1–5, 2019.
  80. M4esh: mmwave-based 3d human mesh construction for multiple subjects. In Proceedings of the 20th ACM Conference on Embedded Networked Sensor Systems, pages 391–406, 2022.
  81. High-resolution 2d sar imaging by the millimeter-wave automobile radar. In 2017 IEEE Conference on Antenna Measurements & Applications (CAMA), pages 149–150. IEEE, 2017.
  82. Near-field mimo-sar millimeter-wave imaging with sparsely sampled aperture data. Ieee Access, 7:31801–31819, 2019.
  83. Plenoctrees for real-time rendering of neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5752–5761, 2021.
  84. Nerf2: Neural radio-frequency radiance fields. arXiv preprint arXiv:2305.06118, 2023.
  85. Point cloud features-based kernel svm for human-vehicle classification in millimeter wave radar. IEEE Access, 8:26012–26021, 2020.
  86. Towards deep radar perception for autonomous driving: Datasets, methods, and challenges. Sensors, 22(11), 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now