Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 190 tok/s
Gemini 2.5 Pro 51 tok/s Pro
GPT-5 Medium 39 tok/s Pro
GPT-5 High 46 tok/s Pro
GPT-4o 130 tok/s Pro
Kimi K2 202 tok/s Pro
GPT OSS 120B 439 tok/s Pro
Claude Sonnet 4.5 37 tok/s Pro
2000 character limit reached

DeepIPCv2: LiDAR-powered Robust Environmental Perception and Navigational Control for Autonomous Vehicle (2307.06647v3)

Published 13 Jul 2023 in cs.RO, cs.AI, and cs.CV

Abstract: We present DeepIPCv2, an autonomous driving model that perceives the environment using a LiDAR sensor for more robust drivability, especially when driving under poor illumination conditions where everything is not clearly visible. DeepIPCv2 takes a set of LiDAR point clouds as the main perception input. Since point clouds are not affected by illumination changes, they can provide a clear observation of the surroundings no matter what the condition is. This results in a better scene understanding and stable features provided by the perception module to support the controller module in estimating navigational control properly. To evaluate its performance, we conduct several tests by deploying the model to predict a set of driving records and perform real automated driving under three different conditions. We also conduct ablation and comparative studies with some recent models to justify its performance. Based on the experimental results, DeepIPCv2 shows a robust performance by achieving the best drivability in all driving scenarios. Furthermore, to support future research, we will upload the codes and data to https://github.com/oskarnatan/DeepIPCv2.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. J. Horgan, C. Hughes, J. McDonald, and S. Yogamani, “Vision-based driver assistance systems: Survey, taxonomy and advances,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC), Gran Canaria, Spain, Sep. 2015, pp. 2032–2039.
  2. Y. Cui, R. Chen, W. Chu, L. Chen, D. Tian, Y. Li, and D. Cao, “Deep learning for image and point cloud fusion in autonomous driving: A review,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 2, pp. 722–739, Feb. 2022.
  3. K. Muhammad, T. Hussain, H. Ullah, J. D. Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V. H. C. de Albuquerque, “Vision-based semantic segmentation in scene understanding for autonomous driving: Recent achievements, challenges, and outlooks,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 12, pp. 22 694–22 715, Dec. 2022.
  4. D. Omeiza, H. Webb, M. Jirotka, and L. Kunze, “Explanations in autonomous driving: A survey,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 8, pp. 10 142–10 162, Aug. 2022.
  5. G. Adam, V. Chitalia, N. Simha, A. Ismail, S. Kulkarni, V. Narayan, and M. Schulze, “Robustness and deployability of deep object detectors in autonomous driving,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC), Auckland, New Zealand, Oct. 2019, pp. 4128–4133.
  6. C. Wang and N. Aouf, “Fusion attention network for autonomous cars semantic segmentation,” in Proc. IEEE Intell. Veh. Symp. (IV), Aachen, Germany, Jul. 2022, pp. 1525–1530.
  7. S. Matsuzaki, H. Masuzawa, and J. Miura, “Multi-source soft pseudo-label learning with domain similarity-based weighting for semantic segmentation,” in Proc. IEEE/RSJ Inter. Conf. Intell. Robots and Syst. (IROS), Detroit, USA, Oct. 2023, pp. 5852–5857.
  8. O. Natan, D. U. K. Putri, and A. Dharmawan, “Deep learning-based weld spot segmentation using modified UNet with various convolutional blocks,” ICIC Express Letters Part B: Applications, vol. 12, no. 12, pp. 1169–1176, Dec. 2021.
  9. A. Gurram, A. F. Tuna, F. Shen, O. Urfalioglu, and A. M. López, “Monocular depth estimation through virtual-world supervision and real-world SfM self-supervision,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 8, pp. 12 738–12 751, Aug. 2022.
  10. H.-k. Chiu, E. Adeli, and J. C. Niebles, “Segmenting the future,” IEEE Robot. and Autom. Lett., vol. 5, no. 3, pp. 4202–4209, Jul. 2020.
  11. T.-J. Song, J. Jeong, and J.-H. Kim, “End-to-end real-time obstacle detection network for safe self-driving via multi-task learning,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 16 318–16 329, Sep. 2022.
  12. S. Xu, D. Zhou, J. Fang, J. Yin, Z. Bin, and L. Zhang, “FusionPainting: Multimodal fusion with adaptive attention for 3D object detection,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC), Indianapolis, USA, Oct. 2021, pp. 3047–3054.
  13. O. Natan and J. Miura, “Semantic segmentation and depth estimation with RGB and DVS sensor fusion for multi-view driving perception,” in Proc. Asian Conf. Pattern Recog. (ACPR), Jeju Island, South Korea, Nov. 2021, pp. 352–365.
  14. L. Sun, K. Yang, X. Hu, W. Hu, and K. Wang, “Real-time fusion network for RGB-D semantic segmentation incorporating unexpected obstacle detection for road-driving images,” IEEE Robot. and Autom. Lett., vol. 5, no. 4, pp. 5558–5565, Oct. 2020.
  15. O. Natan and J. Miura, “DeepIPC: Deeply integrated perception and control for an autonomous vehicle in real environments,” IEEE Access, Apr. 2024. [Online]. Available: https://doi.org/10.1109/ACCESS.2024.3385122
  16. Z. He, X. Fan, Y. Peng, Z. Shen, J. Jiao, and M. Liu, “EmPointMovSeg: Sparse tensor-based moving-object segmentation in 3-D LiDAR point clouds for autonomous driving-embedded system,” IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst., vol. 42, no. 1, pp. 41–53, Jan. 2023.
  17. S. Zhou, H. Xu, G. Zhang, T. Ma, and Y. Yang, “Leveraging deep convolutional neural networks pre-trained on autonomous driving data for vehicle detection from roadside LiDAR data,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 11, pp. 22 367–22 377, Nov. 2022.
  18. G. Xian, C. Ji, L. Zhou, G. Chen, J. Zhang, B. Li, X. Xue, and J. Pu, “Location-guided LiDAR-based panoptic segmentation for autonomous driving,” IEEE Trans. Intell. Veh., vol. 8, no. 2, pp. 1473–1483, Feb. 2023.
  19. R. W. Wolcott and R. M. Eustice, “Robust LiDAR localization using multiresolution Gaussian mixture maps for autonomous driving,” Int. J. Rob. Res., vol. 36, no. 3, pp. 292–319, Apr. 2017.
  20. S. McCrae and A. Zakhor, “3D object detection for autonomous driving using temporal LiDAR data,” in Proc. Inter. Conf. Image Processing (ICIP), Abu Dhabi, UAE, Oct. 2020, pp. 2661–2665.
  21. Y. Li, L. Ma, Z. Zhong, F. Liu, M. A. Chapman, D. Cao, and J. Li, “Deep learning for LiDAR point clouds in autonomous driving: A review,” IEEE Trans. Neural Networks and Learning Syst., vol. 32, no. 8, pp. 3412–3432, Aug. 2021.
  22. Y. Li and J. Ibanez-Guzman, “LiDAR for autonomous driving: The principles, challenges, and trends for automotive LiDAR and perception systems,” IEEE Signal Process. Mag., vol. 37, no. 4, pp. 50–61, Jul. 2020.
  23. J. Behley, M. Garbade, A. Milioto, J. Quenzel, S. Behnke, J. Gall, and C. Stachniss, “Towards 3D LiDAR-based semantic scene understanding of 3D point cloud sequences: The SemanticKITTI Dataset,” The International Journal on Robotics Research, vol. 40, no. 8-9, pp. 959–967, Apr. 2021.
  24. X. Yan, J. Gao, C. Zheng, C. Zheng, R. Zhang, S. Cui, and Z. Li, “2DPASS: 2D priors assisted semantic segmentation on LiDAR point clouds,” in Proc. European Conf. Comput. Vision (ECCV), Tel Aviv, Israel, Oct. 2022, pp. 677–695.
  25. Y. Hou, X. Zhu, Y. Ma, C. C. Loy, and Y. Li, “Point-to-voxel knowledge distillation for LiDAR semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recog. (CVPR), New Orleans, USA, Jun. 2022, pp. 8469–8478.
  26. Y. Zhang, Z. Zhou, P. David, X. Yue, Z. Xi, B. Gong, and H. Foroosh, “PolarNet: An improved grid representation for online LiDAR point clouds semantic segmentation,” in Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recog. (CVPR), Seattle, USA, Jun. 2020, pp. 9598–9607.
  27. A. Tampuu, T. Matiisen, M. Semikin, D. Fishman, and N. Muhammad, “A survey of end-to-end driving: Architectures and training methods,” IEEE Trans. Neural Networks and Learning Syst., vol. 33, no. 4, pp. 1364–1384, Apr. 2022.
  28. S. Teng, L. Chen, Y. Ai, Y. Zhou, Z. Xuanyuan, and X. Hu, “Hierarchical interpretable imitation learning for end-to-end autonomous driving,” IEEE Trans. Intell. Veh., vol. 8, no. 1, pp. 673–683, Jan. 2023.
  29. K. Ishihara, A. Kanervisto, J. Miura, and V. Hautamaki, “Multi-task learning with attention for end-to-end autonomous driving,” in Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recog. Workshops (CVPRW), Nashville, USA, Jun. 2021, pp. 2896–2905.
  30. E. Kargar and V. Kyrki, “Increasing the efficiency of policy learning for autonomous vehicles by multi-task representation learning,” IEEE Trans. Intell. Veh., vol. 7, no. 3, pp. 701–710, Sep. 2022.
  31. K. Chitta, A. Prakash, and A. Geiger, “NEAT: Neural attention fields for end-to-end autonomous driving,” in Proc. IEEE/CVF Inter. Conf. Comput. Vision (ICCV), Montreal, Canada, Oct. 2021, pp. 15 773–15 783.
  32. A. Prakash, K. Chitta, and A. Geiger, “Multi-modal fusion transformer for end-to-end autonomous driving,” in Proc. IEEE/CVF Conf. Comput. Vision and Pattern Recog. (CVPR), Nashville, USA, Jun. 2021, pp. 7073–7083.
  33. K. Chitta, A. Prakash, B. Jaeger, Z. Yu, K. Renz, and A. Geiger, “TransFuser: Imitation with transformer-based sensor fusion for autonomous driving,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 11, pp. 12 878 – 12 895, Nov. 2023.
  34. N. Rhinehart, R. Mcallister, K. Kitani, and S. Levine, “PRECOG: Prediction conditioned on goals in visual multi-agent settings,” in Proc. IEEE/CVF Inter. Conf. Comput. Vision (ICCV), Seoul, South Korea, Nov. 2019, pp. 2821–2830.
  35. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834–848, Apr. 2018.
  36. O. Natan and J. Miura, “End-to-end autonomous driving with semantic depth cloud mapping and multi-agent,” IEEE Trans. Intell. Veh., vol. 8, no. 1, pp. 557–571, Jan. 2022.
  37. K. Cho, B. van Merrienboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder-decoder approaches,” in Proc. Workshop Syntax, Semantics and Structure in Statistical Translation (SSST), Doha, Qatar, Oct. 2014, pp. 103–111.
  38. O. Natan and J. Miura, “Towards compact autonomous driving perception with balanced learning and multi-sensor fusion,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 9, pp. 16 249–16 266, Sep. 2022.
  39. Z. Huang, C. Lv, Y. Xing, and J. Wu, “Multi-modal sensor fusion-based deep neural network for end-to-end autonomous driving with scene understanding,” IEEE Sensors J., vol. 21, no. 10, pp. 11 781–11 790, May 2021.
  40. F. S. Acerbo, M. Alirczaei, H. Van Der Auweraer, and T. D. Son, “Safe imitation learning on real-life highway data for human-like autonomous driving,” in Proc. IEEE Intell. Transp. Syst. Conf. (ITSC), Indianapolis, USA, Sep. 2021, pp. 3903–3908.
  41. D. Sun, Q. Liao, and A. Loutfi, “Type-2 fuzzy model-based movement primitives for imitation learning,” IEEE Trans. Robot., vol. 38, no. 4, pp. 2462–2480, Aug. 2022.
  42. H. Fujiishi, T. Kobayashi, and K. Sugimoto, “Safe and efficient imitation learning by clarification of experienced latent space,” Adv. Robot., vol. 35, no. 16, pp. 1012–1027, Jul. 2021.
  43. M. Alibeigi, M. N. Ahmadabadi, and B. N. Araabi, “A fast, robust, and incremental model for learning high-level concepts from human motions by imitation,” IEEE Trans. Robot., vol. 33, no. 1, pp. 153–168, Feb. 2017.
  44. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “PyTorch: An imperative style, high performance deep learning library,” in Proc. Inter. Conf. Neural Information Processing Syst. (NIPS), Vancouver, Canada, Dec. 2019, pp. 8024–8035.
  45. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in Proc. Inter. Conf. Learning Representations (ICLR), San Diego, USA, May 2015.
  46. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in Proc. Inter. Conf. Learning Representations (ICLR), New Orleans, USA, May 2019, pp. 1–10.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Oskar Natan
  2. Jun Miura
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - oskarnatan/DeepIPCv2 (8 stars)
Youtube Logo Streamline Icon: https://streamlinehq.com

YouTube