Sparse Laneformer (2404.07821v1)
Abstract: Lane detection is a fundamental task in autonomous driving, and has achieved great progress as deep learning emerges. Previous anchor-based methods often design dense anchors, which highly depend on the training dataset and remain fixed during inference. We analyze that dense anchors are not necessary for lane detection, and propose a transformer-based lane detection framework based on a sparse anchor mechanism. To this end, we generate sparse anchors with position-aware lane queries and angle queries instead of traditional explicit anchors. We adopt Horizontal Perceptual Attention (HPA) to aggregate the lane features along the horizontal direction, and adopt Lane-Angle Cross Attention (LACA) to perform interactions between lane queries and angle queries. We also propose Lane Perceptual Attention (LPA) based on deformable cross attention to further refine the lane predictions. Our method, named Sparse Laneformer, is easy-to-implement and end-to-end trainable. Extensive experiments demonstrate that Sparse Laneformer performs favorably against the state-of-the-art methods, e.g., surpassing Laneformer by 3.0% F1 score and O2SFormer by 0.7% F1 score with fewer MACs on CULane with the same ResNet-34 backbone.
- R. F. Berriel, E. de Aguiar, A. F. De Souza, and T. Oliveira-Santos, “Ego-lane analysis system (elas): Dataset and algorithms,” Image and Vision Computing, vol. 68, pp. 64–75, 2017.
- A. A. Assidiq, O. O. Khalifa, M. R. Islam, and S. Khan, “Real time lane detection for autonomous vehicles,” in International Conference on Computer and Communication Engineering. IEEE, 2008, pp. 82–88.
- Z. Qu, H. Jin, Y. Zhou, Z. Yang, and W. Zhang, “Focus on local: Detecting lane marker from bottom up via key point,” in CVPR, 2021, pp. 14 122–14 130.
- J. Wang, Y. Ma, S. Huang, T. Hui, F. Wang, C. Qian, and T. Zhang, “A keypoint-based global association network for lane detection,” in CVPR, 2022, pp. 1392–1401.
- L. Tabelini, R. Berriel, T. M. P. ao, C. Badue, A. F. D. Souza, and T. Oliveira-Santos, “Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection,” in CVPR, 2021.
- X. Li, J. Li, X. Hu, and J. Yang, “Line-cnn: End-to-end traffic line detection with line proposal unit,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 1, pp. 248–258, 2020.
- A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally, “Scnn: An accelerator for compressed-sparse convolutional neural networks,” ACM SIGARCH computer architecture news, vol. 45, no. 2, pp. 27–40, 2017.
- H. Abualsaud, S. Liu, D. Lu, K. Situ, A. Rangesh, and M. M. Trivedi, “Laneaf: Robust multi-lane detection with affinity fields,” arXiv preprint arXiv:2103.12040, 2021.
- L. Tabelini, R. Berriel, T. M. Paixao, C. Badue, A. F. De Souza, and T. Oliveira-Santos, “Polylanenet: Lane estimation via deep polynomial regression,” in ICPR. IEEE, 2021, pp. 6150–6156.
- J. Su, C. Chen, K. Zhang, J. Luo, X. Wei, and X. Wei, “Structure guided lane detection,” arXiv preprint arXiv:2105.05403, 2021.
- Z. Qin, H. Wang, and X. Li, “Ultra fast structure-aware deep lane detection,” in ECCV. Springer, 2020, pp. 276–291.
- N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV. Springer, 2020, pp. 213–229.
- X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,” arXiv preprint arXiv:2010.04159, 2020.
- T. Zheng, H. Fang, Y. Zhang, W. Tang, Z. Yang, H. Liu, and D. Cai, “Resa: Recurrent feature-shift aggregator for lane detection,” in AAAI, vol. 35, no. 4, 2021, pp. 3547–3554.
- H. Xu, S. Wang, X. Cai, W. Zhang, X. Liang, and Z. Li, “Curvelane-nas: Unifying lane-sensitive architecture search and adaptive point blending,” in ECCV. Springer, 2020, pp. 689–704.
- L. Liu, X. Chen, S. Zhu, and P. Tan, “Condlanenet: A top-to-down lane detection framework based on conditional convolution,” in ICCV, October 2021, pp. 3773–3782.
- R. Liu, Z. Yuan, T. Liu, and Z. Xiong, “End-to-end lane shape prediction with transformers,” in WACV, 2021, pp. 3694–3702.
- T. Zheng, Y. Huang, Y. Liu, W. Tang, Z. Yang, D. Cai, and X. He, “Clrnet: Cross layer refinement network for lane detection,” in CVPR, 2022, pp. 898–907.
- J. Han, X. Deng, X. Cai, Z. Yang, H. Xu, C. Xu, and X. Liang, “Laneformer: Object-aware row-column transformers for lane detection,” in AAAI. AAAI Press, 2022, pp. 799–807. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/19961
- K. Zhou and R. Zhou, “End to end lane detection with one-to-several transformer,” arXiv preprint arXiv:2305.00675, 2023.
- P. Sun, R. Zhang, Y. Jiang, T. Kong, C. Xu, W. Zhan, M. Tomizuka, L. Li, Z. Yuan, C. Wang, et al., “Sparse r-cnn: End-to-end object detection with learnable proposals,” in CVPR, 2021, pp. 14 454–14 463.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
- L. Chen, C. Sima, Y. Li, Z. Zheng, J. Xu, X. Geng, H. Li, C. He, J. Shi, Y. Qiao, and J. Yan, “Persformer: 3d lane detection via perspective transformer and the openlane benchmark,” in ECCV, 2022.
- Z. Chen, Q. Liu, and C. Lian, “Pointlanenet: Efficient end-to-end cnns for accurate real-time lane detection,” in 2019 IEEE Intelligent Vehicles Symposium (IV), 2019, pp. 2563–2568.
- M. Lee, J. Lee, D. Lee, W. Kim, S. Hwang, and S. Lee, “Robust lane detection via expanded self attention,” arXiv preprint arXiv:2102.07037, 2021.
- X. Pan, J. Shi, P. Luo, X. Wang, and X. Tang, “Spatial as deep: Spatial cnn for traffic scene understanding,” in AAAI, vol. 32, no. 1, 2018.
- Tusimple, “Tusimple benchmark,” https://github. com/TuSimple/tusimple-benchmark/, Accessed September, 2020.
- K. Behrendt and R. Soussan, “Unsupervised labeled lane markers using maps,” in Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019, pp. 0–0.
- J. Philion, “Fastdraw: Addressing the long tail of lane detection by adapting a sequential prediction network,” in CVPR, 2019, pp. 11 582–11 591.
- M. Ghafoorian, C. Nugteren, N. Baka, O. Booij, and M. Hofmann, “El-gan: Embedding loss driven generative adversarial networks for lane detection,” in ECCV Workshops, 2018, pp. 0–0.
- Y. Hou, Z. Ma, C. Liu, and C. C. Loy, “Learning lightweight lane detection cnns by self attention distillation,” in ICCV, 2019, pp. 1013–1021.
- L. Liu, X. Chen, S. Zhu, and P. Tan, “Condlanenet: a top-to-down lane detection framework based on conditional convolution,” in CVPR, 2021, pp. 3773–3782.