Spatial As Deep: Spatial CNN for Traffic Scene Understanding (1712.06080v1)

Abstract: Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-byslice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8.7% and 4.6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96.53%.

• The paper introduces SCNN, a novel CNN that propagates spatial information across feature maps, improving lane detection accuracy (96.53% on TuSimple).
• It employs a slice-by-slice convolution approach that reduces computational redundancy and enhances training via residual message passing.
• The study presents a large-scale lane detection dataset with over 133K annotated frames, setting a robust benchmark for autonomous driving research.

Spatial As Deep: Spatial CNN for Traffic Scene Understanding

The paper "Spatial As Deep: Spatial CNN for Traffic Scene Understanding" introduces a novel convolutional neural network (CNN) architecture, Spatial CNN (SCNN), aimed at improving performance in tasks such as lane detection and semantic segmentation within traffic scenes. This new architecture leverages spatial relationship propagation across rows and columns of feature maps within a layer, filling a critical gap in traditional CNNs that excel at extracting semantic features but often lack effective mechanisms for capturing spatial relationships.

Contributions

1. SCNN Architecture: The SCNN generalizes traditional deep layer-by-layer convolutions to slice-by-slice convolutions. By treating rows or columns of feature maps as layers and applying convolutions and nonlinear activations sequentially across these slices, SCNN enables the passing of messages between pixels within a single layer. This architectural innovation enhances the network's ability to capture and propagate spatial relationships crucial for tasks involving continuous shape structures such as traffic lanes and poles.
2. Large-Scale Lane Detection Dataset: The authors present an extensive and challenging lane detection dataset collected in urban and rural environments, capturing various scenarios, including occlusions and faded lane markings. This dataset contains over 133,000 annotated frames, vastly exceeding the size of existing public datasets and providing a robust benchmark for evaluating lane detection algorithms.
3. Experimental Results and Benchmarks: SCNN demonstrates significant improvements over several state-of-the-art methods. Specifically, in a challenging lane detection dataset, SCNN outperforms ReNet and MRFNet by 8.7% and 4.6%, respectively. Furthermore, SCNN achieves first place in the TuSimple Benchmark Lane Detection Challenge with an accuracy of 96.53%. In semantic segmentation tasks on the Cityscapes dataset, incorporating SCNN improves mean Intersection-over-Union (mIoU) scores by approximately 1.2% for LargeFOV and 0.8% for ResNet-101 baselines.

Key Findings

• Enhanced Computational Efficiency: SCNN's sequential propagation significantly reduces computational redundancy compared to dense Conditional Random Fields (CRFs), resulting in over fourfold speed improvements in runtime.
• Residual Message Passing: Incorporating messages as residuals facilitates gradient descent optimization, making SCNN easier to train than models using complex gate mechanisms like LSTMs.
• Flexibility in Network Integration: SCNN can be integrated into any layer of a deep neural network, though it demonstrates superior performance when applied to the top hidden layers, which contain richer semantic information.

Implications and Future Directions

• Improvement in Autonomous Systems: The SCNN architecture's ability to capture and propagate spatial information can significantly enhance the performance and reliability of autonomous driving systems, particularly under challenging conditions such as occlusions and adverse lighting.
• Application to Other Domains: Beyond traffic scene understanding, the SCNN architecture could be beneficial in any domain requiring the preservation of spatial continuity, suggesting potential applications in medical imaging, remote sensing, and more.
• Extended Research on Traffic Scene Datasets: The newly introduced lane detection dataset sets a precedent for the collection and annotation of large-scale, real-world data, which is vital for advancing autonomous driving research. Future work could involve extending this dataset to include more diverse scenarios and refining annotation techniques to capture more complex urban traffic patterns.

Conclusion

The SCNN architecture presented in this paper addresses a critical limitation of traditional CNNs in traffic scene understanding by effectively modeling spatial relationships within feature maps. The empirical results demonstrate the architecture's superior performance in lane detection and semantic segmentation tasks. This work sets a promising foundation for future research in autonomous driving and related fields by highlighting the importance of spatial information propagation and providing a robust, publicly available dataset for benchmarking.