Residual Conv-Deconv Grid Network for Semantic Segmentation

Abstract: This paper presents GridNet, a new Convolutional Neural Network (CNN) architecture for semantic image segmentation (full scene labelling). Classical neural networks are implemented as one stream from the input to the output with subsampling operators applied in the stream in order to reduce the feature maps size and to increase the receptive field for the final prediction. However, for semantic image segmentation, where the task consists in providing a semantic class to each pixel of an image, feature maps reduction is harmful because it leads to a resolution loss in the output prediction. To tackle this problem, our GridNet follows a grid pattern allowing multiple interconnected streams to work at different resolutions. We show that our network generalizes many well known networks such as conv-deconv, residual or U-Net networks. GridNet is trained from scratch and achieves competitive results on the Cityscapes dataset.

• The paper introduces a novel GridNet architecture that uses a multi-resolution grid and residual streams for refined semantic segmentation.
• It leverages integrated subsampling and upsampling operations to retain detail and expand the receptive field efficiently.
• Evaluations on the Cityscapes dataset show competitive IoU scores and robust training even without pre-trained weights.

Residual Conv-Deconv Grid Network for Semantic Segmentation

Introduction

The paper introduces GridNet, a novel architecture designed to address the challenges inherent in semantic image segmentation. Unlike traditional networks that follow a singular input-output stream, GridNet is structured as a multi-resolution grid, leveraging interconnected convolutional streams operating at different resolutions. The GridNet’s design purposefully avoids the loss of detail typically associated with subsampling operations in classical convolutional neural networks (CNNs). By enabling multiple resolution streams, GridNet can maintain critical detail throughout its operations, ultimately achieving more precise semantic segmentation.

Approach

GridNet is based on a two-dimensional grid pattern where information flows both horizontally through resolution-preserving residual streams and vertically via down-sampling and up-sampling operations. Key components of the architecture include:

• Residual Streams: Horizontal connections that retain the resolution and enable residual learning for efficient gradient backpropagation.
• Subsampling and Upsampling Operations: Vertical connections that adjust the resolution to provide different levels of detail and context while facilitating the processing of larger receptive fields.

The architecture allows GridNet to generalize existing methods, including conv-deconv networks and U-Net, offering a flexible and comprehensive approach to semantic segmentation tasks.

Evaluation and Performance

GridNet's capabilities were assessed on the Cityscapes dataset, a benchmark collection of urban scene images requiring high-resolution segmentation. The network configuration utilized featured multiple streams with differing resolutions, juxtaposed to balance memory consumption and performance. The evaluation metrics were the Intersection-over-Union (IoU) and instance-level IoU (iIoU), capturing the accuracy of pixel-level predictions.

GridNet demonstrated comparable performance to state-of-the-art techniques, even when trained from scratch—without the usage of pre-trained weights from datasets like ImageNet. The implementation of "total dropout" technique was crucial, which involved the random dropping of streams during training, ensuring all streams in the network contributed effectively to the segmentation task and managed vanishing gradients effectively.

Discussion and Implications

The presented GridNet architecture exhibits significant potential to advance semantic segmentation tasks. By not being reliant on pre-existing models or weight initializations, GridNet offers a level of adaptability and robustness that is attractive for various applications. Furthermore, the study underscores the importance of multi-resolution processing not just for refined pixel classification but also for overall model stability and training efficiency.

From a theoretical perspective, GridNet encourages future exploration into networks with multidimensional architectures where paths can be weighted dynamically depending on the input data's requirements. Practically, this could translate to more efficient use of computational resources and improved model performance on diverse datasets without explicit retraining.

In conclusion, GridNet represents a meaningful contribution to neural network architectures tailored for semantic segmentation, highlighting the benefits of incorporating detailed and context-rich representations across varied resolutions. Future work could explore pre-training strategies and enhanced multi-scale interactions within grids, potentially broadening GridNet's applicability across even more varied computer vision challenges.