RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery

Abstract: Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles. These multi-stage methods are not end-to-end trainable, and suffer from poor overall results. We propose RescueNet, a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-toend. In order to to model the composite nature of this problem, we propose a novel localization aware loss function, which consists of a Binary Cross Entropy loss for building segmentation, and a foreground only selective Categorical Cross-Entropy loss for damage classification, and show significant improvement over the widely used Cross-Entropy loss. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods and achieves generalization across varied geographical regions and disaster types.

• The paper introduces a unified, end-to-end model that integrates building segmentation and damage assessment using a novel localization-aware loss.
• It employs a Dilated ResNet backbone with ASPP for efficient multi-scale feature extraction, significantly improving accuracy over baseline methods.
• Experimental results on the xBD dataset demonstrate superior performance in building localization and damage prediction for effective disaster response.

RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery

Introduction

"RescueNet: Joint Building Segmentation and Damage Assessment from Satellite Imagery" presents a unified model aimed at streamlining the process of building segmentation and damage assessment from satellite imagery. Unlike traditional multi-stage approaches, RescueNet integrates these tasks into a single end-to-end trainable model, optimizing for both precision and computational efficiency. This research offers significant improvements in coordinating Humanitarian Aid and Disaster Response (HADR) operations following natural disasters. RescueNet leverages a novel localization aware loss function alongside an innovative network architecture to advance the capabilities of existing systems.

Figure 1: Samples from the xBD dataset showcase pre- and post-disaster imagery for building damage assessment.

Model Architecture

RescueNet's architecture is designed to simultaneously address building segmentation and damage assessment within the same framework. It incorporates state-of-the-art techniques in neural network design, employing a Dilated ResNet backbone to manage the complex task of multi-scale feature extraction without sacrificing spatial resolution (Figure 2). The design enables the model to operate efficiently on high-resolution satellite imagery, making it suitable for real-time deployment in disaster-stricken regions.

Figure 2: RescueNet model architecture employing a Dilated ResNet backbone for efficient feature extraction at high resolution.

Image Feature Extraction

The model extracts features using a ResNet50 backbone, modified with dilated convolutions starting from the second block. This ensures that the network can output features at a spatial resolution reduced by a factor of eight while preserving the detail necessary for accurate segmentation and assessment. The ASPP module is incorporated to facilitate multi-scale feature extraction, critical for handling diverse damage patterns across different regions and disaster types.

Segmentation and Change Detection

Two architectures are evaluated for segmentation: a simple convolutional design and a more complex encoder-decoder configuration. The encoder-decoder architecture, featuring skip connections, outperformed the simpler design, allowing for more nuanced segmentation results. Concurrently, a change detection head processes the difference between pre- and post-disaster features, emphasizing the semantic feature space's relevance in determining damage levels.

Loss Function Design

RescueNet introduces a localization aware loss function alongside a selective Categorical Cross-Entropy loss for effective model training. This approach acknowledges the hierarchical nature of the task, where building localization serves as a prerequisite for precise damage classification. The integration of Dice Loss further enhances segmentation accuracy, particularly concerning building boundary predictions.

Dataset Utilization

The xBD dataset is pivotal for training and evaluating RescueNet. It provides comprehensive building segmentation and damage assessment labels across 19 diverse disaster events worldwide. This dataset's scale and variety facilitate robust benchmarking, enabling RescueNet to demonstrate substantial improvements in both localization and damage assessment compared to existing methodologies.

Experimental Results

RescueNet exhibits superior performance across various metrics, including the XView Challenge score and class-specific F1 scores. The model's qualitative results indicate significant advancements in accurate damage level prediction and error reduction compared to baseline models (Figure 3). Ablation studies validate the positive contribution of each architectural component and loss function choice, underscoring RescueNet's thoughtful design and efficacy.

Figure 3: Qualitative results demonstrating RescueNet's accurate predictions versus baseline models across diverse disaster scenarios.

Conclusion

RescueNet successfully integrates building segmentation and damage assessment within a unified framework, demonstrating superior performance and generalization capabilities across different geographic regions and disaster types. The model's end-to-end trainability, coupled with a bespoke loss function, addresses existing vulnerabilities in prior methods, paving the way for more efficient HADR operations. Future work will target enhancing instance resolution to further refine segmentation and assessment accuracy.

This paper builds upon prior research, proposing a cohesive model that substantially improves the speed and accuracy of damage assessment following natural disasters, presenting implications for both academic inquiry and practical application in HADR contexts. The acknowledgment section thanks contributors to the XView2 Challenge, which inspired this research direction.