The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery

Abstract: We explore the application of super-resolution techniques to satellite imagery, and the effects of these techniques on object detection algorithm performance. Specifically, we enhance satellite imagery beyond its native resolution, and test if we can identify various types of vehicles, planes, and boats with greater accuracy than native resolution. Using the Very Deep Super-Resolution (VDSR) framework and a custom Random Forest Super-Resolution (RFSR) framework we generate enhancement levels of 2x, 4x, and 8x over five distinct resolutions ranging from 30 cm to 4.8 meters. Using both native and super-resolved data, we then train several custom detection models using the SIMRDWN object detection framework. SIMRDWN combines a number of popular object detection algorithms (e.g. SSD, YOLO) into a unified framework that is designed to rapidly detect objects in large satellite images. This approach allows us to quantify the effects of super-resolution techniques on object detection performance across multiple classes and resolutions. We also quantify the performance of object detection as a function of native resolution and object pixel size. For our test set we note that performance degrades from mean average precision (mAP) = 0.53 at 30 cm resolution, down to mAP = 0.11 at 4.8 m resolution. Super-resolving native 30 cm imagery to 15 cm yields the greatest benefit; a 13-36% improvement in mAP. Super-resolution is less beneficial at coarser resolutions, though still provides a small improvement in performance.

• The paper demonstrates that applying SR methods significantly boosts object detection accuracy with improved mAP scores at finer resolutions.
• It integrates VDSR and RFSR with YOLT and SSD frameworks, highlighting distinct benefits in computational efficiency and detection performance.
• The study underscores practical cost-effective solutions by enhancing the detection of vehicles, planes, and boats in high-resolution satellite images.

The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery

The paper "The Effects of Super-Resolution on Object Detection Performance in Satellite Imagery" investigates the impact of super-resolution (SR) techniques on the performance of object detection algorithms applied to satellite imagery. It proposes a thorough study involving SR methods such as Very Deep Super-Resolution (VDSR) and Random Forest Super-Resolution (RFSR) in conjunction with object detection frameworks like YOLT and SSD. The primary focus is on enhancing the resolution of satellite images to gauge improvements in detecting various objects like vehicles, planes, and boats.

Introduction and Motivation

Super-resolution aims to augment low-resolution images to a higher resolution, often hypothesized to improve the detection of small objects in challenging satellite imagery due to increased distinguishable features. This paper explores whether augmentation beyond native resolution enhances object detection accuracy. Given the high costs associated with satellites, the study speculates on the cost-effectiveness of coupling SR algorithms with smaller satellites that capture coarser resolutions.

Super-Resolution Techniques

VDSR and RFSR Approaches

VDSR, known for its deep convolutional neural network architecture, significantly increases image resolution by transforming low-resolution features into high-resolution outputs using a residual image learning approach. RFSR, on the other hand, is a custom ensemble method leveraging random forest regressors, offering computational simplicity with high inference speeds.

Figure 1: Examples of 15 cm GSD super-resolved output from RFSR and VDSR versus the original 30 cm GSD native imagery.

Training VDSR requires augmented patches from YCbCr-converted images, while RFSR utilizes a residual schema involving shift-based normalization for robust edge enhancement. The paper records substantial training times for VDSR on GPU, contrasting with the CPU-based training efficiency of RFSR.

Figure 2: Ground truth 30 cm imagery (right), and simulated 60 cm input imagery (left).

Object Detection Framework

SIMRDWN Integration

YOLT and SSD models within the SIMRDWN framework facilitate rapid object detection across large-scale satellite images. The study reported superior performance from YOLT due to its deep learning architecture, significantly outperforming SSD in detecting smaller objects.

Figure 3: Example output of YOLT model at native 30 cm resolution. Cars are in green, buses/trucks in blue, and airplanes in orange.

Performance metrics such as mean average precision (mAP) are computed at various resolutions, highlighting the degradation when transitioning from 30 cm to 480 cm resolution.

SR Impact on Object Detection

The study presents comprehensive evaluations indicating significant enhancements in detection performance, particularly when resolving from native 30 cm to 15 cm, using SR methods. Statistical analysis reveals improved mAP scores most pronounced at higher resolutions, demonstrating SR's efficacy in enhancing feature distinction essential for object discrimination.

Figure 4: Performance boost of enhancing 30 and 60 cm imagery to 15 cm GSD.

Nonetheless, SR shows limitations in coarser resolutions, emphasizing the diminishing returns associated with illusory feature recovery from severely degraded inputs.

Conclusion

The paper concludes that SR techniques effectively enhance the detection of objects in satellite imagery at finer resolutions, offering substantial practical implications in satellite imaging. The integration of SR methods into object detection workflows presents a promising approach towards cost-effective satellite data processing and maximization of detection accuracy. Future research could explore end-to-end training paradigms and hybrid models combining SR techniques with novel object detection architectures.