Real-GDSR: Real-World Guided DSM Super-Resolution via Edge-Enhancing Residual Network (2404.03930v1)
Abstract: A low-resolution digital surface model (DSM) features distinctive attributes impacted by noise, sensor limitations and data acquisition conditions, which failed to be replicated using simple interpolation methods like bicubic. This causes super-resolution models trained on synthetic data does not perform effectively on real ones. Training a model on real low and high resolution DSMs pairs is also a challenge because of the lack of information. On the other hand, the existence of other imaging modalities of the same scene can be used to enrich the information needed for large-scale super-resolution. In this work, we introduce a novel methodology to address the intricacies of real-world DSM super-resolution, named REAL-GDSR, breaking down this ill-posed problem into two steps. The first step involves the utilization of a residual local refinement network. This strategic approach departs from conventional methods that trained to directly predict height values instead of the differences (residuals) and utilize large receptive fields in their networks. The second step introduces a diffusion-based technique that enhances the results on a global scale, with a primary focus on smoothing and edge preservation. Our experiments underscore the effectiveness of the proposed method. We conduct a comprehensive evaluation, comparing it to recent state-of-the-art techniques in the domain of real-world DSM super-resolution (SR). Our approach consistently outperforms these existing methods, as evidenced through qualitative and quantitative assessments.
- Detailed digital surface model (dsm) generation and automatic object detection to facilitate modelling of urban flooding. ISPRS J. Photogramm. Remote Sens.
- Adabins: Depth estimation using adaptive bins.
- Dsm-to-lod2: Spaceborne stereo digital surface model refinement. Remote Sensing 10(12), pp. 1926.
- Long-short skip connections in deep neural networks for dsm refinement. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 43(B2), pp. 383–390.
- Toward real-world single image super-resolution: A new benchmark and a new model. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3086–3095.
- Convolutional neural network based dem super resolution. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences 41, pp. 247–250.
- D-srgan: Dem super-resolution with generative adversarial networks. SN Computer Science 2, pp. 1–11.
- Dem super-resolution with efficientnetv2. arXiv preprint arXiv:2109.09661.
- Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, Ieee, pp. 248–255.
- An application of markov random fields to range sensing. Advances in neural information processing systems.
- Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38(2), pp. 295–307.
- Digging into self-supervised monocular depth estimation.
- Unsupervised monocular depth estimation with left-right consistency.
- Generative adversarial nets. Advances in neural information processing systems.
- Automated lod-2 model reconstruction from very-high-resolution satellite-derived digital surface model and orthophoto. ISPRS Journal of Photogrammetry and Remote Sensing 181, pp. 1–19.
- Robust guided image filtering using nonconvex potentials. IEEE transactions on pattern analysis and machine intelligence 40, pp. 192–207.
- Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
- Towards fast and accurate real-world depth super-resolution: Benchmark dataset and baseline. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition pp. 9225–9234.
- Depth map super-resolution by deep multi-scale guidance. pp. 353–369.
- Depth map super-resolution by deep multi-scale guidance. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, Springer, pp. 353–369.
- Model-based multispectral sharpening. Vol. 2231, pp. 60–71.
- Global land use/land cover with sentinel 2 and deep learning. In: 2021 IEEE international geoscience and remote sensing symposium IGARSS, IEEE, pp. 4704–4707.
- A joint intensity and depth co-sparse analysis model for depth map super-resolution. In: Proceedings of the IEEE international conference on computer vision, pp. 1545–1552.
- Deformable kernel networks for joint image filtering. International Journal of Computer Vision 129(2), pp. 579–600.
- Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1646–1654.
- Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1637–1645.
- Joint bilateral upsampling. ACM Transactions on Graphics (ToG) 26(3), pp. 96–es.
- Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4681–4690.
- Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp. 136–144.
- Guided depth enhancement via anisotropic diffusion. In: Advances in Multimedia Information Processing–PCM 2013: 14th Pacific-Rim Conference on Multimedia, Nanjing, China, December 13-16, 2013. Proceedings 14, Springer, pp. 408–417.
- Learning graph regularisation for guided super-resolution. pp. 1979–1988.
- Guided depth super-resolution by deep anisotropic diffusion. pp. 18237–18246.
- Automatic 3-d building model reconstruction from very high resolution stereo satellite imagery. Remote Sensing 11(14), pp. 1660.
- Multispectral band sharpening using pseudoinverse estimation and fuzzy reasoning. Vol. 1693, pp. 170–181.
- Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on pattern analysis and machine intelligence 12(7), pp. 629–639.
- A deep primal-dual network for guided depth super-resolution. arXiv preprint arXiv:1607.08569.
- Atgv-net: Accurate depth super-resolution. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, Springer, pp. 268–284.
- Resdepth: A deep residual prior for 3d reconstruction from high-resolution satellite images. ISPRS Journal of Photogrammetry and Remote Sensing 183, pp. 560–580.
- Implicity: City modeling from satellite images with deep implicit occupancy fields. arXiv preprint arXiv:2201.09968.
- Efficientnetv2: Smaller models and faster training. In: International conference on machine learning, PMLR, pp. 10096–10106.
- Learning joint intensity-depth sparse representations. IEEE Transactions on Image Processing 23(5), pp. 2122–2132.
- Perceptual deep depth super-resolution. Proceedings of the IEEE International Conference on Computer Vision 2019-October, pp. 5652–5662.
- Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 1905–1914.
- Esrgan: Enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision (ECCV) workshops, pp. 0–0.
- Machine-learned 3d building vectorization from satellite imagery. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1072–1081.
- Deep networks for image super-resolution with sparse prior. In: Proceedings of the IEEE international conference on computer vision, pp. 370–378.
- Edge-guided single depth image super resolution. IEEE Transactions on Image Processing 25(1), pp. 428–438.
- Deep gradient prior network for dem super-resolution: Transfer learning from image to dem. ISPRS Journal of Photogrammetry and Remote Sensing 150, pp. 80–90.
- Nonlocal similarity based dem super resolution. ISPRS Journal of Photogrammetry and Remote Sensing 110, pp. 48–54.