Tackling fluffy clouds: robust field boundary delineation across global agricultural landscapes with Sentinel-1 and Sentinel-2 Time Series (2409.13568v2)

Abstract: Accurate delineation of agricultural field boundaries is essential for effective crop monitoring and resource management. However, competing methodologies often face significant challenges, particularly in their reliance on extensive manual efforts for cloud-free data curation and limited adaptability to diverse global conditions. In this paper, we introduce PTAViT3D, a deep learning architecture specifically designed for processing three-dimensional time series of satellite imagery from either Sentinel-1 (S1) or Sentinel-2 (S2). Additionally, we present PTAViT3D-CA, an extension of the PTAViT3D model incorporating cross-attention mechanisms to fuse S1 and S2 datasets, enhancing robustness in cloud-contaminated scenarios. The proposed methods leverage spatio-temporal correlations through a memory-efficient 3D Vision Transformer architecture, facilitating accurate boundary delineation directly from raw, cloud-contaminated imagery. We comprehensively validate our models through extensive testing on various datasets, including Australia's ePaddocks - CSIRO's national agricultural field boundary product - alongside public benchmarks Fields-of-the-World, PASTIS, and AI4SmallFarms. Our results consistently demonstrate state-of-the-art performance, highlighting excellent global transferability and robustness. Crucially, our approach significantly simplifies data preparation workflows by reliably processing cloud-affected imagery, thereby offering strong adaptability across diverse agricultural environments. Our code and models are publicly available at https://github.com/feevos/tfcl.

• The paper introduces PTAViT3D models using a 3D Vision Transformer to delineate field boundaries from Sentinel-1 and Sentinel-2 imagery.
• It achieves robust performance under cloud cover by integrating temporal dynamics with a memory-efficient attention mechanism.
• The method outperforms traditional techniques, demonstrating scalability and multisensor fusion benefits in digital agriculture.

A Novel Strategy for Field Boundary Detection with Sentinel-1 and Sentinel-2 Imagery

The paper presents an advanced methodology for field boundary delineation in digital agriculture using satellite imagery from Sentinel-1 (S1) and Sentinel-2 (S2). The proposed framework addresses the challenges posed by cloud coverage in optical remote sensing, which often hampers the accuracy of existing field boundary detection techniques.

Methodological Advancements

This work leverages time series data from both S1 and S2 to introduce two models utilizing a 3D Vision Transformer architecture: PTAViT3D and PTAViT3D-CA. The former processes either S2 or S1 imagery independently, while the latter fuses data from both sources via cross-attention. The models are evaluated under varying conditions of cloud cover, illustrating their robustness and versatility in extracting field boundaries.

A noteworthy element is the employment of a memory-efficient attention mechanism within a novel 3D Vision Transformer architecture, demonstrating potential improvements over traditional deep learning approaches. The inclusion of temporal dynamics allows for effective handling of cloud interference, setting it apart from prior models, which often rely on manual pre-processing to mitigate cloud effects.

Significant Results and Practical Implications

The experimental results highlight the models' ability to maintain high accuracy in field boundary delineation even under substantial cloud cover. The S1-based model is commensurate in performance with S2 imagery, providing a valuable alternative for regions where cloud coverage is more persistent. The method shows that combining S1's cloud-penetrating SAR capabilities with S2's optical data improves the model's performance, underlining the advantage of multisensor fusion in remote sensing tasks.

Extensive empirical evaluations are presented, showing that the proposed models outperform older methodologies, such as the FracTAL ResNet, by effectively utilizing time series data for semantic segmentation. The practical implications for digital agriculture are substantial, offering scalability and adaptability for diverse agricultural environments, as evidenced by the application of this approach in mapping national field boundaries in Australia through the ePaddocks product.

Future Directions

The paper opens avenues for further explorations in AI-driven remote sensing applications. Prospective developments could involve extending this methodology to crop type classification, integrating geographical information, and enhancing instance segmentation techniques. Direct polygon inference from input imagery is proposed as a natural extension to reduce the reliance on secondary post-processing steps.

Overall, the paper contributes a comprehensive strategy to overcome traditional challenges in field boundary detection, providing a scalable and robust solution for digital agriculture. This work underscores the transformative potential of time series analysis combined with sophisticated neural architectures for advancing remote sensing applications.