SSIN: Self-Supervised Learning for Rainfall Spatial Interpolation (2311.15530v1)
Abstract: The acquisition of accurate rainfall distribution in space is an important task in hydrological analysis and natural disaster pre-warning. However, it is impossible to install rain gauges on every corner. Spatial interpolation is a common way to infer rainfall distribution based on available raingauge data. However, the existing works rely on some unrealistic pre-settings to capture spatial correlations, which limits their performance in real scenarios. To tackle this issue, we propose the SSIN, which is a novel data-driven self-supervised learning framework for rainfall spatial interpolation by mining latent spatial patterns from historical observation data. Inspired by the Cloze task and BERT, we fully consider the characteristics of spatial interpolation and design the SpaFormer model based on the Transformer architecture as the core of SSIN. Our main idea is: by constructing rich self-supervision signals via random masking, SpaFormer can learn informative embeddings for raw data and then adaptively model spatial correlations based on rainfall spatial context. Extensive experiments on two real-world raingauge datasets show that our method outperforms the state-of-the-art solutions. In addition, we take traffic spatial interpolation as another use case to further explore the performance of our method, and SpaFormer achieves the best performance on one large real-world traffic dataset, which further confirms the effectiveness and generality of our method.
- Kriging convolutional networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 3187–3194.
- Data2vec: A general framework for self-supervised learning in speech, vision and language. arXiv preprint arXiv:2202.03555 (2022).
- Feng-Wen Chen and Chen-Wuing Liu. 2012. Estimation of the spatial rainfall distribution using inverse distance weighting (IDW) in the middle of Taiwan. Paddy and Water Environment 10, 3 (2012), 209–222.
- {{\{{TVM}}\}}: An automated {{\{{End-to-End}}\}} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 578–594.
- Leila De Floriani and Paola Magillo. 2018. Triangulated Irregular Network. Springer New York. https://doi.org/10.1007/978-1-4614-8265-9_437
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171–4186.
- High-resolution studies of rainfall on Norfolk Island: Part II: Interpolation of rainfall data. Journal of Hydrology 208, 3-4 (1998), 187–193.
- Relationships between natural terrain landslide magnitudes and triggering rainfall based on a large landslide inventory in Hong Kong. Landslides 15, 4 (2018), 727–740.
- DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In IJCAI. 1725–1731.
- Michael F Hutchinson. 1995. Interpolating mean rainfall using thin plate smoothing splines. Int. J. Geogr. Inf. Syst. 9, 4 (1995), 385–403.
- Sharon A Jewell and Nicolas Gaussiat. 2015. An assessment of kriging-based rain-gauge–radar merging techniques. Q. J. R. Meteorol. Soc. (2015).
- Rethinking Positional Encoding in Language Pre-training. In ICLR.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR.
- Efficient Self-supervised Vision Transformers for Representation Learning. In ICLR.
- Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting. In ICLR.
- Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019).
- Different methods for spatial interpolation of rainfall data for operational hydrology and hydrological modeling at watershed scale: a review. Biotechnologie, Agronomie, Société et Environnement (2013).
- J Eamonn Nash and Jonh V Sutcliffe. 1970. River flow forecasting through conceptual models part I—A discussion of principles. J. Hydrol. (1970).
- Self-Attention with Relative Position Representations. In NAACL-HLT. 464–468.
- Real-time radar–rain-gauge merging using spatio-temporal co-kriging with external drift in the alpine terrain of Switzerland. Quarterly Journal of the Royal Meteorological Society 140, 680 (2014), 1097–1111.
- Raymond Sluiter. 2009. Interpolation methods for climate data: literature review. KNMI, De Bilt (2009).
- AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks. In CIKM. 1161–1170.
- PreQR: Pre-training Representation for SQL Understanding. In SIGMOD. 204–216.
- Immanuel Trummer. 2022. DB-BERT: a Database Tuning Tool that ”Reads the Manual”. In SIGMOD. 190–203.
- Attention is all you need. NeurIPS (2017), 5998–6008.
- Attention is All you Need. In NeurIPS. 5998–6008.
- Hans Wackernagel. 1995a. Ordinary Kriging. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-662-03098-1_11
- Hans Wackernagel. 1995b. Universal Kriging. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-662-03098-1_28
- Max Welling and Thomas N Kipf. 2017. Semi-supervised classification with graph convolutional networks. In ICLR.
- Impact of Scientific Scrutiny after the 2016 Braunsbach Flash Flood on Flood-Risk Management in the State of Baden-Württemberg, Germany. Water 12, 4 (2020), 1165.
- Inductive Graph Neural Networks for Spatiotemporal Kriging. In AAAI.
- S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In CIKM. 1893–1902.