PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation (2302.09746v1)

Abstract: Spatiotemporal data mining plays an important role in air quality monitoring, crowd flow modeling, and climate forecasting. However, the originally collected spatiotemporal data in real-world scenarios is usually incomplete due to sensor failures or transmission loss. Spatiotemporal imputation aims to fill the missing values according to the observed values and the underlying spatiotemporal dependence of them. The previous dominant models impute missing values autoregressively and suffer from the problem of error accumulation. As emerging powerful generative models, the diffusion probabilistic models can be adopted to impute missing values conditioned by observations and avoid inferring missing values from inaccurate historical imputation. However, the construction and utilization of conditional information are inevitable challenges when applying diffusion models to spatiotemporal imputation. To address above issues, we propose a conditional diffusion framework for spatiotemporal imputation with enhanced prior modeling, named PriSTI. Our proposed framework provides a conditional feature extraction module first to extract the coarse yet effective spatiotemporal dependencies from conditional information as the global context prior. Then, a noise estimation module transforms random noise to realistic values, with the spatiotemporal attention weights calculated by the conditional feature, as well as the consideration of geographic relationships. PriSTI outperforms existing imputation methods in various missing patterns of different real-world spatiotemporal data, and effectively handles scenarios such as high missing rates and sensor failure. The implementation code is available at https://github.com/LMZZML/PriSTI.

• The paper introduces PriSTI, a framework that employs conditional diffusion models to overcome missing data challenges in spatiotemporal datasets.
• It uses a conditional feature extraction module with spatiotemporal attention and a noise estimation module to transform Gaussian noise into realistic values.
• Empirical evaluations on datasets like METR-LA and PEMS-BAY highlight robust imputation performance, especially under extreme missing rates.

PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation

The paper "PriSTI: A Conditional Diffusion Framework for Spatiotemporal Imputation" presents a novel approach to address the prevalent issue of missing data in spatiotemporal datasets due to sensor failures or transmission losses. The authors propose PriSTI, a framework leveraging conditional diffusion models that focus on overcoming the limitations of autoregressive models such as error accumulation.

Overview

Spatiotemporal imputation involves estimating missing values in datasets characterized by intrinsic spatial and temporal patterns, crucial for applications in air quality, traffic flow forecasting, and climate prediction. Traditional methods rely heavily on assumptions like temporal smoothness and spatial similarity, often resulting in error accumulation and limited performance when these assumptions do not hold. PriSTI addresses these challenges by employing diffusion probabilistic models, which leverage observed values and spatiotemporal dependencies without the defects of autoregressive techniques.

Contributions

The paper's principal contribution lies in the introduction of PriSTI, which is a conditional diffusion framework that utilizes enhanced prior modeling for spatiotemporal imputation. The approach comprises two main components:

1. Conditional Feature Extraction Module: This module extracts spatiotemporal dependencies from interpolated conditional information to create a global context prior. It employs temporal and spatial attention mechanisms alongside message passing neural networks (MPNNs) to incorporate spatial correlations and geographic information.
2. Noise Estimation Module: Designed to transform Gaussian noise into realistic values, this module leverages the spatiotemporal weights derived from the conditional features. It emphasizes learning the dependencies while mitigating the impact of noise inherent in the diffusion process.

Together, these modules enable PriSTI to effectively infer missing values across different real-world scenarios, outperforming existing models under various missing patterns and settings, notably in situations with high missing rates and sensor failures.

Results

Empirical evaluations on datasets such as AQI-36, METR-LA, and PEMS-BAY demonstrate PriSTI's superior imputation performance over traditional statistical methods, matrix factorization techniques, and even state-of-the-art deep learning models like GRIN and BRITS. Notably, PriSTI exhibits robust performance even under extreme conditions of high missing rates, proving the efficacy of its design and the pivotal role of the constructed conditional information and extracted dependencies.

Implications and Future Directions

The implications of this research are multifaceted, with direct applications in improving data completeness for downstream tasks such as forecasting and anomaly detection. The approach paves the way for the adoption of diffusion models in domains requiring robust handling of missing data, challenging the dominance of autoregressive models in multivariate time series imputation.

Future work may focus on enhancing the scalability and efficiency of PriSTI, particularly on larger spatiotemporal datasets. Additionally, exploring its capabilities in imputation tasks involving more complex spatial structures or dynamic graphs could further solidify its role as a critical tool in spatiotemporal data mining.

Overall, PriSTI embodies a significant advancement in the utilization of generative models for spatiotemporal data imputation, offering a promising alternative to conventional methods hampered by error accumulation and restrictive assumptions.