Scalable Spatiotemporally Varying Coefficient Modelling with Bayesian Kernelized Tensor Regression (2109.00046v4)
Abstract: As a regression technique in spatial statistics, the spatiotemporally varying coefficient model (STVC) is an important tool for discovering nonstationary and interpretable response-covariate associations over both space and time. However, it is difficult to apply STVC for large-scale spatiotemporal analyses due to its high computational cost. To address this challenge, we summarize the spatiotemporally varying coefficients using a third-order tensor structure and propose to reformulate the spatiotemporally varying coefficient model as a special low-rank tensor regression problem. The low-rank decomposition can effectively model the global patterns of large data sets with a substantially reduced number of parameters. To further incorporate the local spatiotemporal dependencies, we use Gaussian process (GP) priors on the spatial and temporal factor matrices. We refer to the overall framework as Bayesian Kernelized Tensor Regression (BKTR), and kernelized tensor factorization can be considered a new and scalable approach to modeling multivariate spatiotemporal processes with a low-rank covariance structure. For model inference, we develop an efficient Markov chain Monte Carlo (MCMC) algorithm, which uses Gibbs sampling to update factor matrices and slice sampling to update kernel hyperparameters. We conduct extensive experiments on both synthetic and real-world data sets, and our results confirm the superior performance and efficiency of BKTR for model estimation and parameter inference.
- “Kernels for Vector-Valued Functions: A Review.” Foundations and Trends® in Machine Learning, 4(3): 195–266.
- “Fast Multivariate Spatio-temporal Analysis via Low Rank Tensor Learning.” Advances in Neural Information Processing Systems, 3491–3499.
- Hierarchical modeling and analysis for spatial data. CRC press.
- “Gaussian predictive process models for large spatial data sets.” Journal of the Royal Statistical Society Series B: Statistical Methodology, 70(4): 825–848.
- “Multi-task Gaussian Process prediction.” Advances in Neural Information Processing Systems, 153–160.
- “Fixed rank kriging for very large spatial data sets.” Journal of the Royal Statistical Society Series B: Statistical Methodology, 70(1): 209–226.
- “Basis-function models in spatial statistics.” Annual Review of Statistics and Its Application, 9: 373–400.
- Statistics for spatio-temporal data. John Wiley & Sons.
- “Hierarchical nearest-neighbor Gaussian process models for large geostatistical datasets.” Journal of the American Statistical Association, 111(514): 800–812.
- “How land-use and urban form impact bicycle flows: evidence from the bicycle-sharing system (BIXI) in Montreal.” Journal of Transport Geography, 41: 306–314.
- Finley, A. O. (2011). “Comparing spatially-varying coefficients models for analysis of ecological data with non-stationary and anisotropic residual dependence.” Methods in Ecology and Evolution, 2(2): 143–154.
- “Bayesian spatially varying coefficient models in the spBayes R package.” Environmental Modelling & Software, 125: 104608.
- “Efficient algorithms for Bayesian nearest neighbor Gaussian processes.” Journal of Computational and Graphical Statistics, 28(2): 401–414.
- Geographically weighted regression: the analysis of spatially varying relationships. John Wiley & Sons.
- “Spatial modeling with spatially varying coefficient processes.” Journal of the American Statistical Association, 98(462): 387–396.
- “Strictly proper scoring rules, prediction, and estimation.” Journal of the American statistical Association, 102(477): 359–378.
- “Bayesian Data Sketching for Varying Coefficient Regression Models.” Technical report.
- “Bayesian tensor regression.” The Journal of Machine Learning Research, 18(1): 2733–2763.
- “A case study competition among methods for analyzing large spatial data.” Journal of Agricultural, Biological and Environmental Statistics, 24(3): 398–425.
- “Geographically and temporally weighted regression for modeling spatio-temporal variation in house prices.” International Journal of Geographical Information Science, 24(3): 383–401.
- Izenman, A. J. (1975). “Reduced-rank regression for the multivariate linear model.” Journal of Multivariate Analysis, 5(2): 248–264.
- “On block updating in Markov random field models for disease mapping.” Scandinavian Journal of Statistics, 29(4): 597–614.
- “Tensor decompositions and applications.” SIAM Review, 51(3): 455–500.
- “Bayesian Kernelized Matrix Factorization for Spatiotemporal Traffic Data Imputation and Kriging.” IEEE Transactions on Intelligent Transportation Systems.
- “Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization.” arXiv preprint arXiv:2302.14510.
- “Spatial dynamic factor analysis.” Bayesian Analysis, 3(4): 759–792.
- “Variational Gaussian-process factor analysis for modeling spatio-temporal data.” Advances in Neural Information Processing Systems, 22: 1177–1185.
- “Towards a multidimensional approach to Bayesian disease mapping.” Bayesian analysis, 12(1): 239.
- “Elliptical slice sampling.” In Proceedings of the thirteenth international conference on artificial intelligence and statistics, 541–548. JMLR Workshop and Conference Proceedings.
- “Slice sampling covariance hyperparameters of latent Gaussian models.” Advances in Neural Information Processing Systems, 1723–1731.
- Neal, R. M. (2003). “Slice sampling.” The annals of statistics, 31(3): 705–767.
- “A unifying view of sparse approximate Gaussian process regression.” The Journal of Machine Learning Research, 6: 1939–1959.
- “Low-rank regression with tensor responses.” Advances in Neural Information Processing Systems, 29: 1867–1875.
- “Scalable Bayesian low-rank decomposition of incomplete multiway tensors.” International Conference on Machine Learning, 1800–1808.
- “Collaborative Filtering with Graph Information: Consistency and Scalable Methods.” Advances in Neural Information Processing Systems, 2107–2115.
- Gaussian Processes for Machine Learning. MIT Press.
- Gaussian Markov random fields: theory and applications. CRC press.
- Saatçi, Y. (2012). “Scalable inference for structured Gaussian process models.” Ph.D. thesis, University of Cambridge.
- Titsias, M. (2009). “Variational learning of inducing variables in sparse Gaussian processes.” In Artificial intelligence and statistics, 567–574. PMLR.
- “Modeling bike-sharing demand using a regression model with spatially varying coefficients.” Journal of Transport Geography, 93: 103059.
- “Fast Kernel Learning for Multidimensional Pattern Extrapolation.” Advances in Neural Information Processing Systems, 3626–3634.
- “Tensor regression meets gaussian processes.” International Conference on Artificial Intelligence and Statistics, 482–490.
- ‘‘Learning from multiway data: Simple and efficient tensor regression.” International Conference on Machine Learning, 373–381.
- “Spatial factor modeling: A Bayesian matrix-normal approach for misaligned data.” Biometrics, 78(2): 560–573.
- “Tensor regression with applications in neuroimaging data analysis.” Journal of the American Statistical Association, 108(502): 540–552.