Continuous Field Reconstruction from Sparse Observations with Implicit Neural Networks (2401.11611v1)
Abstract: Reliably reconstructing physical fields from sparse sensor data is a challenge that frequently arises in many scientific domains. In practice, the process generating the data often is not understood to sufficient accuracy. Therefore, there is a growing interest in using the deep neural network route to address the problem. This work presents a novel approach that learns a continuous representation of the physical field using implicit neural representations (INRs). Specifically, after factorizing spatiotemporal variability into spatial and temporal components using the separation of variables technique, the method learns relevant basis functions from sparsely sampled irregular data points to develop a continuous representation of the data. In experimental evaluations, the proposed model outperforms recent INR methods, offering superior reconstruction quality on simulation data from a state-of-the-art climate model and a second dataset that comprises ultra-high resolution satellite-based sea surface temperature fields.
- Inferring latent velocities from weather radar data using gaussian processes. Advances in Neural Information Processing Systems, 31, 2018.
- Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 28(3):24, 2009.
- Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 417–424, 2000.
- Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences, 113(15):3932–3937, 2016.
- Aerodynamic data reconstruction and inverse design using proper orthogonal decomposition. AIAA journal, 42(8):1505–1516, 2004.
- Any-resolution training for high-resolution image synthesis. In European Conference on Computer Vision, pp. 170–188. Springer, 2022.
- Learning continuous image representation with local implicit image function. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 8628–8638, 2021.
- Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=OnD9zGAGT0k.
- The community earth system model version 2 (cesm2). Journal of Advances in Modeling Earth Systems, 12(2):e2019MS001916, 2020.
- Learning vortex dynamics for fluid inference and prediction. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=nYWqxUwFc3x.
- {PDE}-driven spatiotemporal disentanglement. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=vLaHRtHvfFp.
- Meshfreeflownet: A physics-constrained deep continuous space-time super-resolution framework. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–15. IEEE, 2020.
- Karhunen–loeve procedure for gappy data. JOSA A, 12(8):1657–1664, 1995.
- Multiplicative filter networks. In International Conference on Learning Representations, 2020.
- Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy. Nature Physics, 18(1):112–117, 2022.
- Compressing multidimensional weather and climate data into neural networks. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=Y5SEe3dfniJ.
- Thomas JR Hughes. The finite element method: linear static and dynamic finite element analysis. Courier Corporation, 2012.
- Deep fluids: A generative network for parameterized fluid simulations. In Computer graphics forum, volume 38, pp. 59–70. Wiley Online Library, 2019.
- Auto-encoding variational bayes. In ICLR, 2014.
- Maskgan: Towards diverse and interactive facial image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5549–5558, 2020.
- Model reduction of dynamical systems on nonlinear manifolds using deep convolutional autoencoders. Journal of Computational Physics, 404:108973, 2020.
- Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=c8P9NQVtmnO.
- Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=Bkg6RiCqY7.
- Reinstating continuous climate patterns from small and discretized data. In 1st Workshop on the Synergy of Scientific and Machine Learning Modeling@ ICML2023, 2023.
- Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1):4950, 2018.
- Group for high resolution sea surface temperature (ghrsst) analysis fields inter-comparisons. part 1: A ghrsst multi-product ensemble (gmpe). Deep Sea Research Part II: Topical Studies in Oceanography, 77:21–30, 2012.
- Inferring propagation paths for sparsely observed perturbations on complex networks. Science advances, 2(10):e1501638, 2016.
- Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4460–4470, 2019.
- Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021.
- Improving sparse network seismic location with bayesian kriging and teleseismically constrained calibration events. Bulletin of the Seismological Society of America, 90(1):199–211, 2000.
- Arbitrary-scale image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11533–11542, 2022.
- Spin: Simplifying polar invariance for neural networks application to vision-based irradiance forecasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5182–5191, 2022.
- Neural implicit flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data. Journal of Machine Learning Research, 24(41):1–60, 2023. URL http://jmlr.org/papers/v24/22-0365.html.
- Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 165–174, 2019.
- Learning mesh-based simulation with graph networks. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=roNqYL0_XP.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019.
- Gaussian processes for machine learning, volume 1. Springer, 2006.
- Dynamic ct reconstruction from limited views with implicit neural representations and parametric motion fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2258–2268, 2021.
- Deep learning and process understanding for data-driven earth system science. Nature, 566(7743):195–204, 2019.
- Integrated study of triboelectric nanogenerator for ocean wave energy harvesting: Performance assessment in realistic sea conditions. Nano Energy, 84:105890, 2021.
- Donald Shepard. A two-dimensional interpolation function for irregularly-spaced data. In Proceedings of the 1968 23rd ACM national conference, pp. 517–524, 1968.
- Alex Sherstinsky. Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D: Nonlinear Phenomena, 404:132306, 2020.
- Lawrence Sirovich. Turbulence and the dynamics of coherent structures. i. coherent structures. Quarterly of applied mathematics, 45(3):561–571, 1987.
- Implicit neural representations with periodic activation functions. Advances in neural information processing systems, 33:7462–7473, 2020.
- Piner: Prior-informed implicit neural representation learning for test-time adaptation in sparse-view ct reconstruction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1928–1938, 2023a.
- Pseudoinverse-guided diffusion models for inverse problems. In International Conference on Learning Representations, 2023b. URL https://openreview.net/forum?id=9_gsMA8MRKQ.
- Learned simulators for turbulence. In International conference on learning representations, 2021.
- Modal analysis of fluid flows: An overview. Aiaa Journal, 55(12):4013–4041, 2017.
- Fourier features let networks learn high frequency functions in low dimensional domains. Advances in Neural Information Processing Systems, 33:7537–7547, 2020.
- Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
- Neural fields in visual computing and beyond. In Computer Graphics Forum, volume 41, pp. 641–676. Wiley Online Library, 2022.
- Faster kernel interpolation for gaussian processes. In International Conference on Artificial Intelligence and Statistics, pp. 2971–2979. PMLR, 2021.
- Generative image inpainting with contextual attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5505–5514, 2018.
- Cryodrgn: reconstruction of heterogeneous cryo-em structures using neural networks. Nature methods, 18(2):176–185, 2021.