Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
139 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Physics-informed Neural Networks with Periodic Activation Functions for Solute Transport in Heterogeneous Porous Media (2212.08965v2)

Published 17 Dec 2022 in cs.LG

Abstract: Simulating solute transport in heterogeneous porous media poses computational challenges due to the high-resolution meshing required for traditional solvers. To overcome these challenges, this study explores a mesh-free method based on deep learning to accelerate solute transport simulation. We employ Physics-informed Neural Networks (PiNN) with a periodic activation function to solve solute transport problems in both homogeneous and heterogeneous porous media governed by the advection-dispersion equation. Unlike traditional neural networks that rely on large training datasets, PiNNs use strong-form mathematical models to constrain the network in the training phase and simultaneously solve for multiple dependent or independent field variables, such as pressure and solute concentration fields. To demonstrate the effectiveness of using PiNNs with a periodic activation function to resolve solute transport in porous media, we construct PiNNs using two activation functions, sin and tanh, for seven case studies, including 1D and 2D scenarios. The accuracy of the PiNNs' predictions is then evaluated using absolute point error and mean square error metrics and compared to the ground truth solutions obtained analytically or numerically. Our results demonstrate that the PiNN with sin activation function, compared to tanh activation function, is up to two orders of magnitude more accurate and up to two times faster to train, especially in heterogeneous porous media. Moreover, PiNN's simultaneous predictions of pressure and concentration fields can reduce computational expenses in terms of inference time by three orders of magnitude compared to FEM simulations for two-dimensional cases.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (72)
  1. Determining timescales for groundwater flow and solute transport. In Environmental tracers in subsurface hydrology, pages 1–30. Springer, 2000. DOI: https://doi.org/10.1007/978-1-4615-4557-6˙1.
  2. Geochemical and solute transport modelling for co2 storage, what to expect from it? International journal of greenhouse gas control, 2(4):605–625, 2008. DOI: https://doi.org/10.1016/j.ijggc.2008.02.011.
  3. Fluid flow, heat transfer, and solute transport at nuclear waste storage tanks in the hanford vadose zone. Vadose Zone Journal, 1(1):68–88, 2002. DOI: https://doi.org/10.2136/vzj2002.6800.
  4. Membrane transport of hydrogen peroxide. Biochimica et Biophysica Acta (BBA)-Biomembranes, 1758(8):994–1003, 2006. DOI: https://doi.org/10.1016/j.bbamem.2006.02.015.
  5. Transport of carbon dioxide and ammonium in bioturbated (nereis diversicolor) coastal, marine sediments. Biogeochemistry, 45(2):147–168, 1999. DOI: https://doi.org/10.1023/A:1006131710076.
  6. A dfn based 3d numerical approach for modeling coupled groundwater flow and solute transport in fractured rock mass. International Journal of Heat and Mass Transfer, 149:119179, 2020. DOI: https://doi.org/10.1016/j.ijheatmasstransfer.2019.119179.
  7. Direct characterization of solute transport in unsaturated porous media using fast x-ray synchrotron microtomography. Proceedings of the National Academy of Sciences, 117(38):23443–23449, 2020. DOI: https://doi.org/10.1073/pnas.2011716117.
  8. Mathematical model of solute transport in rivers with storage zones using nonlinear dispersion flux approach. Hydrological Sciences Journal, pages 1–13, 2022. DOI: https://doi.org/10.1080/02626667.2022.2099280.
  9. Ubiquity of anomalous transport in porous media: Numerical evidence, continuous time random walk modelling, and hydrodynamic interpretation. Scientific reports, 9(1):1–11, 2019. DOI: https://doi.org/10.1038/s41598-019-39363-3.
  10. Numerical modeling of stress effects on solute transport in fractured rocks. Computers and Geotechnics, 38(2):113–126, 2011. DOI: https://doi.org/10.1016/j.compgeo.2010.10.001.
  11. A one-dimensional transport model for multi-component solute in saturated soil. Water Science and Engineering, 11(3):236–242, 2018. DOI: https://doi.org/10.1016/j.wse.2018.09.007.
  12. Nikhil Bagalkot and G Suresh Kumar. Effect of nonlinear sorption on multispecies radionuclide transport in a coupled fracture-matrix system with variable fracture aperture: a numerical study. ISH Journal of Hydraulic Engineering, 21(3):242–254, 2015. DOI: https://doi.org/10.1080/09715010.2015.1016125.
  13. Numerical simulation of reactive transport on micro-ct images. Mathematical Geosciences, 48(8):963–983, 2016. DOI: https://doi.org/10.1007/s11004-016-9640-3.
  14. 3-d simulation and analysis of reactive dissolution and wormhole formation in carbonate rocks. Chemical Engineering Science, 90:258–274, 2013. DOI: https://doi.org/10.1016/j.ces.2012.12.032.
  15. Random walk methods for modeling hydrodynamic transport in porous and fractured media from pore to reservoir scale. Transport in Porous Media, 115(2):345–385, 2016. DOI: https://doi.org/10.1007/s11242-016-0693-z.
  16. Surrogate modeling of elasto-plastic problems via long short-term memory neural networks and proper orthogonal decomposition. Computer Methods in Applied Mechanics and Engineering, 385:114030, 2021. DOI: https://doi.org/10.1016/j.cma.2021.114030.
  17. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021. DOI: https://doi.org/10.1038/s42254-021-00314-5.
  18. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686–707, 2019. DOI: https://doi.org/10.1016/j.jcp.2018.10.045.
  19. Solving high-dimensional partial differential equations using deep learning. Proceedings of the National Academy of Sciences, 115(34):8505–8510, 2018. DOI: https://doi.org/10.1073/pnas.1718942115.
  20. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28–41, 2018. DOI: https://doi.org/10.1016/j.neucom.2018.06.056.
  21. Dgm: A deep learning algorithm for solving partial differential equations. Journal of computational physics, 375:1339–1364, 2018. DOI: https://doi.org/10.1016/j.jcp.2018.08.029.
  22. Physics-guided, physics-informed, and physics-encoded neural networks in scientific computing. arXiv preprint arXiv:2211.07377, 2022. DOI: https://doi.org/10.48550/arXiv.2211.07377.
  23. Automatic differentiation in ml: Where we are and where we should be going. Advances in neural information processing systems, 31, 2018. DOI: https://doi.org/10.48550/arXiv.1810.11530.
  24. Automatic differentiation in machine learning: a survey. Journal of Marchine Learning Research, 18:1–43, 2018. DOI: https://doi.org/10.48550/arXiv.1502.05767.
  25. Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations. arXiv preprint arXiv:1711.10561, 2017. DOI: https://doi.org/10.48550/arXiv.1711.10561.
  26. Hidden fluid mechanics: A navier-stokes informed deep learning framework for assimilating flow visualization data. arXiv preprint arXiv:1808.04327, 2018. DOI: https://doi.org/10.48550/arXiv.1808.04327.
  27. Prediction of porous media fluid flow using physics informed neural networks. Journal of Petroleum Science and Engineering, 208:109205, 2022. DOI: https://doi.org/10.1016/j.petrol.2021.109205.
  28. Residual-based adaptivity for two-phase flow simulation in porous media using physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering, 396:115100, 2022. DOI: https://doi.org/10.1016/j.cma.2022.115100.
  29. Physics-informed neural network simulation of multiphase poroelasticity using stress-split sequential training. Computer Methods in Applied Mechanics and Engineering, 397:115141, 2022. DOI: https://doi.org/10.1016/j.cma.2022.115141.
  30. Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport. Advances in Water Resources, 141:103610, 2020. DOI: https://doi.org/10.1016/j.advwatres.2020.103610.
  31. Physics-informed neural network method for forward and backward advection-dispersion equations. Water Resources Research, 57(7):e2020WR029479, 2021. DOI: https://doi.org/10.1029/2020WR029479.
  32. Physics-informed neural network method for solving one-dimensional advection equation using pytorch. Array, 13:100110, 2022. DOI: https://doi.org/10.1016/j.array.2021.100110.
  33. Physics-informed attention-based neural network for hyperbolic partial differential equations: application to the buckley–leverett problem. Scientific reports, 12(1):7557, 2022. DOI: https://doi.org/10.1038/s41598-022-11058-2.
  34. On the monotonicity and positivity of physics-informed neural networks for highly anisotropic diffusion equations. Energies, 15(18):6823, 2022. DOI: https://doi.org/10.3390/en15186823.
  35. Limitations of physics informed machine learning for nonlinear two-phase transport in porous media. Journal of Machine Learning for Modeling and Computing, 1(1), 2020. DOI: https://doi.org/10.1615/jmachlearnmodelcomput.2020033905.
  36. A physics-informed convolutional neural network for the simulation and prediction of two-phase darcy flows in heterogeneous porous media. Journal of Computational Physics, 477:111919, 2023. DOI: https://doi.org/10.1016/j.jcp.2023.111919.
  37. How important are activation functions in regression and classification? a survey, performance comparison, and future directions. Journal of Machine Learning for Modeling and Computing, 2022. DOI: 10.1615/JMachLearnModelComput.2023047367.
  38. Taming the waves: sine as activation function in deep neural networks. 2016.
  39. Influence of stagnant zones on solute transport in heterogeneous porous media at the pore scale. Physics of Fluids, 33(3):036605, 2021. DOI: https://doi.org/10.1063/5.0038133.
  40. On the numerical approximation of mobile-immobile advection-dispersion model of fractional order arising from solute transport in porous media. Fractal and Fractional, 6(8):445, 2022. DOI: https://doi.org/10.3390/fractalfract6080445.
  41. Solute transport in heterogeneous porous media. Massachusetts Institute of Technology. Earth Resources Laboratory, 1994. DOI: https://doi.org/10.1017/S0022112084002858.
  42. M Th Van Genuchten and WJ Alves. Analytical solutions of the one-dimensional convective-dispersive solute transport equation. United States. Dept. of Agriculture. Technical bulletin (USA), 1982. DOI: https://doi.org/10.22004/ag.econ.157268.
  43. A review of applications of fractional advection–dispersion equations for anomalous solute transport in surface and subsurface water. Wiley Interdisciplinary Reviews: Water, 7(4):e1448, 2020. DOI: https://doi.org/10.1002/wat2.1448.
  44. On eddy transport in the ocean. part ii: The advection tensor. Ocean Modelling, 165:101845, 2021. DOI: https://doi.org/10.1016/j.ocemod.2021.101845.
  45. Effective high-order energy stable flux reconstruction methods for first-order hyperbolic linear and nonlinear systems. Journal of Computational Physics, 414:109475, 2020. DOI: https://doi.org/10.1016/j.jcp.2020.109475.
  46. Laurent Talon. On the statistical properties of fluid flows with transitional power-law rheology in heterogeneous porous media. Journal of Non-Newtonian Fluid Mechanics, 304:104789, 2022. DOI: https://doi.org/10.1016/j.jnnfm.2022.104789.
  47. Modeling solute transport and mixing in heterogeneous porous media under turbulent flow conditions. Physics of Fluids, 33(10):106604, 2021. DOI: https://doi.org/10.1063/5.0065734.
  48. Physical pictures of transport in heterogeneous media: Advection-dispersion, random-walk, and fractional derivative formulations. Water Resources Research, 38(10):9–1, 2002. DOI: https://doi.org/10.1029/2001WR001030.
  49. Conservative physics-informed neural networks on discrete domains for conservation laws: Applications to forward and inverse problems. Computer Methods in Applied Mechanics and Engineering, 365:113028, 2020. DOI: https://doi.org/10.1016/j.cma.2020.113028.
  50. Extended physics-informed neural networks (xpinns): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations. In AAAI Spring Symposium: MLPS, 2021. DOI: https://doi.org/10.4208/cicp.OA-2020-0164.
  51. Scientific machine learning through physics-informed neural networks: Where we are and what’s next. arXiv preprint arXiv:2201.05624, 2022a. DOI: https://doi.org/10.48550/arXiv.2201.05624.
  52. Application of physics-informed neural networks to inverse problems in unsaturated groundwater flow. Georisk: Assessment and Management of Risk for Engineered Systems and Geohazards, 16(1):21–36, 2022. DOI: https://doi.org/10.1080/17499518.2021.1971251.
  53. Deepxde: A deep learning library for solving differential equations. SIAM Review, 63(1):208–228, 2021. DOI: https://doi.org/10.1137/19M1274067.
  54. Implicit neural representations with periodic activation functions, 2020. URL https://arxiv.org/abs/2006.09661.
  55. Learning representations by back-propagating errors. nature, 323(6088):533–536, 1986. DOI: https://doi.org/10.1038/323533a0.
  56. Solving partial differential equations using deep learning and physical constraints. Applied Sciences, 10(17):5917, 2020. DOI: https://doi.org/10.3390/app10175917.
  57. Physics-informed neural networks (pinns) for fluid mechanics: A review. Acta Mechanica Sinica, pages 1–12, 2022. DOI: https://doi.org/10.1007/s10409-021-01148-1.
  58. Physics informed neural networks: A case study for gas transport problems. Journal of Computational Physics, page 112041, 2023. DOI: https://doi.org/10.1016/j.jcp.2023.112041.
  59. A physics-informed learning approach to bernoulli-type free boundary problems. Computers & Mathematics with Applications, 128:34–43, 2022b. DOI: https://doi.org/10.1016/j.camwa.2022.10.003.
  60. Physics-informed neural networks as solvers for the time-dependent schr\\\backslash\” odinger equation. arXiv preprint arXiv:2210.12522, 2022. DOI: https://doi.org/10.48550/arXiv.2210.12522.
  61. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. DOI: http://arxiv.org/abs/1412.6980.
  62. A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 16(5):1190–1208, 1995. DOI: https://doi.org/10.1137/0916069.
  63. Yoshua Bengio. Gradient-based optimization of hyperparameters. Neural computation, 12(8):1889–1900, 2000. DOI: https://doi.org/10.1162/089976600300015187.
  64. Gpu accelerated sub-sampled newton’s method for convex classification problems. In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 702–710. SIAM, 2019. DOI: https://doi.org/10.1137/1.9781611975673.79.
  65. Alan Richardson. Seismic full-waveform inversion using deep learning tools and techniques. arXiv preprint arXiv:1801.07232, 2018. DOI: https://doi.org/10.48550/arXiv.1801.07232.
  66. Physics-driven convolutional autoencoder approach for cfd data compressions. arXiv preprint arXiv:2210.09262, 2022. DOI: https://doi.org/10.48550/arXiv.2210.09262.
  67. Hyper-parameter tuning of physics-informed neural networks: Application to helmholtz problems. Neurocomputing, 561:126826, 2023. DOI: https://doi.org/10.1016/j.neucom.2023.126826.
  68. Physics-informed neural networks (pinns) for wave propagation and full waveform inversions. Journal of Geophysical Research: Solid Earth, 127(5):e2021JB023120, 2022. DOI: https://doi.org/10.1029/2021JB023120.
  69. Jian Guo Zhou. A lattice boltzmann method for solute transport. International journal for numerical methods in fluids, 61(8):848–863, 2009. DOI: https://doi.org/10.1002/fld.1978.
  70. Gradient-enhanced physics-informed neural networks for forward and inverse pde problems. Computer Methods in Applied Mechanics and Engineering, 393:114823, 2022. DOI: https://doi.org/10.1016/j.cma.2022.114823.
  71. Can-pinn: A fast physics-informed neural network based on coupled-automatic–numerical differentiation method. Computer Methods in Applied Mechanics and Engineering, 395:114909, 2022. DOI: https://doi.org/10.1016/j.cma.2022.114909.
  72. A study on the kelvin-helmholtz instability using two different computational fluid dynamics methods. The Journal of Computational Multiphase Flows, 2(1):33–45, 2010. DOI: https://doi.org/10.1260/1757-482X.2.1.33.
Citations (19)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now