2000 character limit reached
Efficient Shallow Ritz Method For 1D Diffusion Problems (2404.17750v3)
Published 27 Apr 2024 in math.NA and cs.NA
Abstract: This paper studies the shallow Ritz method for solving the one-dimensional diffusion problem. It is shown that the shallow Ritz method improves the order of approximation dramatically for non-smooth problems. To realize this optimal or nearly optimal order of the shallow Ritz approximation, we develop a damped block Newton (dBN) method that alternates between updates of the linear and non-linear parameters. Per each iteration, the linear and the non-linear parameters are updated by exact inversion and one step of a modified, damped Newton method applied to a reduced non-linear system, respectively. The computational cost of each dBN iteration is $O(n)$. Starting with the non-linear parameters as a uniform partition of the interval, numerical experiments show that the dBN is capable of efficiently moving mesh points to nearly optimal locations. To improve efficiency of the dBN further, we propose an adaptive damped block Newton (AdBN) method by combining the dBN with the adaptive neuron enhancement (ANE) method [26].
- M. Ainsworth and Y. Shin. Plateau phenomenon in gradient descent training of ReLU networks: Explanation, quantification and avoidance. SIAM Journal on Scientific Computing, 43:A3438–A3468, 2020.
- M. Ainsworth and Y. Shin. Active neuron least squares: A training method for multivariate rectified neural networks. SIAM Journal on Scientific Computing, 44(4):A2253–A2275, 2022.
- Unicity of best mean approximation by second order splines with variable knots. Mathematics of Computation, 32(144):1131–1143, 1978.
- J. Berg and K. Nyström. A unified deep artificial neural network approach to partial differential equations in complex geometries. Neurocomputing, 317:28–41, 2018.
- H. G. Burchard. Splines (with optimal knots) are better. Applicable Analysis, 3:309–319, 1974.
- Least-squares ReLU neural network (LSNN) method for linear advection-reaction equation. Journal of Computational Physics, 443 (2021) 110514.
- Least-squares ReLU neural network (LSNN) method for scalar nonlinear hyperbolic conservation law. Applied Numerical Mathematics, 174:163–176, 2022.
- Least-squares neural network (LSNN) method for scalar nonlinear hyperbolic conservation laws: Discrete divergence operator. Journal of Computational and Applied Mathematics, 433:115298, 2023.
- Deep least-squares methods: An unsupervised learning-based numerical method for solving elliptic PDEs. Journal of Computational Physics, 420:109707, 2020.
- A structure-guided gauss-newton method for shallow ReLU neural network. arXiv:2404.05064v1 [cs.LG], 2024.
- Z. Cai and S. Zhang. Recovery-based error estimators for interface problems: conforming linear elements. SIAM Journal on Numerical Analysis, 47(3):2132–2156, 2009.
- Nonlinear approximation and (deep) ReLU networks. Constructive Approximation, 55(1):127–172, February 2022.
- W. C. Davidon. Variable metric method for minimization. SIAM Journal on Optimization, 1(1):1–17, 1991.
- C. deBoor and J. R. Rice. Least squares cubic spline approximation ii: variable knots. Department of Computer Sciences Purdue University, CSD TR 21, 1968.
- Neural network based approximations for solving partial differential equations. Communications in Numerical Methods in Engineering, 10(3):195–201, 1994.
- T. Dockhorn. A discussion on solving partial differential equations using neural networks. arXiv:1904.07200 [cs.LG], abs/1904.07200, 2019.
- On mesh geometry and stiffness matrix conditioning for general finite element spaces. SIAM Journal on Numerical Analysis, 47(2):1421–1444, 2009.
- W. E and B. Yu. The deep Ritz method: A deep learning-based numerical algorithm for solving variational problems. Communications in Mathematics and Statistics, 6(1):1–12, March 2018.
- R. Fletcher and M. J. D. Powell. A rapidly convergent descent method for minimization. The Computer Journal, 6(2):163–168, 1963.
- G. H. Golub and V. Pereyra. The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate. SIAM Journal on Numerical Analysis, 10(2):413–432, 1973.
- D. L. B. Jupp. Approximation to data by splines with free knots. SIAM Journal on Numerical Analysis, 15(2):328–343, 1978.
- Neural-network methods for boundary value problems with irregular boundaries. IEEE Trans. on Neural Networks, 11(5):1041–1049, 2000.
- Artificial neural networks for solving ordinary and partial differential equations. IEEE Transactions on Neural Networks, 9(5):987–1000, 1998.
- M. Liu and Z. Cai. Adaptive two-layer ReLU neural network: II. Ritz approximation to elliptic pdes. Computers & Mathematics with Applications, 113:103–116, May 2022.
- Adaptive two-layer ReLU neural network: I. best least-squares approximation. Computers & Mathematics with Applications, 113:34–44, May 2022.
- On the best least squares approximation of continuous functions using linear splines with free knots. IMA Journal of Numerical Analysis, 11(3):393–409, July 1991.
- J. Nocedal and S. J. Wright. Numerical Optimization. Springer, New York, NY, USA, 2e edition, 2006.
- M. R. Osborne. Some special nonlinear least squares problems. SIAM Journal on Numerical Analysis, 12:571–592, 1975.
- Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019.
- L. Schumaker. Spline functions: Basic theory. Wiley, New York, 1981.
- J. Sirignano and K. Spiliopoulos. DGM: A deep learning algorithm for solving partial differential equations. Journal of Computational Physics, 375:1339–1364, 2018.