Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

On Properties of Adjoint Systems for Evolutionary PDEs (2404.02320v2)

Published 2 Apr 2024 in math.OC, cs.NA, and math.NA

Abstract: We investigate the geometric structure of adjoint systems associated with evolutionary partial differential equations at the fully continuous, semi-discrete, and fully discrete levels and the relations between these levels. We show that the adjoint system associated with an evolutionary partial differential equation has an infinite-dimensional Hamiltonian structure, which is useful for connecting the fully continuous, semi-discrete, and fully discrete levels. We subsequently address the question of discretize-then-optimize versus optimize-then-discrete for both semi-discretization and time integration, by characterizing the commutativity of discretize-then-optimize methods versus optimize-then-discretize methods uniquely in terms of an adjoint-variational quadratic conservation law. For Galerkin semi-discretizations and one-step time integration methods in particular, we explicitly construct these commuting methods by using structure-preserving discretization techniques.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. M. Alexe and A. Sandu. On the discrete adjoints of adaptive time stepping algorithms. Journal of Computational and Applied Mathematics, 233(4):1005–1020, 2009. ISSN 0377-0427. doi: https://doi.org/10.1016/j.cam.2009.08.109.
  2. Unified analysis of discontinuous Galerkin methods for elliptic problems. SIAM Journal on Numerical Analysis, 39(5):1749–1779, 2002. doi: 10.1137/S0036142901384162.
  3. Deep learning as optimal control problems: Models and numerical methods. J. Comput. Dyn., 6(2):171–198, 2019.
  4. J. F. Bonnans and J. Laurent-Varin. Computation of order conditions for symplectic partitioned Runge–Kutta schemes with application to optimal control. Numerische Mathematik, 103:1–10, 2006. doi: 10.1007/s00211-005-0661-y.
  5. Multi-symplectic integrators: numerical schemes for Hamiltonian PDEs that conserve symplecticity. Physics Letters A, 284(4):184 – 193, 2001.
  6. D. G. Cacuci. Sensitivity theory for nonlinear systems. I. Nonlinear functional analysis approach. J. Math. Phys., 22(12):2794–2802, 1981.
  7. Adjoint sensitivity analysis for differential-algebraic equations: The adjoint DAE system and its numerical solution. SIAM J. Sci. Comput., 24(3):1076–1089 (14 pages), 2003.
  8. J. Chen. Variational formulation for the multisymplectic Hamiltonian systems. Lett. Math. Phys., 71(3):243–253, 2005.
  9. Neural ordinary differential equations. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, page 6572–6583, Red Hook, NY, USA, 2018. Curran Associates Inc.
  10. Properties of Infinite Dimensional Hamiltonian Systems. Springer Berlin, Heidelberg, 1974. doi: 10.1007/BFb0073665.
  11. Space-time block preconditioning for incompressible flow. SIAM Journal on Scientific Computing, 44(1):A337–A363, 2022.
  12. Direct and Adjoint Sensitivity Analysis of Ordinary Differential Equation Multibody Formulations. Journal of Computational and Nonlinear Dynamics, 10(1):7 pages, 2014. ISSN 1555-1415. doi: 10.1115/1.4026492.
  13. The Adjoint Method for Time-Optimal Control Problems. Journal of Computational and Nonlinear Dynamics, 16(2):021003, 2020. ISSN 1555-1415. doi: 10.1115/1.4048808.
  14. Well-balanced and energy stable schemes for the shallow water equations with discontinuous topography. Journal of Computational Physics, 230(14):5587–5609, 2011.
  15. Anode: Unconditionally accurate memory-efficient gradients for neural odes. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 730–736. IJCAI, 2019. doi: 10.24963/ijcai.2019/103.
  16. An introduction to the adjoint approach to design. Flow, Turbulence and Combustion, 65:393–415, 2000.
  17. M. B. Giles and E. Süli. Adjoint methods for PDEs: a posteriori error analysis and postprocessing by duality. Acta Numerica, 11:145–236, 2002. doi: 10.1017/S096249290200003X.
  18. A. Griewank. A mathematical view of automatic differentiation. In Acta Numer., volume 12, pages 321–398. Cambridge University Press, 2003.
  19. Geometric Numerical Integration, Structure-Preserving Algorithms for Ordinary Differential Equations. Springer Series in Computational Mathematics. Springer Berlin, Heidelberg, 2006. doi: 10.1007/3-540-30666-8.
  20. Adaptation of the CPR Preconditioner for Efficient Solution of the Adjoint Equation. SPE Journal, 18(02):207–213, 2013. doi: 10.2118/141300-PA.
  21. R. Hartmann. Adjoint consistency analysis of discontinuous Galerkin discretizations. SIAM Journal on Numerical Analysis, 45(6):2671–2696, 2007. doi: 10.1137/060665117.
  22. Adjoint consistency analysis of residual-based variational multiscale methods. Journal of Computational Physics, 255:396–406, 2013. ISSN 0021-9991. doi: https://doi.org/10.1016/j.jcp.2013.07.039.
  23. G. Horton and S. Vandewalle. A space-time multigrid method for parabolic partial differential equations. SIAM Journal on Scientific Computing, 16(4):848–864, 1995.
  24. Space-time finite element methods for second-order hyperbolic equations. Computer methods in applied mechanics and engineering, 84(3):327–348, 1990.
  25. N. H. Ibragimov. Integrating factors, adjoint equations and Lagrangians. Journal of Mathematical Analysis and Applications, 318(2):742–757, 2006.
  26. N. H. Ibragimov. A new conservation theorem. J. Math. Anal. Appl., 333(1):311–328, 2007.
  27. Discrete adjoint formulation for continuum sensitivity analysis. AIAA Journal, 54(2):758–766, 2016. doi: 10.2514/1.J053827.
  28. S. Li and L. Petzold. Adjoint sensitivity analysis for time-dependent partial differential equations with adaptive mesh refinement. Journal of Computational Physics, 198(1):310–325, 2004. doi: 10.1016/j.jcp.2003.01.001.
  29. S. Li and L. R. Petzold. Solution adapted mesh refinement and sensitivity analysis for parabolic partial differential equation systems. In L. T. Biegler, M. Heinkenschloss, O. Ghattas, and B. van Bloemen Waanders, editors, Large-Scale PDE-Constrained Optimization, pages 117–132. Springer, Berlin, 2003.
  30. Introduction to Mechanics and Symmetry. Texts in Applied Mathematics. Springer New York, NY, 2 edition, 1999. doi: 10.1007/978-0-387-21792-5.
  31. Multisymplectic geometry, covariant Hamiltonians, and water waves. Mathematical Proceedings of the Cambridge Philosophical Society, 125(3):553–575, 1999.
  32. The symplectic adjoint method: Memory-efficient backpropagation of neural-network-based differential equations. IEEE Transactions on Neural Networks and Learning Systems, pages 1–13, 2023. doi: 10.1109/TNNLS.2023.3242345.
  33. Goal-oriented a posteriori estimation of numerical errors in the solution of multiphysics systems. arXiv preprint arXiv:2001.08824, 2020.
  34. J. Neerven. The Adjoint of a Semigroup of Linear Operators. Springer Berlin, Heidelberg, 1992. doi: 10.1007/BFb0085008.
  35. State and parameter estimation in 1-D hyperbolic PDEs based on an adjoint method. Automatica, 67(C):185–191, May 2016. ISSN 0005-1098.
  36. A. Noack and A. Walther. Adjoint concepts for the optimal control of Burgers equations. Comput Optim Applic, 36:109–133, 2007. doi: 10.1007/s10589-006-0393-7.
  37. D. Onken and L. Ruthotto. Discretize-optimize vs. optimize-discretize for time-series regression and continuous normalizing flows. arXiv preprint arXiv:2005.13420, 2020.
  38. A. Pazy. Semigroups of Linear Operators and Applications to Partial Differential Equations. Springer New York, New York, NY, 1983. doi: 10.1007/978-1-4612-5561-1.
  39. Adjoint recovery of superconvergent functionals from PDE approximations. SIAM Rev., 42(2):247–264, 2000.
  40. Sensitivity analysis using adjoint parabolized stability equations for compressible flows. Flow, Turbulence and Combustion, 65:321–346, 2000. doi: 10.1023/A:1011434805046.
  41. A space–time discontinuous galerkin method for the incompressible navier–stokes equations. Journal of computational physics, 233:339–358, 2013.
  42. I. M. Ross. A roadmap for optimal control: The right way to commute. Ann. NY Acad. Sci., 1065(1):210–231, 2005.
  43. M. Ross and F. Fahroo. A pseudospectral transformation of the convectors of optimal control systems. IFAC Proc. Ser., 34(13):543–548, 2001.
  44. Adjoint sensitivity analysis for shallow-water wave control. Journal of Engineering Mechanics, 126(9):909–919, 2000. doi: 10.1061/(ASCE)0733-9399(2000)126:9(909).
  45. A. Sandu. On the properties of Runge-Kutta discrete adjoints. In Computational Science – ICCS 2006, page 550–557, Berlin, Heidelberg, 2006. Springer-Verlag.
  46. A. Sandu. On consistency properties of discrete adjoint linear multistep methods. Tech. Rep. TR-07-40, 2007.
  47. J. M. Sanz-Serna. Symplectic Runge–Kutta schemes for adjoint equations, automatic differentiation, optimal control, and more. SIAM Review, 58(1):3–33, 2016.
  48. Topological Vector Spaces. Graduate Texts in Mathematics. Springer New York, NY, 2 edition, 1999. doi: 10.1007/978-1-4612-1468-7.
  49. Efficient preconditioning of the discrete adjoint equations for the incompressible navier–stokes equations. International Journal for Numerical Methods in Fluids, 47, 2005. doi: 10.1002/fld.874.
  50. Z. Sirkes and E. Tziperman. Finite difference of adjoint or adjoint of finite difference? Mon. Weather Rev., 125(12):3373–3378, 1997.
  51. B. Tran and M. Leok. Multisymplectic Hamiltonian variational integrators. International Journal of Computer Mathematics (Special Issue on Geometric Numerical Integration, Twenty-Five Years Later), 99(1):113–157, 2022.
  52. B. K. Tran and M. Leok. Geometric methods for adjoint systems. J Nonlinear Sci, 34(25), 2024. doi: 10.1007/s00332-023-09999-7.
  53. A. Walther. Automatic differentiation of explicit Runge–Kutta methods for optimal control. Comput Optim Applic, 36:83–108, 2007. doi: 10.1007/s10589-006-0397-3.
  54. Risk assessment of scramjet unstart using adjoint-based sampling. AIAA J., 50(3):581–592, 2012.
  55. K. Yano and S. Ishihara. Tangent and cotangent bundles: differential geometry. Pure Appl. Math., No. 16. Marcel Dekker, Inc., New York, 1973.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now