Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
143 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Learning Networked Dynamical System Models with Weak Form and Graph Neural Networks (2407.16779v1)

Published 23 Jul 2024 in eess.SY and cs.SY

Abstract: This paper presents a sequence of two approaches for the data-driven control-oriented modeling of networked systems, i.e., the systems that involve many interacting dynamical components. First, a novel deep learning approach named the weak Latent Dynamics Model (wLDM) is developed for learning generic nonlinear dynamics with control. Leveraging the weak form, the wLDM enables more numerically stable and computationally efficient training as well as more accurate prediction, when compared to conventional methods such as neural ordinary differential equations. Building upon the wLDM framework, we propose the weak Graph Koopman Bilinear Form (wGKBF) model, which integrates geometric deep learning and Koopman theory to learn latent space dynamics for networked systems, especially for the challenging cases having multiple timescales. The effectiveness of the wLDM framework and wGKBF model are demonstrated on three example systems of increasing complexity - a controlled double pendulum, the stiff Brusselator dynamics, and an electrified aircraft energy system. These numerical examples show that the wLDM and wGKBF achieve superior predictive accuracy and training efficiency as compared to baseline models. Parametric studies provide insights into the effects of hyperparameters in the weak form. The proposed framework shows the capability to efficiently capture control-dependent dynamics in these systems, including stiff dynamics and multi-physics interactions, offering a promising direction for learning control-oriented models of complex networked systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (63)
  1. Boglietti, A., Cavagnino, A., Tenconi, A., and Vaschetto, S., “The Safety Critical Electric Machines and Drives in the More Electric Aircraft: A Survey,” Industrial Electronics Conference, 2009, pp. 2587–2594.
  2. Roberts, R. A., and Decker, D. D., “Energy Optimization of an Aircraft Focused on Component Sizing and Control Architecture Interactions,” 11th International Energy Conversion Engineering Conference, 2013, p. 3805.
  3. Karimi, K. J., “Future Aircraft Power Systems-Integration Challenges,” , 2007.
  4. Gerstler, W. D., and Bunker, R. S., “Aircraft Engine Thermal Management: The Impact of Aviation Electric Power Demands,” ASME Global Gas Turbine News, 2008.
  5. Park, S., and Pangborn, H. C., “Power and Thermal Management with Battery Degradation for Hybrid Electric Vehicles,” IEEE Conference on Control Technology and Applications, 2021, pp. 832–838.
  6. Aksland, C. T., Tannous, P. J., Wagenmaker, M. J., Pangborn, H. C., and Alleyne, A. G., “Hierarchical Predictive Control of an Unmanned Aerial Vehicle Integrated Power, Propulsion, and Thermal Management System,” IEEE Transactions on Control Systems Technology, Vol. 31, No. 3, 2023. 10.1109/TCST.2022.3220913.
  7. Koeln, J. P., Pangborn, H. C., Williams, M. A., Kawamura, M. L., and Alleyne, A. G., “Hierarchical Control of Aircraft Electro-Thermal Systems,” IEEE Transactions on Control Systems Technology, Vol. 28, No. 4, 2019, pp. 1–15. 10.1109/tcst.2019.2905221.
  8. Park, S., and Pangborn, H. C., “Lifted Graph-based Modeling for Linear Predictive Control of Nonlinear Energy Systems,” 2023 IEEE Conference on Control Technology and Applications (CCTA), 2023, pp. 926–933. 10.1109/CCTA54093.2023.10252310.
  9. Bruder, D., Fu, X., and Vasudevan, R., “Advantages of bilinear Koopman realizations for the modeling and control of systems with unknown dynamics,” IEEE Robotics and Automation Letters, Vol. 6, No. 3, 2021, pp. 4369–4376.
  10. Kanai, M., and Yamakita, M., “Lifted Bilinear Model-based Linear Model Predictive Control with Scalability,” IFAC-PapersOnLine, Vol. 56, No. 2, 2023, pp. 9405–9410. https://doi.org/10.1016/j.ifacol.2023.10.232, 22nd IFAC World Congress.
  11. Lian, Y., Jiang, Y., Opila, D. F., and Jones, C. N., “A Proximal-Point Lagrangian-Based Parallelizable Nonconvex Solver for Bilinear Model Predictive Control,” IEEE Transactions on Control Systems Technology, Vol. 31, No. 5, 2023, pp. 2245–2260. 10.1109/TCST.2023.3291537.
  12. Budišić, M., Mohr, R., and Mezić, I., “Applied Koopmanism,” Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 22, No. 4, 2012, p. 047510.
  13. Surana, A., and Banaszuk, A., “Linear Observer Synthesis for Nonlinear Systems Using Koopman Operator Framework,” IFAC-PapersOnLine, Vol. 49, No. 18, 2016, pp. 716–723. 10th IFAC Symposium on Nonlinear Control Systems NOLCOS 2016.
  14. Mauroy, A., and Mezić, I., “Global Stability Analysis Using the Eigenfunctions of the Koopman Operator,” IEEE Transactions on Automatic Control, Vol. 61, No. 11, 2016, pp. 3356–3369.
  15. Goldschmidt, A., Kaiser, E., Dubois, J. L., Brunton, S. L., and Kutz, J. N., “Bilinear dynamic mode decomposition for quantum control,” New Journal of Physics, Vol. 23, No. 3, 2021, p. 033035.
  16. Goswami, D., and Paley, D. A., “Bilinearization, Reachability, and Optimal Control of Control-Affine Nonlinear Systems: A Koopman Spectral Approach,” IEEE Transactions on Automatic Control, Vol. 67, 2022, pp. 2715–2728.
  17. Jiang, X., Li, Y., and Huang, D., “Modularized Bilinear Koopman Operator for Modeling and Predicting Transients of Microgrids,” IEEE Transactions on Smart Grid, 2024. 10.1109/TSG.2024.3399076.
  18. Abraham, I., and Murphey, T. D., “Active learning of dynamics for data-driven control using Koopman operators,” IEEE Transactions on Robotics, Vol. 35, No. 5, 2019, pp. 1071–1083.
  19. Rosenfeld, J. A., and Kamalapurkar, R., “Dynamic mode decomposition with control Liouville operators,” IFAC-PapersOnLine, Vol. 54, No. 9, 2021, pp. 707–712.
  20. Schulze, J. C., and Mitsos, A., “Data-Driven Nonlinear Model Reduction Using Koopman Theory: Integrated Control Form and NMPC Case Study,” IEEE Control Systems Letters, Vol. 6, 2022, pp. 2978–2983.
  21. Proctor, J. L., Brunton, S. L., and Kutz, J. N., “Dynamic Mode Decomposition with Control,” SIAM Journal on Applied Dynamical Systems, Vol. 15, No. 1, 2016, pp. 142–161. 10.1137/15M1013857.
  22. Li, Q., Dietrich, F., Bollt, E. M., and Kevrekidis, I. G., “Extended dynamic mode decomposition with dictionary learning: A data-driven adaptive spectral decomposition of the Koopman operator,” Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 27, No. 10, 2017.
  23. Huang, D., Prasetyo, M. B., Yu, Y., and Geng, J., “Learning Koopman Operators with Control Using Bi-level Optimization,” IEEE CDC 2023, 2023a. 10.1109/CDC49753.2023.10383799.
  24. Folkestad, C., Wei, S. X., and Burdick, J. W., “KoopNet: Joint learning of Koopman bilinear models and function dictionaries with application to quadrotor trajectory tracking,” 2022 International Conference on Robotics and Automation (ICRA), IEEE, 2022, pp. 1344–1350.
  25. Shi, H., and Meng, M. Q.-H., “Deep Koopman operator with control for nonlinear systems,” IEEE Robotics and Automation Letters, Vol. 7, No. 3, 2022, pp. 7700–7707.
  26. Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al., “Relational inductive biases, deep learning, and graph networks,” arXiv preprint arXiv:1806.01261, 2018.
  27. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G., “The graph neural network model,” IEEE Transactions on Neural Networks, Vol. 20, No. 1, 2009, pp. 61–80. 10.1109/TNN.2008.2005605.
  28. Yu, Y., Harlim, J., Huang, D., and Li, Y., “Learning Coarse-Grained Dynamics on Graph,” arXiv preprint arXiv:2405.09324, 2024a.
  29. Yu, Y., Jiang, X., Huang, D., Li, Y., Yue, M., and Zhao, T., “PIDGeuN: Graph Neural Network-Enabled Transient Dynamics Prediction of Networked Microgrids Through Full-Field Measurement,” IEEE Access, Vol. 12, 2024b, pp. 49464–49475. 10.1109/ACCESS.2024.3384457.
  30. Yu, Y., Major, D. A., Huang, D., and Schmitz, S., “Reduced-Order Modeling of Ship Airwakes with Atmospheric Turbulence Effects using Dynamic Graph Networks,” AIAA SCITECH 2022 Forum, 2022, p. 2533.
  31. Massegur Sampietro, D., and Da Ronch, A., “Gradient-Guided Graph Convolutional Multi-Mesh Frameworks for Aircraft Aerodynamics Modelling,” AIAA SCITECH 2024 Forum, 2024, p. 0456.
  32. Li, P., Yu, Y., Huang, D., Wang, Z.-H., and Sharma, A., “Regional Heatwave Prediction Using Graph Neural Network and Weather Station Data,” Geophysical Research Letters, Vol. 50, No. 7, 2023, p. e2023GL103405. https://doi.org/10.1029/2023GL103405, URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2023GL103405, e2023GL103405 2023GL103405.
  33. Yu, Y., Li, P., Huang, D., and Sharma, A., “Street-level temperature estimation using Graph Neural Networks: Performance, feature embedding and interpretability,” Authorea Preprints, 2024c.
  34. Sanchez-Gonzalez, A., Heess, N., Springenberg, J. T., Merel, J., Riedmiller, M., Hadsell, R., and Battaglia, P., “Graph networks as learnable physics engines for inference and control,” International Conference on Machine Learning, PMLR, 2018, pp. 4470–4479.
  35. Li, Y., Wu, J., Zhu, J.-Y., Tenenbaum, J. B., Torralba, A., and Tedrake, R., “Propagation networks for model-based control under partial observation,” 2019 International Conference on Robotics and Automation (ICRA), IEEE, 2019, pp. 1205–1211.
  36. Karimi, A. M., Wu, Y., Koyuturk, M., and French, R. H., “Spatiotemporal Graph Neural Network for Performance Prediction of Photovoltaic Power Systems,” Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, No. 17, 2021, pp. 15323–15330.
  37. 10.1007/978-3-540-27752-1_5.
  38. Hochreiter, S., and Schmidhuber, J., “Long short-term memory,” Neural computation, Vol. 9, No. 8, 1997a, pp. 1735–1780.
  39. Yu, Y., Park, S., Huang, D., and Pangborn, H., “Towards Automated Timescale Decomposition for Graph-based Hierarchical Control of Electrified Aircraft Systems,” AIAA AVIATION 2023 Forum, 2023, p. 4509.
  40. Brunton, S. L., Brunton, B. W., Proctor, J. L., and Kutz, J. N., “Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control,” PloS one, Vol. 11, No. 2, 2016, pp. 1–19.
  41. Messenger, D. A., and Bortz, D. M., “Weak SINDy for partial differential equations,” Journal of Computational Physics, Vol. 443, 2021, p. 110525. 10.1016/j.jcp.2021.110525, URL http://dx.doi.org/10.1016/j.jcp.2021.110525.
  42. Kaptanoglu, A. A., Zhang, L., Nicolaou, Z. G., Fasel, U., and Brunton, S. L., “Benchmarking sparse system identification with low-dimensional chaos,” Nonlinear Dynamics, Vol. 111, No. 14, 2023, pp. 13143–13164. 10.1007/s11071-023-08525-4.
  43. Bengio, Y., Courville, A., and Vincent, P., “Representation learning: A review and new perspectives,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, No. 8, 2013, pp. 1798–1828.
  44. Hinton, G. E., and Salakhutdinov, R. R., “Reducing the dimensionality of data with neural networks,” Science, Vol. 313, No. 5786, 2006, pp. 504–507.
  45. Tenenbaum, J. B., De Silva, V., and Langford, J. C., “A global geometric framework for nonlinear dimensionality reduction,” Science, Vol. 290, No. 5500, 2000, pp. 2319–2323.
  46. Tran, A., He, X., Messenger, D. A., Choi, Y., and Bortz, D. M., “Weak-form latent space dynamics identification,” Computer Methods in Applied Mechanics and Engineering, Vol. 427, 2024, p. 116998. https://doi.org/10.1016/j.cma.2024.116998, URL https://www.sciencedirect.com/science/article/pii/S0045782524002548.
  47. Chen, R. T., Rubanova, Y., Bettencourt, J., and Duvenaud, D., “Neural ordinary differential equations,” arXiv preprint arXiv:1806.07366, 2018.
  48. Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H., and Park, J., “Graph neural ordinary differential equations,” arXiv preprint arXiv:1911.07532, 2019.
  49. Zang, C., and Wang, F., “Neural dynamics on complex networks,” Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, 2020, pp. 892–902.
  50. Luo, X., Yuan, J., Huang, Z., Jiang, H., Qin, Y., Ju, W., Zhang, M., and Sun, Y., “Hope: High-order graph ode for modeling interacting dynamics,” International Conference on Machine Learning, PMLR, 2023, pp. 23124–23139.
  51. Huang, Z., Zhao, W., Gao, J., Hu, Z., Luo, X., Cao, Y., Chen, Y., Sun, Y., and Wang, W., “TANGO: Time-Reversal Latent GraphODE for Multi-Agent Dynamical Systems,” , 2023b.
  52. Gholami, A., Keutzer, K., and Biros, G., “ANODE: Unconditionally accurate memory-efficient gradients for neural ODEs,” arXiv preprint arXiv:1902.10298, 2019.
  53. Zhuang, J., Dvornek, N., Li, X., Tatikonda, S., Papademetris, X., and Duncan, J., “Adaptive checkpoint adjoint method for gradient estimation in neural ODE,” arXiv preprint arXiv:2006.02493, 2020.
  54. Kim, S., Ji, W., Deng, S., Ma, Y., and Rackauckas, C., “Stiff neural ordinary differential equations,” Chaos: An Interdisciplinary Journal of Nonlinear Science, Vol. 31, No. 9, 2021.
  55. Ross, I. M., and Fahroo, F., “A perspective on methods for trajectory optimization,” AIAA/AAS Astrodynamics Specialist Conference and Exhibit, 2002, p. 4727.
  56. Patterson, M. A., and Rao, A. V., “GPOPS-II: A MATLAB software for solving multiple-phase optimal control problems using hp-adaptive Gaussian quadrature collocation methods and sparse nonlinear programming,” ACM Transactions on Mathematical Software (TOMS), Vol. 41, No. 1, 2014, pp. 1–37.
  57. Proctor, J. L., Brunton, S. L., and Kutz, J. N., “Generalizing Koopman theory to allow for inputs and control,” SIAM Journal on Applied Dynamical Systems, Vol. 17, No. 1, 2018, pp. 909–930.
  58. Defferrard, M., Bresson, X., and Vandergheynst, P., “Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering,” arXiv:1606.09375, 2017.
  59. Hochreiter, S., and Schmidhuber, J., “Long Short-Term Memory,” Neural Computation, Vol. 9, No. 8, 1997b, pp. 1735–1780. 10.1162/neco.1997.9.8.1735.
  60. Sutskever, I., Vinyals, O., and Le, Q. V., “Sequence to Sequence Learning with Neural Networks,” Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, MIT Press, Cambridge, MA, USA, 2014, p. 3104–3112.
  61. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S., “PyTorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems 32, edited by H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, Curran Associates, Inc., 2019, pp. 8024–8035.
  62. Aksland, C. T., Bixel, T. W., Raymond, L. C., Rottmayer, M. A., and Alleyne, A. G., “Graph-Based Electro-Mechanical Modeling of a Hybrid Unmanned Aerial Vehicle for Real-Time Applications,” American Control Conference, American Automatic Control Council, 2019, pp. 4253–4259.
  63. Pangborn, H. C., Koeln, J. P., Williams, M. A., and Alleyne, A. G., “Experimental Validation of Graph-Based Hierarchical Control for Thermal Management,” Journal of Dynamic Systems, Measurement, and Control, Vol. 140, No. 10, 2018, pp. 1–16.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now