Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
156 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Physics-Enhanced Graph Neural Networks For Soft Sensing in Industrial Internet of Things (2404.08061v2)

Published 11 Apr 2024 in cs.LG, cs.AI, and eess.SP

Abstract: The Industrial Internet of Things (IIoT) is reshaping manufacturing, industrial processes, and infrastructure management. By fostering new levels of automation, efficiency, and predictive maintenance, IIoT is transforming traditional industries into intelligent, seamlessly interconnected ecosystems. However, achieving highly reliable IIoT can be hindered by factors such as the cost of installing large numbers of sensors, limitations in retrofitting existing systems with sensors, or harsh environmental conditions that may make sensor installation impractical. Soft (virtual) sensing leverages mathematical models to estimate variables from physical sensor data, offering a solution to these challenges. Data-driven and physics-based modeling are the two main methodologies widely used for soft sensing. The choice between these strategies depends on the complexity of the underlying system, with the data-driven approach often being preferred when the physics-based inference models are intricate and present challenges for state estimation. However, conventional deep learning models are typically hindered by their inability to explicitly represent the complex interactions among various sensors. To address this limitation, we adopt Graph Neural Networks (GNNs), renowned for their ability to effectively capture the complex relationships between sensor measurements. In this research, we propose physics-enhanced GNNs, which integrate principles of physics into graph-based methodologies. This is achieved by augmenting additional nodes in the input graph derived from the underlying characteristics of the physical processes. Our evaluation of the proposed methodology on the case study of district heating networks reveals significant improvements over purely data-driven GNNs, even in the presence of noise and parameter inaccuracies.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (52)
  1. Y. Wu, H.-N. Dai, and H. Tang, “Graph neural networks for anomaly detection in industrial internet of things,” IEEE Internet of Things Journal, vol. 9, no. 12, pp. 9214–9231, 2021.
  2. B. Babayigit and M. Abubaker, “Industrial internet of things: A review of improvements over traditional scada systems for industrial automation,” IEEE Systems Journal, 2023.
  3. L. Xing, “Reliability in internet of things: Current status and future perspectives,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 6704–6721, 2020.
  4. Y. Jiang, S. Yin, J. Dong, and O. Kaynak, “A review on soft sensors for monitoring, control, and optimization of industrial processes,” IEEE Sensors Journal, vol. 21, no. 11, pp. 12 868–12 881, 2021.
  5. C. Shang, F. Yang, D. Huang, and W. Lyu, “Data-driven soft sensor development based on deep learning technique,” Journal of Process Control, vol. 24, no. 3, pp. 223–233, 2014.
  6. W. Yan, D. Tang, and Y. Lin, “A data-driven soft sensor modeling method based on deep learning and its application,” IEEE Transactions on Industrial Electronics, vol. 64, no. 5, pp. 4237–4245, 2016.
  7. X. Yuan, B. Huang, Y. Wang, C. Yang, and W. Gui, “Deep learning-based feature representation and its application for soft sensor modeling with variable-wise weighted sae,” IEEE Transactions on Industrial Informatics, vol. 14, no. 7, pp. 3235–3243, 2018.
  8. K. Chen, I. Castillo, L. H. Chiang, and J. Yu, “Soft sensor model maintenance: A case study in industrial processes,” IFAC-PapersOnLine, vol. 48, no. 8, pp. 427–432, 2015.
  9. Y. Guo, Y. Zhao, and B. Huang, “Development of soft sensor by incorporating the delayed infrequent and irregular measurements,” Journal of Process Control, vol. 24, no. 11, pp. 1733–1739, 2014.
  10. M. Schimmack, B. Haus, and P. Mercorelli, “An extended kalman filter as an observer in a control structure for health monitoring of a metal–polymer hybrid soft actuator,” IEEE/ASME Transactions on Mechatronics, vol. 23, no. 3, pp. 1477–1487, 2018.
  11. X. Yang, Y. Zhang, Y. A. Shardt, X. Li, J. Cui, and C. Tong, “A kpi-based soft sensor development approach incorporating infrequent, variable time delayed measurements,” IEEE Transactions on Control Systems Technology, vol. 28, no. 6, pp. 2523–2531, 2019.
  12. E. A. Wan and R. Van Der Merwe, “The unscented kalman filter for nonlinear estimation,” in Proceedings of the IEEE 2000 adaptive systems for signal processing, communications, and control symposium (Cat. No. 00EX373).   Ieee, 2000, pp. 153–158.
  13. R. Van Der Merwe and E. A. Wan, “The square-root unscented kalman filter for state and parameter-estimation,” in 2001 IEEE international conference on acoustics, speech, and signal processing. Proceedings (Cat. No. 01CH37221), vol. 6.   IEEE, 2001, pp. 3461–3464.
  14. M. I. Ribeiro, “Kalman and extended kalman filters: Concept, derivation and properties,” Institute for Systems and Robotics, vol. 43, no. 46, pp. 3736–3741, 2004.
  15. X. Yuan, L. Li, Y. A. Shardt, Y. Wang, and C. Yang, “Deep learning with spatiotemporal attention-based lstm for industrial soft sensor model development,” IEEE Transactions on Industrial Electronics, vol. 68, no. 5, pp. 4404–4414, 2020.
  16. D. Kim, S.-H. Kim, T. Kim, B. B. Kang, M. Lee, W. Park, S. Ku, D. Kim, J. Kwon, H. Lee et al., “Review of machine learning methods in soft robotics,” Plos one, vol. 16, no. 2, p. e0246102, 2021.
  17. C. Shang and F. You, “Data analytics and machine learning for smart process manufacturing: Recent advances and perspectives in the big data era,” Engineering, vol. 5, no. 6, pp. 1010–1016, 2019.
  18. Q. Sun and Z. Ge, “A survey on deep learning for data-driven soft sensors,” IEEE Transactions on Industrial Informatics, vol. 17, no. 9, pp. 5853–5866, 2021.
  19. W. Cao, D. Wang, J. Li, H. Zhou, L. Li, and Y. Li, “Brits: Bidirectional recurrent imputation for time series,” in Advances in Neural Information Processing Systems, vol. 31, 2018.
  20. Y. Luo, Y. Zhang, X. Cai, and X. Yuan, “E2gan: End-to-end generative adversarial network for multivariate time series imputation,” in Proceedings of the 28th international joint conference on artificial intelligence.   AAAI Press Palo Alto, CA, USA, 2019, pp. 3094–3100.
  21. Y. Tashiro, J. Song, Y. Song, and S. Ermon, “Csdi: Conditional score-based diffusion models for probabilistic time series imputation,” Advances in Neural Information Processing Systems, vol. 34, pp. 24 804–24 816, 2021.
  22. J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, “Graph neural networks: A review of methods and applications,” AI open, vol. 1, pp. 57–81, 2020.
  23. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, “A comprehensive survey on graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2021.
  24. M. Jin, H. Y. Koh, Q. Wen, D. Zambon, C. Alippi, G. I. Webb, I. King, and S. Pan, “A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection,” arXiv preprint arXiv:2307.03759, 2023.
  25. A. Cini, I. Marisca, and C. Alippi, “Filling the g_ap_s: Multivariate time series imputation by graph neural networks,” in International Conference on Learning Representations, 2022.
  26. G. D. Felice, A. Cini, D. Zambon, V. Gusev, and C. Alippi, “Graph-based virtual sensing from sparse and partial multivariate observations,” in International Conference on Learning Representations, 2024.
  27. M. Zhao, C. Taal, S. Baggerohr, and O. Fink, “Virtual sensor for real-time bearing load prediction using heterogeneous temporal graph neural networks,” arXiv preprint arXiv:2404.02304, 2024.
  28. M. Zhao and O. Fink, “DyEdgeGAT: Dynamic Edge via Graph Attention for Early Fault Detection in IIoT Systems,” IEEE Internet of Things Journal, pp. 1–1, 2024.
  29. A. Deng and B. Hooi, “Graph neural network-based anomaly detection in multivariate time series,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 5, 2021, pp. 4027–4035.
  30. H. Lund, S. Werner, R. Wiltshire, S. Svendsen, J. E. Thorsen, F. Hvelplund, and B. V. Mathiesen, “4th generation district heating (4gdh): Integrating smart thermal grids into future sustainable energy systems,” Energy, vol. 68, pp. 1–11, 2014.
  31. J. van Dreven, V. Boeva, S. Abghari, H. Grahn, J. Al Koussa, and E. Motoasca, “Intelligent approaches to fault detection and diagnosis in district heating: Current trends, challenges, and opportunities,” Electronics, vol. 12, no. 6, p. 1448, 2023.
  32. S. Månsson, P.-O. J. Kallioniemi, M. Thern, T. Van Oevelen, and K. Sernhed, “Faults in district heating customer installations and ways to approach them: Experiences from swedish utilities,” Energy, vol. 180, pp. 163–174, 2019.
  33. D. S. Østergaard, K. M. Smith, M. Tunzi, and S. Svendsen, “Low-temperature operation of heating systems to enable 4th generation district heating: A review,” Energy, vol. 248, p. 123529, 2022.
  34. P. Xue, Y. Jiang, Z. Zhou, X. Chen, X. Fang, and J. Liu, “Machine learning-based leakage fault detection for district heating networks,” Energy and Buildings, vol. 223, p. 110161, 2020.
  35. P. Wang, P. Poovendran, and K. B. Manokaran, “Fault detection and control in integrated energy system using machine learning,” Sustainable Energy Technologies and Assessments, vol. 47, p. 101366, 2021.
  36. S. Liu, R. Ying, H. Dong, L. Li, T. Xu, Y. Rong, P. Zhao, J. Huang, and D. Wu, “Local augmentation for graph neural networks,” in International Conference on Machine Learning (ICML).   PMLR, 2022, pp. 14 054–14 072.
  37. T. Zhao, Y. Liu, L. Neves, O. Woodford, M. Jiang, and N. Shah, “Data augmentation for graph neural networks,” in Proceedings of the aaai conference on artificial intelligence, vol. 35, no. 12, 2021, pp. 11 015–11 023.
  38. Y. Xue, Y. Liao, X. Chen, and J. Zhao, “Node augmentation methods for graph neural network based object classification,” in 2021 2nd International Conference on Computing and Data Science (CDS).   IEEE, 2021, pp. 556–561.
  39. A. Ortega, P. Frossard, J. Kovačević, J. M. F. Moura, and P. Vandergheynst, “Graph signal processing: Overview, challenges, and applications,” Proceedings of the IEEE, vol. 106, no. 5, pp. 808–828, 2018.
  40. D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst, “The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains,” IEEE signal processing magazine, vol. 30, no. 3, pp. 83–98, 2013.
  41. M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional neural networks on graphs with fast localized spectral filtering,” Advances in neural information processing systems, vol. 29, 2016.
  42. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
  43. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in International Conference on Learning Representations, 2018.
  44. S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?” in International Conference on Learning Representations, 2022.
  45. Y. Shi, Z. Huang, S. Feng, H. Zhong, W. Wang, and Y. Sun, “Masked label prediction: Unified message passing model for semi-supervised classification,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2021, pp. 1548–1554.
  46. A. A. Cook, G. Mısırlı, and Z. Fan, “Anomaly detection for iot time-series data: A survey,” IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6481–6494, 2019.
  47. G. Örn Garðarsson, F. Boem, and L. Toni, “Graph-based learning for leak detection and localisation in water distribution networks,” IFAC-PapersOnLine, vol. 55, no. 6, pp. 661–666, 2022.
  48. M. Tariq, M. Ali, F. Naeem, and H. V. Poor, “Vulnerability assessment of 6g-enabled smart grid cyber–physical systems,” IEEE internet of things journal, vol. 8, no. 7, pp. 5468–5475, 2020.
  49. A. Zanfei, A. Menapace, B. M. Brentan, R. Sitzenfrei, and M. Herrera, “Shall we always use hydraulic models? a graph neural network metamodel for water system calibration and uncertainty assessment,” Water Research, vol. 242, p. 120264, 2023.
  50. F. Witte and I. Tuschy, “TESPy: Thermal Engineering Systems in Python,” Journal of Open Source Software, vol. 5, no. 49, p. 2178, 2020.
  51. M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in ICLR 2019 Workshop on Representation Learning on Graphs and Manifolds, 2019.
  52. X. Wu, A. Ajorlou, Z. Wu, and A. Jadbabaie, “Demystifying oversmoothing in attention-based graph neural networks,” in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 35 084–35 106.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets