Papers
Topics
Authors
Recent
Search
2000 character limit reached

A Hierarchical Framework with Spatio-Temporal Consistency Learning for Emergence Detection in Complex Adaptive Systems

Published 18 Jan 2024 in cs.MA, cs.AI, and cs.LG | (2401.10300v2)

Abstract: Emergence, a global property of complex adaptive systems (CASs) constituted by interactive agents, is prevalent in real-world dynamic systems, e.g., network-level traffic congestions. Detecting its formation and evaporation helps to monitor the state of a system, allowing to issue a warning signal for harmful emergent phenomena. Since there is no centralized controller of CAS, detecting emergence based on each agent's local observation is desirable but challenging. Existing works are unable to capture emergence-related spatial patterns, and fail to model the nonlinear relationships among agents. This paper proposes a hierarchical framework with spatio-temporal consistency learning to solve these two problems by learning the system representation and agent representations, respectively. Spatio-temporal encoders composed of spatial and temporal transformers are designed to capture agents' nonlinear relationships and the system's complex evolution. Agents' and the system's representations are learned to preserve the spatio-temporal consistency by minimizing the spatial and temporal dissimilarities in a self-supervised manner in the latent space. Our method achieves more accurate detection than traditional methods and deep learning methods on three datasets with well-known yet hard-to-detect emergent behaviors. Notably, our hierarchical framework is generic in incorporating other deep learning methods for agent-level and system-level detection.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (55)
  1. O. Artime and M. De Domenico, “From the origin of life to pandemics: Emergent phenomena in complex systems,” Philosophical Transactions of the Royal Society A, vol. 380, no. 2227, p. 20200410, 2022.
  2. S. Kalantari, E. Nazemi, and B. Masoumi, “Emergence phenomena in self-organizing systems: A systematic literature review of concepts, researches, and future prospects,” Journal of Organizational Computing and Electronic Commerce, vol. 30, no. 3, pp. 224–265, 2020.
  3. E. O’toole, “Decentralised detection of emergence in complex adaptive systems,” Ph.D. dissertation, Trinity College (Dublin, Ireland), 2016.
  4. J. Fromm, “Types and forms of emergence,” arXiv preprint nlin/0506028, 2005.
  5. M. A. Bedau, “Weak emergence,” Philosophical Perspectives, vol. 11, pp. 375–399, 1997.
  6. G. Zeng, Z. Sun, S. Liu, X. Chen, D. Li, J. Wu, and Z. Gao, “Percolation-based health management of complex traffic systems,” Frontiers of Engineering Management, vol. 8, no. 4, pp. 557–571, 2021.
  7. E. O’toole, V. Nallur, and S. Clarke, “Decentralised detection of emergence in complex adaptive systems,” ACM Transactions on Autonomous and Adaptive Systems, vol. 12, no. 1, pp. 1–31, 2017.
  8. S. Aminikhanghahi and D. J. Cook, “A survey of methods for time series change point detection,” Knowledge and Information Systems, vol. 51, pp. 339–367, 2016.
  9. J. Gignoux, G. Chérel, I. D. Davies, S. R. Flint, and E. Lateltin, “Emergence and complex systems: The contribution of dynamic graph theory,” Ecological Complexity, vol. 31, pp. 34–49, 2017.
  10. M. Mnif and C. Müller-Schloer, “Quantitative emergence,” in Organic Computing—A Paradigm Shift for Complex Systems.   Springer, 2011, pp. 39–52.
  11. J. Procházka and K. Olševičová, “Monitoring lane formation of pedestrians: Emergence and entropy,” in Asian Conference on Intelligent Information and Database Systems.   Springer, 2015, pp. 221–228.
  12. Q. Liu, M. He, D. Xu, N. Ding, and Y. Wang, “A mechanism for recognizing and suppressing the emergent behavior of UAV swarm,” Mathematical Problems in Engineering, vol. 2018, 2018.
  13. J. Luo, J. Xin, G. Binghui, H. Zheng, W. Wu, and W. Lv, “Dynamic model and crowd entropy measurement of crowd intelligence system (in Chinese with English abstract),” SCIENTIA SINICA Informationis, vol. 52, no. 1, pp. 99–110, 2022.
  14. D. Fisch, M. Jänicke, B. Sick, and C. Müller-Schloer, “Quantitative emergence–A refined approach based on divergence measures,” in Fourth IEEE International Conference on Self-Adaptive and Self-Organizing Systems.   IEEE Computer Society, 2010, pp. 94–103.
  15. A. K. Seth, “Measuring emergence via nonlinear granger causality.” in ALIFE, vol. 2008, 2008, pp. 545–552.
  16. R. L. Grossman, M. Sabala, Y. Gu, A. Anand, M. Handley, R. Sulo, and L. Wilkinson, “Discovering emergent behavior from network packet data: Lessons from the angle project,” in Next Generation of Data Mining.   Chapman and Hall/CRC, 2008, pp. 267–284.
  17. M. A. Niazi and A. Hussain, “Sensing emergence in complex systems,” IEEE Sensors Journal, vol. 11, no. 10, pp. 2479–2480, 2011.
  18. E. Santos, Y. Zhao, and S. Gómez, “Automatic emergence detection in complex systems,” Complex., vol. 2017, 2017.
  19. Y. Wang, A. Chakrabarti, D. Sivakoff, and S. Parthasarathy, “Fast change point detection on dynamic social networks,” in Proceedings of the 26th International Joint Conference on Artificial Intelligence, 2017, pp. 2992–2998.
  20. S. Huang, Y. Hitti, G. Rabusseau, and R. Rabbany, “Laplacian change point detection for dynamic graphs,” in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2020, pp. 349–358.
  21. D. Sulem, H. Kenlay, M. Cucuringu, and X. Dong, “Graph similarity learning for change-point detection in dynamic networks,” Machine Learning, vol. 113, pp. 1–44, 2023.
  22. X. Ma, J. Wu, S. Xue, J. Yang, C. Zhou, Q. Z. Sheng, H. Xiong, and L. Akoglu, “A comprehensive survey on graph anomaly detection with deep learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 12, pp. 12 012–12 038, 2023.
  23. A. D. Pazho, G. A. Noghre, A. A. Purkayastha, J. Vempati, O. Martin, and H. Tabkhi, “A comprehensive survey of graph-based deep learning approaches for anomaly detection in complex distributed systems,” arXiv preprint arXiv:2206.04149, 2022.
  24. A. Deng and B. Hooi, “Graph neural network-based anomaly detection in multivariate time series,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 5, 2021, pp. 4027–4035.
  25. Y. Zheng, H. Y. Koh, M. Jin, L. Chi, K. T. Phan, S. Pan, Y.-P. P. Chen, and W. Xiang, “Correlation-aware spatial–temporal graph learning for multivariate time-series anomaly detection,” IEEE Transactions on Neural Networks and Learning Systems, November 2023, doi: 10.1109/TNNLS.2023.3325667.
  26. L. Zhao and L. Akoglu, “On using classification datasets to evaluate graph outlier detection: Peculiar observations and new insights,” vol. 11, no. 3, pp. 151–180, 2023.
  27. C. Qiu, M. Kloft, S. Mandt, and M. Rudolph, “Raising the bar in graph-level anomaly detection,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, 2022, pp. 2196–2203.
  28. A. Protogerou, S. Papadopoulos, A. Drosou, D. Tzovaras, and I. Refanidis, “A graph neural network method for distributed anomaly detection in IoT,” Evolving Systems, vol. 12, no. 1, pp. 19–36, 2021.
  29. L. Wu, H. Lin, C. Tan, Z. Gao, and S. Z. Li, “Self-supervised learning on graphs: Contrastive, generative, or predictive,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 4216–4235, 2023.
  30. S. Tian, R. Wu, L. Shi, L. Zhu, and T. Xiong, “Self-supervised representation learning on dynamic graphs,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 1814–1823.
  31. X. Liu, Y. Liang, C. Huang, Y. Zheng, B. Hooi, and R. Zimmermann, “When do contrastive learning signals help spatio-temporal graph forecasting?” in Proceedings of the 30th International Conference on Advances in Geographic Information Systems, 2022, pp. 1–12.
  32. L. Pei, Y. Cao, Y. Kang, Z. Xu, and Z. Zhao, “Self-supervised spatiotemporal clustering of vehicle emissions with graph convolutional network,” IEEE Transactions on Neural Networks and Learning Systems, July 2023, doi: 10.1109/TNNLS.2023.3293463.
  33. Q. Zhang, C. Huang, L. Xia, Z. Wang, Z. Li, and S. Yiu, “Automated spatio-temporal graph contrastive learning,” in Proceedings of the ACM Web Conference, 2023, pp. 295–305.
  34. S. Ranshous, S. Shen, D. Koutra, S. Harenberg, C. Faloutsos, and N. F. Samatova, “Anomaly detection in dynamic networks: A survey,” WIREs Computational Statistics, vol. 7, no. 3, pp. 223–247, 2015.
  35. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A comprehensive survey on graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 1, pp. 4–24, 2020.
  36. Q. He, Z. Dong, F. Chen, S. Deng, W. Liang, and Y. Yang, “Pyramid: Enabling hierarchical neural networks with edge computing,” in Proceedings of the ACM Web Conference, 2022, pp. 1860–1870.
  37. J. Skarding, B. Gabrys, and K. Musial, “Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey,” IEEE Access, vol. 9, pp. 79 143–79 168, 2021.
  38. S. Wang, J. Cao, and P. S. Yu, “Deep learning for spatio-temporal data mining: A survey,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 8, pp. 3681–3700, 2022.
  39. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, pp. 5998–6008, 2017.
  40. J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar et al., “Bootstrap your own latent: A new approach to self-supervised learning,” Advances in Neural Information Processing Systems, vol. 33, pp. 21 271–21 284, 2020.
  41. C. Liu, Y. Zhan, J. Wu, C. Li, B. Du, W. Hu, T. Liu, and D. Tao, “Graph pooling for graph neural networks: Progress, challenges, and opportunities,” arXiv preprint arXiv:2204.07321, 2022.
  42. D. Lee, S. Lee, and H. Yu, “Learnable dynamic temporal pooling for time series classification,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, 2021, pp. 8288–8296.
  43. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in International Conference on Learning Representations, 2015.
  44. J. Sun, J. Zhang, Q. Li, X. Yi, Y. Liang, and Y. Zheng, “Predicting citywide crowd flows in irregular regions using multi-view graph convolutional networks,” IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 5, pp. 2348–2359, 2020.
  45. F. Li, J. Feng, H. Yan, D. Jin, and Y. Li, “Crowd flow prediction for irregular regions with semantic graph attention network,” ACM Transactions on Intelligent Systems and Technology, vol. 13, no. 5, pp. 1–14, 2022.
  46. S. Tisue and U. Wilensky, “Netlogo: A simple environment for modeling complexity,” in International Conference on Complex Systems, vol. 21.   Boston, MA, 2004, pp. 16–21.
  47. C. W. Reynolds, “Flocks, herds and schools: A distributed behavioral model,” in Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, M. C. Stone, Ed.   ACM, 1987, pp. 25–34.
  48. P. Karich and S. Schöder, “Graphhopper.” [Online]. Available: https://graphhopper.com/
  49. C. Truong, L. Oudre, and N. Vayatis, “Selective review of offline change point detection methods,” Signal Processing, vol. 167, p. 107299, 2020.
  50. G. J. Van den Burg and C. K. Williams, “An evaluation of change point detection algorithms,” arXiv preprint arXiv:2003.06222, 2020.
  51. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898–916, 2010.
  52. P. Jaccard, “The distribution of the flora in the alpine zone. 1,” New Phytologist, vol. 11, no. 2, pp. 37–50, 1912.
  53. S. Deldari, D. V. Smith, H. Xue, and F. D. Salim, “Time series change point detection with self-supervised contrastive predictive coding,” in Proceedings of the Web Conference, 2021, pp. 3124–3135.
  54. S. Bai, J. Z. Kolter, and V. Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” arXiv preprint arXiv:1803.01271, 2018.
  55. C. Qiu, T. Pfrommer, M. Kloft, S. Mandt, and M. Rudolph, “Neural transformation learning for deep anomaly detection beyond images,” in International Conference on Machine Learning.   PMLR, 2021, pp. 8703–8714.

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (3)

  1. Siyuan Chen 
  2. Xin Du 
  3. Jiahai Wang 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up