Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
126 tokens/sec
GPT-4o
47 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Neural network-based CUSUM for online change-point detection (2210.17312v6)

Published 31 Oct 2022 in cs.LG, stat.ME, and stat.ML

Abstract: Change-point detection, detecting an abrupt change in the data distribution from sequential data, is a fundamental problem in statistics and machine learning. CUSUM is a popular statistical method for online change-point detection due to its efficiency from recursive computation and constant memory requirement, and it enjoys statistical optimality. CUSUM requires knowing the precise pre- and post-change distribution. However, post-change distribution is usually unknown a priori since it represents anomaly and novelty. Classic CUSUM can perform poorly when there is a model mismatch with actual data. While likelihood ratio-based methods encounter challenges facing high dimensional data, neural networks have become an emerging tool for change-point detection with computational efficiency and scalability. In this paper, we introduce a neural network CUSUM (NN-CUSUM) for online change-point detection. We also present a general theoretical condition when the trained neural networks can perform change-point detection and what losses can achieve our goal. We further extend our analysis by combining it with the Neural Tangent Kernel theory to establish learning guarantees for the standard performance metrics, including the average run length (ARL) and expected detection delay (EDD). The strong performance of NN-CUSUM is demonstrated in detecting change-point in high-dimensional data using both synthetic and real-world data.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (31)
  1. UC Irvine Machine Learning Repository. https://archive.ics.uci.edu/.
  2. Neutrinos could shed light on why the universe has so much more matter than antimatter. Nature, 580:305, 2020.
  3. Two moments suffice for poisson approximations: the Chen-Stein method. The Annals of Probability, pages 9–25, 1989.
  4. Searching for exotic particles in high-energy physics with deep learning. Nature Communications, 5(1):4308, 2014.
  5. Change point detection in correlation networks. Scientific reports, 6(1):18893, 2016.
  6. Classification logit two-sample testing by neural networks for differentiating near manifold densities. IEEE Transactions on Information Theory, 68(10):6631–6662, 2022.
  7. Neural tangent kernel maximum mean discrepancy. Advances in Neural Information Processing Systems, 34:6658–6670, 2021.
  8. A kernel two-sample test. The Journal of Machine Learning Research, 13(1):723–773, 2012.
  9. Real-time change-point detection: A deep neural network-based adaptive approach for detecting changes in multivariate time series data. Expert Systems with Applications, 209:118260, 2022.
  10. Online neural networks for change-point detection. arXiv preprint arXiv:2010.01388, 2020.
  11. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015.
  12. Training neural networks for sequential change-point detection. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023.
  13. Automatic change-point detection in time series via deep learning. to appear, Journal of the Royal Statistical Society Series B, arXiv preprint arXiv:2211.03860, 2022.
  14. Scan B-statistic for kernel change-point detection. Sequential Analysis, 38(4):503–544, 2019.
  15. Detecting changes in dynamic events over networks. IEEE Transactions on Signal and Information Processing over Networks, 3(2):346–359, 2017.
  16. Density estimation using deep generative neural networks. Proceedings of the National Academy of Sciences, 118(15):e2101344118, 2021.
  17. A multivariate exponentially weighted moving average control chart. Technometrics, 34(1):46–53, 1992.
  18. George V Moustakides. Extension of Wald’s first lemma to Markov processes. Journal of applied probability, 36(1):48–59, 1999.
  19. Training neural networks for likelihood/density ratio estimation. arXiv preprint arXiv:1911.00405, 2019.
  20. Detecting change points in the large-scale structure of evolving networks. In Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.
  21. Quickest detection. Cambridge University Press, 2008.
  22. David Siegmund. Sequential analysis: tests and confidence intervals. Springer Science & Business Media, 1985.
  23. The statistics of gene mapping, volume 1. Springer, 2007.
  24. Neural estimation of statistical divergences. The Journal of Machine Learning Research, 23(1):5460–5534, 2022.
  25. Sequential analysis: Hypothesis testing and changepoint detection. CRC Press, 2014.
  26. Online kernel cusum for change-point detection. arXiv preprint arXiv:2211.15070, 2022.
  27. Window-limited CUSUM for sequential change detection. IEEE Transactions on Information Theory, 69(9):5990–6005, 2023.
  28. Sequential (quickest) change detection: Classical results and new directions. IEEE Journal on Selected Areas in Information Theory, 2(2):494–514, 2021.
  29. Sequential multi-sensor change-point detection. The Annals of Statistics, 41(2):670 – 692, 2013.
  30. Benjamin Yakir. Multi-channel change-point detection statistic with applications in dna copy-number variation and sequential monitoring. In Proceedings of Second International Workshop in Sequential Methodologies, pages 15–17, 2009.
  31. On the discrimination-generalization tradeoff in GANs. In International Conference on Learning Representations, 2018.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now