Distributionally Robust Quickest Change Detection using Wasserstein Uncertainty Sets (2309.16171v1)
Abstract: The problem of quickest detection of a change in the distribution of a sequence of independent observations is considered. It is assumed that the pre-change distribution is known (accurately estimated), while the only information about the post-change distribution is through a (small) set of labeled data. This post-change data is used in a data-driven minimax robust framework, where an uncertainty set for the post-change distribution is constructed using the Wasserstein distance from the empirical distribution of the data. The robust change detection problem is studied in an asymptotic setting where the mean time to false alarm goes to infinity, for which the least favorable post-change distribution within the uncertainty set is the one that minimizes the Kullback-Leibler divergence between the post- and the pre-change distributions. It is shown that the density corresponding to the least favorable distribution is an exponentially tilted version of the pre-change density and can be calculated efficiently. A Cumulative Sum (CuSum) test based on the least favorable distribution, which is referred to as the distributionally robust (DR) CuSum test, is then shown to be asymptotically robust. The results are extended to the case where the post-change uncertainty set is a finite union of multiple Wasserstein uncertainty sets, corresponding to multiple post-change scenarios, each with its own labeled data. The proposed method is validated using synthetic and real data examples.
- Bolley, François, Arnaud Guillin, and Cédric Villani. 2007. “Quantitative concentration inequalities for empirical measures on non-compact spaces.” Probability Theory and Related Fields 137: 541–593.
- Gao, Rui, Liyan Xie, Yao Xie, and Huan Xu. 2018. “Robust hypothesis testing using Wasserstein uncertainty sets.” In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 7902–7912.
- Huber, Peter J. 1965. “A robust version of the probability ratio test.” Annals of Mathematical Statistics 36 (6): 1753–1758.
- Kawahara, Yoshinobu, and Masashi Sugiyama. 2012. “Sequential change-point detection based on direct density-ratio estimation.” Statistical Analysis and Data Mining: The ASA Data Science Journal 5 (2): 114–127. https://onlinelibrary.wiley.com/doi/abs/10.1002/sam.10124.
- Kuhn, Daniel, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieezadeh-Abadeh. 2019. “Wasserstein distributionally robust optimization: Theory and applications in machine learning.” In Operations research & management science in the age of analytics, 130–166. Informs.
- Lai, Tze Leung. 1998. “Information bounds and quick detection of parameter changes in stochastic systems.” IEEE Transactions on Information Theory 44 (7): 2917–2929.
- Lai, Tze Leung, and Haipeng Xing. 2010. “Sequential change-point detection when the pre- and post-change parameters are unknown.” Sequential Analysis 29 (2): 162–175.
- Levy, Bernard C. 2008. Principles of Signal Detection and Parameter Estimation. Springer Science & Business Media.
- Li, Shuang, Yao Xie, Hanjun Dai, and Le Song. 2015. “M-statistic for kernel change-point detection.” In Advances in Neural Information Processing Systems, edited by C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Vol. 28, 3366–3374. Curran Associates, Inc.
- Liang, Yuchen, and Venugopal V Veeravalli. 2022. “Non-parametric quickest mean-change detection.” IEEE Transactions on Information Theory 68 (12): 8040–8052.
- Liang, Yuchen, and Venugopal V Veeravalli. 2023. “Quickest change detection with leave-one-out density estimation.” In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5. IEEE.
- Lockhart, Jeffrey W, Gary M Weiss, Jack C Xue, Shaun T Gallagher, Andrew B Grosner, and Tony T Pulickal. 2011. “Design considerations for the WISDM smart phone-based sensor mining architecture.” In Proceedings of the Fifth International Workshop on Knowledge Discovery from Sensor Data, 25–33. ACM.
- Lorden, Gary. 1971. “Procedures for reacting to a change in distribution.” Annals of Mathematical Statistics 42 (6): 1897–1908.
- Magesh, Akshayaa, Zhongchang Sun, Venugopal V Veeravalli, and Shaofeng Zou. 2023. “Robust hypothesis testing with moment constrained uncertainty sets.” In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 1–5. IEEE.
- Mohajerin Esfahani, Peyman, and Daniel Kuhn. 2018. “Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations.” Mathematical Programming 171 (1-2): 115–166.
- Molloy, Timothy L, and Jason J Ford. 2017. “Misspecified and asymptotically minimax robust quickest change detection.” IEEE Transactions on Signal Processing 65 (21): 5730–5742.
- Moustakides, George V. 1986. “Optimal stopping times for detecting changes in distributions.” Annals of Statistics 14 (4): 1379–1387.
- Moustakides, George V, and Kalliopi Basioti. 2019. “Training neural networks for likelihood/density ratio estimation.” https://arxiv.org/abs/1911.00405.
- Mukhopadhyay, Subhas Chandra. 2014. “Wearable sensors for human activity monitoring: A review.” IEEE sensors journal 15 (3): 1321–1330.
- Page, E. S. 1954. “Continuous inspection schemes.” Biometrika 41 (1/2): 100–115.
- Pollak, Moshe. 1978. “Optimality and almost optimality of mixture stopping rules.” Annals of Statistics 6 (4): 910–916.
- Pollak, Moshe. 1985. “Optimal detection of a change in distribution.” Annals of Statistics 13 (1): 206–227.
- Raginsky, Maxim, and Igal Sason. 2013. “Concentration of measure inequalities in information theory, communications, and coding.” Foundations and Trends® in Communications and Information Theory 10 (1-2): 1–246.
- Siegmund, David. 1985. Sequential Analysis: Tests and Confidence Intervals. Springer Series in Statistics. Springer.
- Sun, Zhongchang, and Shaofeng Zou. 2021. “A data-driven approach to robust hypothesis testing using kernel MMD uncertainty sets.” In 2021 IEEE International Symposium on Information Theory (ISIT), 3056–3061. IEEE.
- Unnikrishnan, Jayakrishnan, Venugopal V Veeravalli, and Sean P Meyn. 2011. “Minimax robust quickest change detection.” IEEE Transactions on Information Theory 57 (3): 1604–1614.
- Veeravalli, Venugopal V, and Taposh Banerjee. 2013. “Quickest change detection.” Academic Press Library in Signal Processing: Array and Statistical Signal Processing 3: 209–256.
- Villani, Cédric. 2003. Topics in Optimal Transportation. American Mathematical Society.
- Wang, Jie, and Yao Xie. 2022. “A data-driven approach to robust hypothesis testing using Sinkhorn uncertainty sets.” In Proceedings of the IEEE International Symposium on Information Theory (ISIT), 3315–3320. IEEE.
- Xie, Liyan, Shaofeng Zou, Yao Xie, and Venugopal V Veeravalli. 2021. “Sequential (quickest) change detection: Classical results and new directions.” IEEE Journal on Selected Areas in Information Theory 2 (2): 494–514.