Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
134 tokens/sec
GPT-4o
10 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Expert enhanced dynamic time warping based anomaly detection (2310.02280v1)

Published 2 Oct 2023 in cs.LG and cs.AI

Abstract: Dynamic time warping (DTW) is a well-known algorithm for time series elastic dissimilarity measure. Its ability to deal with non-linear time distortions makes it helpful in variety of data mining tasks. Such a task is also anomaly detection which attempts to reveal unexpected behaviour without false detection alarms. In this paper, we propose a novel anomaly detection method named Expert enhanced dynamic time warping anomaly detection (E-DTWA). It is based on DTW with additional enhancements involving human-in-the-loop concept. The main benefits of our approach comprise efficient detection, flexible retraining based on strong consideration of the expert's detection feedback while retaining low computational and space complexity.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (70)
  1. On improving dynamic time warping for pattern matching. Measurement 45, 1609–1620.
  2. Ganomaly: Semi-supervised anomaly detection via adversarial training. URL: https://arxiv.org/abs/1805.06725, doi:10.48550/ARXIV.1805.06725.
  3. Nearest-neighbor and clustering based anomaly detection algorithms for rapidminer, in: Proc. of the 3rd RapidMiner Community Meeting and Conference (RCOMM 2012), pp. 1–12.
  4. Speech recognition using dynamic time warping, in: 2008 2nd international conference on advances in space technologies, IEEE. pp. 74–79.
  5. Anomaly detection in finance: editors’ introduction, in: KDD 2017 Workshop on Anomaly Detection in Finance, PMLR. pp. 1–7.
  6. Using dynamic time warping to find patterns in time series., in: KDD workshop, Seattle, WA, USA:. pp. 359–370.
  7. Little help makes a big difference: Leveraging active learning to improve unsupervised time series anomaly detection. URL: https://arxiv.org/abs/2201.10323, doi:10.48550/ARXIV.2201.10323.
  8. Lof: identifying density-based local outliers, in: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 93–104.
  9. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis 71. doi:10.1016/j.media.2021.102062.
  10. Together We Learn More: Algorithms and Applications for User-Centric Anomaly Detection. Ph.D. thesis. Halmstad University Press.
  11. Human-in-the-loop outlier detection, in: Maier, D., Pottinger, R., Doan, A., Tan, W., Alawini, A., Ngo, H.Q. (Eds.), Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14-19, 2020, ACM. pp. 19–33. URL: https://doi.org/10.1145/3318464.3389772, doi:10.1145/3318464.3389772.
  12. Anomaly detection: A survey. ACM computing surveys (CSUR) 41, 1–58.
  13. Modified dynamic time warping based on direction similarity for fast gesture recognition. Mathematical Problems in Engineering 2018.
  14. Fast constrained dynamic time warping for similarity measure of time series data. IEEE Access 8, 222841–222858.
  15. Anomaly detection for iot time-series data: A survey. IEEE Internet of Things Journal 7, 6481–6494.
  16. Human-in-the-loop selection of optimal time series anomaly detection methods, in: AAAI Conference on Human Computation and Crowdsourcing, pp. 193–198.
  17. Incorporating expert feedback into active anomaly discovery, in: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 853–858. doi:10.1109/ICDM.2016.0102.
  18. Incorporating feedback into tree-based anomaly detection. URL: https://arxiv.org/abs/1708.09441, doi:10.48550/ARXIV.1708.09441.
  19. Anomaly detection using dynamic time warping, in: 2019 IEEE International Conference on Computational Science and Engineering (CSE) and IEEE International Conference on Embedded and Ubiquitous Computing (EUC), IEEE. pp. 193–198.
  20. Evaluation of machine learning algorithms for anomaly detection, in: 2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security), IEEE. pp. 1–8.
  21. On-line signature recognition based on vq-dtw. Pattern Recognition 40, 981–992.
  22. Extended isolation forest. IEEE Transactions on Knowledge and Data Engineering 33, 1479–1489.
  23. Nearest-neighbor-based active learning for rare category detection, in: Platt, J., Koller, D., Singer, Y., Roweis, S. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 1–8. URL: https://proceedings.neurips.cc/paper/2007/file/2838023a778dfaecdc212708f721b788-Paper.pdf.
  24. Discovering cluster-based local outliers. Pattern recognition letters 24, 1641–1650.
  25. Financial fraud: A review of anomaly detection techniques and recent advances. Expert Syst. Appl. 193. URL: https://doi.org/10.1016/j.eswa.2021.116429, doi:10.1016/j.eswa.2021.116429.
  26. Minimum prediction residual principle applied to speech recognition. IEEE Transactions on acoustics, speech, and signal processing 23, 67–72.
  27. Weighted dynamic time warping for time series classification. Pattern recognition 44, 2231–2240.
  28. Exemplar learning for extremely efficient anomaly detection in real-valued time series. Data mining and knowledge discovery 30, 1427–1454.
  29. On the hidden markov model and dynamic time warping for speech recognition—a unified view. AT&T Bell Laboratories Technical Journal 63, 1213–1243.
  30. Ecg heartbeat classification: A deep transferable representation, in: 2018 IEEE international conference on healthcare informatics (ICHI), IEEE. pp. 443–444.
  31. Using dynamic time warping distances as features for improved time series classification. Data Mining and Knowledge Discovery 30, 283–312.
  32. Automatic hand gesture recognition using hybrid meta-heuristic-based feature selection and classification with dynamic time warping. Computer Science Review 39, 100320.
  33. A fast and accurate similarity measure for long time series classification based on local extrema and dynamic time warping. Expert Systems with Applications 168, 114374.
  34. Evaluating real-time anomaly detection algorithms–the numenta anomaly benchmark, in: 2015 IEEE 14th international conference on machine learning and applications (ICMLA), IEEE. pp. 38–44.
  35. Similarity-measured isolation forest: anomaly detection method for machine monitoring data. IEEE Transactions on Instrumentation and Measurement 70, 1–12.
  36. Adaptively constrained dynamic time warping for time series classification and clustering. Information Sciences 534, 97–116.
  37. Time series classification with ensembles of elastic distance measures. Data Mining and Knowledge Discovery 29, 565–592.
  38. Intelligent data-analytics for condition monitoring: smart grid applications. Academic Press.
  39. Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-Centered A. Shelter Island, NY.
  40. An efficient semi-supervised svm for anomaly detection, in: 2017 International Joint Conference on Neural Networks (IJCNN), pp. 2843–2850. doi:10.1109/IJCNN.2017.7966207.
  41. Federated learning-based anomaly detection for iot security attacks. IEEE Internet of Things Journal .
  42. Performance tradeoffs in dynamic time warping algorithms for isolated word recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing 28, 623–635.
  43. Enhanced network anomaly detection based on deep neural networks. IEEE access 6, 48231–48246.
  44. On-line elastic similarity measures for time series. Pattern Recognition 88, 506–517.
  45. Loci: Fast outlier detection using the local correlation integral, in: Proceedings 19th international conference on data engineering (Cat. No. 03CH37405), IEEE. pp. 315–326.
  46. Learning representations from healthcare time series data for unsupervised anomaly detection, in: 2019 IEEE international conference on big data and smart computing (BigComp), IEEE. pp. 1–7.
  47. Idarts–towards intelligent data analysis and real-time supervision for industry 4.0. Computers in industry 101, 138–146.
  48. Dynamic time warping averaging of time series allows faster and more accurate classification, in: 2014 IEEE international conference on data mining, IEEE. pp. 470–479.
  49. A global averaging method for dynamic time warping, with applications to clustering. Pattern recognition 44, 678–693.
  50. Searching and mining trillions of time series subsequences under dynamic time warping, in: Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 262–270.
  51. Making time-series classification more accurate using learned constraints, in: Proceedings of the 2004 SIAM international conference on data mining, SIAM. pp. 11–22.
  52. Online anomaly detection over big data streams, in: Applied Data Science. Springer, pp. 289–312.
  53. Dynamic programming algorithm optimization for spoken word recognition. IEEE transactions on acoustics, speech, and signal processing 26, 43–49.
  54. f-anogan: Fast unsupervised anomaly detection with generative adversarial networks. Medical image analysis 54, 30–44.
  55. Active Learning Literature Survey. Computer Sciences Technical Report 1648. University of Wisconsin–Madison.
  56. Robust anomaly detection for multivariate time series through stochastic recurrent neural network, in: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2828–2837.
  57. Indexing and classifying gigabytes of time series under time warping, in: Proceedings of the 2017 SIAM international conference on data mining, SIAM. pp. 282–290.
  58. Enhancing effectiveness of outlier detections for low density patterns, in: Pacific-Asia conference on knowledge discovery and data mining, Springer. pp. 535–548.
  59. Anomaly detection on time series, in: 2010 IEEE International Conference on Progress in Informatics and Computing, IEEE. pp. 603–608.
  60. Smart data collection system for brownfield cnc milling machines: A new benchmark dataset for data-driven machine monitoring. Procedia CIRP 107, 131–136.
  61. Anomaly detection for data streams based on isolation forest using scikit-multiflow, in: International Conference on Computational Science and Its Applications, Springer. pp. 15–30.
  62. Iot healthcare analytics: The importance of anomaly detection, in: 2016 IEEE 30th international conference on advanced information networking and applications (AINA), IEEE. pp. 994–997.
  63. Ai^ 2: training a big data machine to defend, in: 2016 IEEE 2nd international conference on big data security on cloud (BigDataSecurity), IEEE international conference on high performance and smart computing (HPSC), and IEEE international conference on intelligent data and security (IDS), IEEE. pp. 49–54.
  64. Practical and white-box anomaly detection through unsupervised and active learning. 2020 29th International Conference on Computer Communications and Networks (ICCCN) , 1–9.
  65. Rlad: Time series anomaly detection through reinforcement learning and active learning. URL: https://arxiv.org/abs/2104.00543, doi:10.48550/ARXIV.2104.00543.
  66. Deep dynamic time warping: end-to-end local representation learning for online signature verification, in: 2019 International Conference on Document Analysis and Recognition (ICDAR), IEEE. pp. 1103–1110.
  67. A survey of human-in-the-loop for machine learning. Future Generation Computer Systems 135, 364–381. URL: https://doi.org/10.1016%2Fj.future.2022.05.014, doi:10.1016/j.future.2022.05.014.
  68. A perspective from human- computer interactionn. ACM , 1072–5520.
  69. A survey of anomaly detection methods in networks, in: 2009 International Symposium on Computer Network and Multimedia Technology, IEEE. pp. 1–3.
  70. Siamese neural network based few-shot learning for anomaly detection in industrial cyber-physical systems. IEEE Transactions on Industrial Informatics 17, 5790–5798.
Citations (13)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now