OneLog: Towards End-to-End Training in Software Log Anomaly Detection
Abstract: With the growth of online services, IoT devices, and DevOps-oriented software development, software log anomaly detection is becoming increasingly important. Prior works mainly follow a traditional four-staged architecture (Preprocessor, Parser, Vectorizer, and Classifier). This paper proposes OneLog, which utilizes a single Deep Neural Network (DNN) instead of multiple separate components. OneLog harnesses Convolutional Neural Networks (CNN) at the character level to take digits, numbers, and punctuations, which were removed in prior works, into account alongside the main natural language text. We evaluate our approach in six message- and sequence-based data sets: HDFS, Hadoop, BGL, Thunderbird, Spirit, and Liberty. We experiment with Onelog with single-, multi-, and cross-project setups. Onelog offers state-of-the-art performance in our datasets. Onelog can utilize multi-project datasets simultaneously during training, which suggests our model can generalize between datasets. Multi-project training also improves Onelog performance making it ideal when limited training data is available for an individual project. We also found that cross-project anomaly detection is possible with a single project pair (Liberty and Spirit). Analysis of model internals shows that one log has multiple modes of detecting anomalies and that the model learns manually validated parsing rules for the log messages. We conclude that character-based CNNs are a promising approach toward end-to-end learning in log anomaly detection. They offer good performance and generalization over multiple datasets. We will make our scripts publicly available upon the acceptance of this paper.
- On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7):e0130140, 2015.
- Dana H Ballard. Modular learning in neural networks. In Aaai, volume 647, pages 279–284, 1987.
- End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316, 2016.
- Signature verification using a” siamese” time delay neural network. In Advances in neural information processing systems, pages 737–744, 1994.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
- M. Du and F. Li. Spell: Streaming parsing of system event logs. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 859–864, 2016. 10.1109/ICDM.2016.0103.
- Deeplog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1285–1298, 2017.
- Log message anomaly detection and classification using auto-b/lstm and auto-gru. arXiv preprint arXiv:1911.08744, 2019.
- Tobias Glasmachers. Limits of end-to-end learning. In Asian Conference on Machine Learning, pages 17–32. PMLR, 2017.
- Logbert: Log anomaly detection via bert. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE, 2021.
- Sialog: detecting anomalies in software execution logs using the siamese network. Automated Software Engineering, 29(2):61, Oct 2022. ISSN 1573-7535. 10.1007/s10515-022-00365-7. URL https://doi.org/10.1007/s10515-022-00365-7.
- Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
- Log-based anomaly detection without log parsing. CoRR, abs/2108.01955, 2021. URL https://arxiv.org/abs/2108.01955.
- Yann LeCun et al. Lenet-5, convolutional neural networks. URL: http://yann. lecun. com/exdb/lenet, 20(5):14, 2015.
- Log clustering based problem identification for online service systems. In 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pages 102–111, 2016.
- Detecting anomaly in big data system logs using convolutional neural network. In 2018 IEEE 16th Intl Conf on Dependable, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech), pages 151–158. IEEE, 2018.
- Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018.
- Loganomaly: Unsupervised detection of sequential and quantitative anomalies in unstructured logs. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, volume 7, pages 4739–4745, 2019.
- Self-attentive classification-based anomaly detection in unstructured logs. arXiv preprint arXiv:2008.09340, 2020.
- A. Oliner and J. Stearley. What supercomputers say: A study of five system logs. In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN’07), pages 575–584, 2007. 10.1109/DSN.2007.103.
- Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
- Cluster quality analysis using silhouette score. In 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), pages 747–748. IEEE, 2020.
- Minimum risk training for neural machine translation. arXiv preprint arXiv:1512.02433, 2015.
- Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713, 2016.
- Learning important features through propagating activation differences. In International Conference on Machine Learning, pages 3145–3153. PMLR, 2017.
- Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489, 2016.
- Performance evaluation of the silhouette index. In International conference on artificial intelligence and soft computing, pages 49–58. Springer, 2015.
- Axiomatic attribution for deep networks. In International Conference on Machine Learning, pages 3319–3328. PMLR, 2017.
- Eyal Winter. The shapley value. Handbook of game theory with economic applications, 3:2025–2054, 2002.
- Experimentation in software engineering. Springer Science & Business Media, 2012.
- Fixes that fail: Self-defeating improvements in machine-learning systems. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 11745–11756. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper/2021/file/619427579e7b067421f6aa89d4a8990c-Paper.pdf.
- Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, pages 117–132, 2009.
- Character-level convolutional networks for text classification. arXiv preprint arXiv:1509.01626, 2015.
- Robust log-based anomaly detection on unstable log data. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 807–817, 2019.
- Tools and benchmarks for automated log parsing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), pages 121–130. IEEE, 2019.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
