Nissist: An Incident Mitigation Copilot based on Troubleshooting Guides (2402.17531v2)
Abstract: Effective incident management is pivotal for the smooth operation of enterprises-level cloud services. In order to expedite incident mitigation, service teams compile troubleshooting knowledge into Troubleshooting Guides (TSGs) accessible to on-call engineers (OCEs). While automated pipelines are enabled to resolve the most frequent and easy incidents, there still exist complex incidents that require OCEs' intervention. However, TSGs are often unstructured and incomplete, which requires manual interpretation by OCEs, leading to on-call fatigue and decreased productivity, especially among new-hire OCEs. In this work, we propose Nissist which leverages TSGs and incident mitigation histories to provide proactive suggestions, reducing human intervention. Leveraging LLMs (LLM), Nissist extracts insights from unstructured TSGs and historical incident mitigation discussions, forming a comprehensive knowledge base. Its multi-agent system design enhances proficiency in precisely discerning user queries, retrieving relevant information, and delivering systematic plans consecutively. Through our user case and experiment, we demonstrate that Nissist significant reduce Time to Mitigate (TTM) in incident mitigation, alleviating operational burdens on OCEs and improving service reliability. Our demo is available at https://aka.ms/nissist_demo.
- Empowering practical root cause analysis by large language models for cloud incidents. arXiv preprint arXiv:2305.15778, 2023.
- How to mitigate the incident? an effective troubleshooting guide recommendation technique for online service systems. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 1410–1420, 2020.
- Xpert: Empowering incident management with query recommendations via large language models. arXiv preprint arXiv:2312.11988, 2023.
- Learning to answer programming questions with software documentation through social context embedding. Information Sciences, 448:36–52, 2018.
- Modelling the ‘hurried’bug report reading process to summarize bug reports. Empirical Software Engineering, 20:516–548, 2015.
- Caroline O’Donovan. Amazon’s cloud computing service recovers from outage. Accessed 23 June 2023. https://www.washingtonpost.com/technology/2023/06/13/amazon-aws-down-outage/, 2023.
- Autotsg: Learning and synthesis for incident troubleshooting. ESEC/FSE 2022, New York, NY, USA, 2022. Association for Computing Machinery.
- React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023.