Consistency and stability of AIOps model interpretations

Determine rigorous criteria and practical methodologies to evaluate and ensure the internal consistency, external consistency, and temporal stability of explanations produced by interpretable AIOps models for incident management, specifically under (i) arrival of new data distributions, (ii) across different models that yield similar predictive performance but differing explanations, and (iii) across successive model updates over time.

Background

The paper emphasizes interpretability as a core desideratum for AIOps solutions and formalizes three dimensions of explanation consistency: internal consistency across repeated trainings of the same model setup, external consistency across different models with similar performance, and time consistency across different time periods. These dimensions are introduced in the desiderata section to guide trustworthy adoption of AIOps in incident management.

In the concluding discussion, the authors explicitly note unresolved questions regarding how to achieve and assess these forms of consistency and stability in practice. Addressing these questions is crucial to build practitioner trust, integrate human-in-the-loop workflows, and ensure that explanations remain reliable as models face new data, architectures, and periodic retraining.