Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Cloud-based XAI Services for Assessing Open Repository Models Under Adversarial Attacks (2401.12261v4)

Published 22 Jan 2024 in cs.CR and cs.AI

Abstract: The opacity of AI models necessitates both validation and evaluation before their integration into services. To investigate these models, explainable AI (XAI) employs methods that elucidate the relationship between input features and output predictions. The operations of XAI extend beyond the execution of a single algorithm, involving a series of activities that include preprocessing data, adjusting XAI to align with model parameters, invoking the model to generate predictions, and summarizing the XAI results. Adversarial attacks are well-known threats that aim to mislead AI models. The assessment complexity, especially for XAI, increases when open-source AI models are subject to adversarial attacks, due to various combinations. To automate the numerous entities and tasks involved in XAI-based assessments, we propose a cloud-based service framework that encapsulates computing components as microservices and organizes assessment tasks into pipelines. The current XAI tools are not inherently service-oriented. This framework also integrates open XAI tool libraries as part of the pipeline composition. We demonstrate the application of XAI services for assessing five quality attributes of AI models: (1) computational cost, (2) performance, (3) robustness, (4) explanation deviation, and (5) explanation resilience across computer vision and tabular cases. The service framework generates aggregated analysis that showcases the quality attributes for more than a hundred combination scenarios.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. TensorFlow: a system for Large-Scale machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265–283.
  2. Kumar Abhishek and Deeksha Kamath. 2022. Attribution-based XAI methods in computer vision: A review. arXiv preprint arXiv:2211.14736 (2022).
  3. Software Engineering for Machine Learning: A Case Study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 291–300. https://doi.org/10.1109/ICSE-SEIP.2019.00042
  4. Current challenges and future opportunities for XAI in machine learning-based clinical decision support systems: a systematic review. Applied Sciences 11, 11 (2021), 5088.
  5. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion 58 (2020), 82–115.
  6. The effects of data quality on machine learning performance. arXiv preprint arXiv:2207.14529 (2022).
  7. Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). Ieee, 39–57.
  8. Bahzad Charbuty and Adnan Abdulazeez. 2021. Classification based on decision tree algorithm for machine learning. Journal of Applied Science and Technology Trends 2, 01 (2021), 20–28.
  9. Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks. In 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). 839–847. https://doi.org/10.1109/WACV.2018.00097
  10. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security. 15–26.
  11. Houdini: Fooling deep structured prediction models. arXiv preprint arXiv:1707.05373 (2017).
  12. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248–255. https://doi.org/10.1109/CVPR.2009.5206848
  13. Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).
  14. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).
  15. Rachel Lea Draelos and Lawrence Carin. 2020. HiResCAM: Faithful location representation in visual attention for explainable 3d medical image classification. arXiv preprint arXiv:2011.08891 (2020).
  16. Computing the cumulative distribution function of the Kolmogorov–Smirnov statistic. Computational statistics & data analysis 34, 1 (2000), 1–15.
  17. Michael Felderer and Rudolf Ramler. 2021. Quality assurance for AI-based systems: overview and challenges. arXiv preprint arXiv:2102.05351 (2021).
  18. Juliana J Ferreira and Mateus S Monteiro. 2020. What are people doing about XAI user experience? A survey on AI explainability research and practice. In Design, User Experience, and Usability. Design for Contemporary Interactive Environments: 9th International Conference, DUXU 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22. Springer, 56–73.
  19. Axiom-based grad-cam: Towards accurate visualization and explanation of cnns. arXiv preprint arXiv:2008.02312 (2020).
  20. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
  21. David Gunning and David Aha. 2019. DARPA’s explainable artificial intelligence (XAI) program. AI magazine 40, 2 (2019), 44–58.
  22. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
  23. Dan Hendrycks and Thomas Dietterich. 2019a. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019).
  24. Dan Hendrycks and Thomas Dietterich. 2019b. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019).
  25. The Analysis and Development of an XAI Process on Feature Contribution Explanation. In 2022 IEEE International Conference on Big Data (Big Data). 5039–5048. https://doi.org/10.1109/BigData55660.2022.10020313
  26. Layercam: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing 30 (2021), 5875–5888.
  27. Predicting the computational cost of deep learning models. In 2018 IEEE international conference on big data (Big Data). IEEE, 3873–3882.
  28. Zijad Kurtanović and Walid Maalej. 2017. Automatically classifying functional and non-functional requirements using supervised machine learning. In 2017 IEEE 25th International Requirements Engineering Conference (RE). Ieee, 490–495.
  29. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019).
  30. Trustworthy AI: From principles to practices. Comput. Surveys 55, 9 (2023), 1–46.
  31. A Trustworthy View on Explainable Artificial Intelligence Method Evaluation. Computer 56, 4 (2023), 50–60. https://doi.org/10.1109/MC.2022.3233806
  32. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.
  33. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 11976–11986.
  34. XAI Systems Evaluation: A Review of Human and Computer-Centred Methods. Applied Sciences 12, 19 (2022), 9423.
  35. Jayawant N Mandrekar. 2010. Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology 5, 9 (2010), 1315–1316.
  36. Christopher D Manning. 2009. An introduction to information retrieval. Cambridge university press.
  37. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1765–1773.
  38. DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2574–2582. https://doi.org/10.1109/CVPR.2016.282
  39. Kshirasagar Naik and Priyadarshi Tripathy. 2011. Software testing and quality assurance: theory and practice. John Wiley & Sons.
  40. The limitations of deep learning in adversarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P). IEEE, 372–387.
  41. Automatic differentiation in PyTorch. In NIPS-W.
  42. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision. 618–626.
  43. Covid-transformer: Interpretable covid-19 detection using vision transformer for healthcare. International Journal of Environmental Research and Public Health 18, 21 (2021), 11086.
  44. Explainable COVID-19 detection using fractal dimension and vision transformer with Grad-CAM on cough sounds. Biocybernetics and Biomedical Engineering 42, 3 (2022), 1066–1080.
  45. One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23, 5 (2019), 828–841.
  46. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).
  47. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Medical Image Analysis 79 (2022), 102470.
  48. Beyond explaining: Opportunities and challenges of XAI-based model improvement. Information Fusion (2022).
  49. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771 (2019).
  50. Cvt: Introducing convolutions to vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision. 22–31.
  51. Generating adversarial examples with adversarial networks. arXiv preprint arXiv:1801.02610 (2018).
  52. SegFormer: Simple and efficient design for semantic segmentation with transformers. Advances in Neural Information Processing Systems 34 (2021), 12077–12090.
  53. Habitat: A Runtime-Based Computational Performance Predictor for Deep Neural Network Training. In USENIX Annual Technical Conference. https://api.semanticscholar.org/CorpusID:236992542
  54. Dast: Data-free substitute training for adversarial attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 234–243.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets