Papers
Topics
Authors
Recent
Search
2000 character limit reached

ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study

Published 12 Jan 2024 in cs.SE, cs.AI, and cs.LG | (2401.06513v1)

Abstract: Machine learning (ML), especially with the emergence of LLMs, has significantly transformed various industries. However, the transition from ML model prototyping to production use within software systems presents several challenges. These challenges primarily revolve around ensuring safety, security, and transparency, subsequently influencing the overall robustness and trustworthiness of ML models. In this paper, we introduce ML-On-Rails, a protocol designed to safeguard ML models, establish a well-defined endpoint interface for different ML tasks, and clear communication between ML providers and ML consumers (software engineers). ML-On-Rails enhances the robustness of ML models via incorporating detection capabilities to identify unique challenges specific to production ML. We evaluated the ML-On-Rails protocol through a real-world case study of the MoveReminder application. Through this evaluation, we emphasize the importance of safeguarding ML models in production.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. [n. d.]. Adversarial Machine Learning. https://ibm.co/36fhajg [Online]. Accessed on 2023-11-19.
  2. [n. d.]. Responsible AI Practices. https://ai.google/responsibilities/responsible-ai-practices/?category=security [Online]. Accessed on 2023-11-19.
  3. [n. d.]. Securing the Future of AI and ML at Microsoft. https://docs.microsoft.com/en-us/security/securing-artificial-intelligence-machine-learning [Online]. Accessed on 2023-11-19.
  4. [n. d.]. Status codes in HTTP. https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml Accessed on 2023-08-08.
  5. SSDPose: A Single Shot Deep Pose Estimation and Analysis. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). IEEE, 1862–1868.
  6. Rgb-d fall detection via deep residual convolutional lstm networks. In 2018 Digital Image Computing: Techniques and Applications (DICTA). IEEE, 1–7.
  7. Software engineering for machine learning: A case study. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 291–300.
  8. FactSheets: Increasing trust in AI services through supplier’s declarations of conformity. IBM Journal of Research and Development 63, 4/5 (2019), 6–1.
  9. Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when its lying. arXiv preprint arXiv:2304.13734 (2023).
  10. Challenges in the Deployment and Operation of Machine Learning in Practice.. In ECIS, Vol. 1.
  11. Michael Balas and Edsel B Ing. 2023. Conversational ai models for ophthalmic diagnosis: Comparison of chatgpt and the isabel pro differential diagnosis generator. JFO Open Ophthalmology 1 (2023), 100005.
  12. End to end learning for self-driving cars. arXiv preprint arXiv:1604.07316 (2016).
  13. Deepdriving: Learning affordance for direct perception in autonomous driving. In Proceedings of the IEEE international conference on computer vision. 2722–2730.
  14. Robust physical-world attacks on deep learning visual classification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1625–1634.
  15. Large language models improve Alzheimer’s disease diagnosis using multi-modality data. arXiv preprint arXiv:2305.19280 (2023).
  16. Deep learning. MIT press.
  17. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).
  18. Symbolic execution for attribution and attack synthesis in neural networks. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 282–283.
  19. An empirical study towards characterizing deep learning development and deployment across different frameworks and platforms. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 810–822.
  20. Unsolved problems in ml safety. arXiv preprint arXiv:2109.13916 (2021).
  21. Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR.
  22. Misspelling Correction with Pre-trained Contextual Language Model. In 2020 IEEE 19th International Conference on Cognitive Informatics & Cognitive Computing (ICCI* CC). IEEE Computer Society, 144–149.
  23. Automatic language identification in texts: A survey. Journal of Artificial Intelligence Research 65 (2019), 675–782.
  24. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. arXiv preprint arXiv:2302.05733 (2023).
  25. Adversarial machine learning-industry perspectives. In 2020 IEEE Security and Privacy Workshops (SPW). IEEE, 69–75.
  26. Fumihiro Kumeno. 2019. Sofware engneering challenges for machine learning applications: A literature review. Intelligent Decision Technologies 13, 4 (2019), 463–476.
  27. Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks. In ICLR.
  28. Energy-based out-of-distribution detection. Advances in Neural Information Processing Systems 33 (2020), 21464–21475.
  29. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017).
  30. A unified gradient regularization family for adversarial examples. In 2015 IEEE international conference on data mining. IEEE, 301–309.
  31. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896 (2023).
  32. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220–229.
  33. Leveraging Implicit Feedback from Deployment Data in Dialogue. arXiv preprint arXiv:2307.14117 (2023).
  34. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813 (2023).
  35. Failing loudly: An empirical study of methods for detecting dataset shift. In Advances in Neural Information Processing Systems (NIPS). 1396–1408.
  36. Inioluwa Deborah Raji and Jingying Yang. 2019. About ml: Annotation and benchmarking on understanding and transparency of machine learning lifecycles. arXiv preprint arXiv:1912.06166 (2019).
  37. NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails. arXiv preprint arXiv:2310.10501 (2023).
  38. On challenges in machine learning model management. IEEE Data Engineering Bulletin 41, 4 (2018), 5–15.
  39. Identifying and Extracting Rare Disease Phenotypes with Large Language Models. arXiv preprint arXiv:2306.12656 (2023).
  40. React: Out-of-distribution detection with rectified activations. Advances in Neural Information Processing Systems 34 (2021), 144–157.
  41. Yiyou Sun and Yixuan Li. 2022. Dice: Leveraging sparsification for out-of-distribution detection. In European Conference on Computer Vision. Springer, 691–708.
  42. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations, ICLR 2014.
  43. Deeptest: Automated testing of deep-neural-network-driven autonomous cars. In Proceedings of the 40th International Conference on Software Engineering. 303–314.
  44. A reproduction of Apple’s bi-directional LSTM models for language identification in short strings. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop. 36–42.
  45. Erik Tromp and Mykola Pechenizkiy. 2011. Graph-based n-gram language identification on short texts. In Proc. 20th Machine Learning conference of Belgium and The Netherlands. 27–34.
  46. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483 (2023).
  47. Generalized out-of-distribution detection: A survey. arXiv preprint arXiv:2110.11334 (2021).
  48. Skingpt-4: An interactive dermatology diagnostic system with visual large language model. (2023).
  49. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023).
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 1 like about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free