Papers
Topics
Authors
Recent
Search
2000 character limit reached

WASP: A Weight-Space Approach to Detecting Learned Spuriousness

Published 24 Oct 2024 in cs.AI and cs.LG | (2410.18970v3)

Abstract: It is of crucial importance to train machine learning models such that they clearly understand what defines each class in a given task. Though there is a sum of works dedicated to identifying the spurious correlations featured by a dataset that may impact the model's understanding of the classes, all current approaches rely solely on data or error analysis. That is, they cannot point out spurious correlations learned by the model that are not already pointed out by the counterexamples featured in the validation or training sets. We propose a method that transcends this limitation, switching the focus from analyzing a model's predictions to analyzing the model's weights, the mechanism behind the making of the decisions, which proves to be more insightful. Our proposed Weight-space Approach to detecting Spuriousness (WASP) relies on analyzing the weights of foundation models as they drift towards capturing various (spurious) correlations while being fine-tuned on a given dataset. We demonstrate that different from previous works, our method (i) can expose spurious correlations featured by a dataset even when they are not exposed by training or validation counterexamples, (ii) it works for multiple modalities such as image and text, and (iii) it can uncover previously untapped spurious correlations learned by ImageNet-1k classifiers.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (37)
  1. Machine Bias. There’s software used across the country to predict future criminals. And it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing, 2016. Accessed on: 2024-08-31.
  2. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019.
  3. Recognition in terra incognita. In Proceedings of the European conference on computer vision (ECCV), 2018.
  4. Interpreting clip with sparse linear concept embeddings (splice), 2024. URL https://arxiv.org/abs/2402.10376.
  5. Nuanced metrics for measuring unintended bias with real data for text classification. CoRR, abs/1903.04561, 2019. URL http://arxiv.org/abs/1903.04561.
  6. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183–186, 2017. doi: 10.1126/science.aal4230. URL https://www.science.org/doi/abs/10.1126/science.aal4230.
  7. Yake! keyword extraction from single documents using multiple local features. Information Sciences, 509:257–289, 2020. ISSN 0020-0255. doi: https://doi.org/10.1016/j.ins.2019.09.013. URL https://www.sciencedirect.com/science/article/pii/S0020025519308588.
  8. S. Caton and C. Haas. Fairness in machine learning: A survey. ACM Comput. Surv., 56(7), apr 2024. ISSN 0360-0300. doi: 10.1145/3616865. URL https://doi.org/10.1145/3616865.
  9. Environment inference for invariant learning. In International Conference on Machine Learning, pages 2189–2200. PMLR, 2021.
  10. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020.
  11. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15262–15271, 2021.
  12. On feature learning in the presence of spurious correlations. Advances in Neural Information Processing Systems, 35:38516–38532, 2022.
  13. Discovering and mitigating visual biases through keyword explanation. In CVPR, 2024.
  14. Out-of-distribution generalization via risk extrapolation (rex). In Proceedings of the 38th International Conference on Machine Learning, ICML, 2021.
  15. Microsoft COCO: common objects in context. In ECCV, 2014.
  16. Just train twice: Improving group robustness without training group information. In M. Meila and T. Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 6781–6792. PMLR, 18–24 Jul 2021. URL https://proceedings.mlr.press/v139/liu21f.html.
  17. Deep learning face attributes in the wild. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 3730–3738, 2015a. doi: 10.1109/ICCV.2015.425.
  18. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015b.
  19. Spawrious: A benchmark for fine control of spurious correlation biases, 2023. URL https://arxiv.org/abs/2303.05470.
  20. G. A. Miller. Wordnet: a lexical database for english. Commun. ACM, 38(11):39–41, nov 1995. ISSN 0001-0782. doi: 10.1145/219717.219748. URL https://doi.org/10.1145/219717.219748.
  21. Learning from failure: De-biasing classifier from biased classifier. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 20673–20684. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/eddc3427c5d77843c2253f1e799fe933-Paper.pdf.
  22. Automatic differentiation in pytorch. 2017.
  23. Discovering environments with xrm. In Forty-first International Conference on Machine Learning, 2024.
  24. Dataset Shift in Machine Learning. The MIT Press, 12 2008. ISBN 9780262255103. doi: 10.7551/mitpress/9780262170055.001.0001. URL https://doi.org/10.7551/mitpress/9780262170055.001.0001.
  25. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  26. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistics Surveys, 16, 01 2022. doi: 10.1214/21-SS133.
  27. Distributionally robust neural networks. In International Conference on Learning Representations, 2020. URL https://api.semanticscholar.org/CorpusID:213662188.
  28. A Deeper Look at Dataset Bias, pages 37–55. Springer International Publishing, Cham, 2017. ISBN 978-3-319-58347-1. doi: 10.1007/978-3-319-58347-1_2. URL https://doi.org/10.1007/978-3-319-58347-1_2.
  29. V. Vapnik. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10(5):988–999, 1999. doi: 10.1109/72.788640.
  30. S. Verma and J. Rubin. Fairness definitions explained. In Proceedings of the International Workshop on Software Fairness, FairWare ’18, page 1–7, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450357463. doi: 10.1145/3194770.3194776. URL https://doi.org/10.1145/3194770.3194776.
  31. The caltech-ucsd birds-200-2011 dataset. https://api.semanticscholar.org/CorpusID:16119123, 2011. URL https://api.semanticscholar.org/CorpusID:16119123.
  32. Git: A generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205.14100, 2022.
  33. Caltech-ucsd birds 200. 09 2010.
  34. Improving out-of-distribution robustness via selective augmentation. In International Conference on Machine Learning, pages 25407–25437. PMLR, 2022.
  35. Nico++: Towards better benchmarking for domain generalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16036–16047, 2023.
  36. mgte: Generalized long-context text representation and reranking models for multilingual text retrieval. CoRR, abs/2407.19669, 2024. doi: 10.48550/ARXIV.2407.19669. URL https://doi.org/10.48550/arXiv.2407.19669.
  37. Language-guided detection and mitigation of unknown dataset bias, 2024. URL https://arxiv.org/abs/2406.02889.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

Sign Up for Free