Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Best of Both Worlds: A Pliable and Generalizable Neuro-Symbolic Approach for Relation Classification (2403.03305v1)

Published 5 Mar 2024 in cs.CL and cs.AI

Abstract: This paper introduces a novel neuro-symbolic architecture for relation classification (RC) that combines rule-based methods with contemporary deep learning techniques. This approach capitalizes on the strengths of both paradigms: the adaptability of rule-based systems and the generalization power of neural networks. Our architecture consists of two components: a declarative rule-based model for transparent classification and a neural component to enhance rule generalizability through semantic text matching. Notably, our semantic matcher is trained in an unsupervised domain-agnostic way, solely with synthetic data. Further, these components are loosely coupled, allowing for rule modifications without retraining the semantic matcher. In our evaluation, we focused on two few-shot relation classification datasets: Few-Shot TACRED and a Few-Shot version of NYT29. We show that our proposed method outperforms previous state-of-the-art models in three out of four settings, despite not seeing any human-annotated training data. Further, we show that our approach remains modular and pliable, i.e., the corresponding rules can be locally modified to improve the overall model. Human interventions to the rules for the TACRED relation \texttt{org:parents} boost the performance on that relation by as much as 26\% relative improvement, without negatively impacting the other relations, and without retraining the semantic matching component.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (59)
  1. Amina Adadi and Mohammed Berrada. 2018. Peeking inside the black-box: A survey on explainable artificial intelligence (xai). IEEE Access, 6:52138–52160.
  2. Towards realistic few-shot relation extraction: A new meta dataset and evaluation. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation.
  3. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473.
  4. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In NIPS.
  5. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Understanding the origins of bias in word embeddings. In ICML.
  7. Proceedings of the First Workshop on Pattern-based Approaches to NLP in the Age of Deep Learning. International Conference on Computational Linguistics, Gyeongju, Republic of Korea.
  8. Explainability for natural language processing. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD ’21, page 4033–4034, New York, NY, USA. Association for Computing Machinery.
  9. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
  10. Fewrel 2.0: Towards more challenging few-shot relation classification. In EMNLP/IJCNLP.
  11. Bryce Goodman and Seth Flaxman. 2016. European union regulations on algorithmic decision-making and a "right to explanation". AI Mag., 38:50–57.
  12. Attention guided graph convolutional networks for relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 241–251, Florence, Italy. Association for Computational Linguistics.
  13. Improved pattern learning for bootstrapped entity extraction. In CoNLL.
  14. Umbc_ebiquity-core: Semantic textual similarity systems. In *SEMEVAL.
  15. Marti A. Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In COLING 1992 Volume 2: The 14th International Conference on Computational Linguistics.
  16. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9:1735–1780.
  17. Adversarial examples are not bugs, they are features. ArXiv, abs/1905.02175.
  18. Alon Jacovi and Yoav Goldberg. 2020. Towards faithfully interpretable nlp systems: How should we define and evaluate faithfulness? ArXiv, abs/2004.03685.
  19. Sarthak Jain and Byron C. Wallace. 2019. Attention is not explanation. CoRR, abs/1902.10186.
  20. SpanBERT: Improving pre-training by representing and predicting spans. Transactions of the Association for Computational Linguistics, 8:64–77.
  21. Attention is not only a weight: Analyzing transformers with vector norms. In EMNLP.
  22. Roberta: A robustly optimized bert pretraining approach. ArXiv, abs/1907.11692.
  23. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786.
  24. Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4765–4774. Curran Associates, Inc.
  25. Christopher D. Manning. 2015. Computational linguistics and deep learning. Computational Linguistics, 41:701–707.
  26. The stanford corenlp natural language processing toolkit. In Annual Meeting of the Association for Computational Linguistics.
  27. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In ACL.
  28. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54:1 – 35.
  29. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4034–4043, Marseille, France. European Language Resources Association.
  30. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning.
  31. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  32. "why should i trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, page 1135–1144, New York, NY, USA. Association for Computing Machinery.
  33. Modeling relations and their mentions without labeled text. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 148–163. Springer.
  34. Ellen Riloff. 1993. Automatically constructing a dictionary for information extraction tasks. In AAAI.
  35. Ellen Riloff. 1996. Automatically generating extraction patterns from untagged text. In AAAI/IAAI, Vol. 2.
  36. Learning dictionaries for information extraction by multi-level bootstrapping. In AAAI/IAAI.
  37. Ellen Riloff and Janyce Wiebe. 2003. Learning extraction patterns for subjective expressions. In EMNLP.
  38. Revisiting few-shot relation classification: Evaluation data and classification schemes. Transactions of the Association for Computational Linguistics, 9:691–706.
  39. Timo Schick and Hinrich Schütze. 2020. Exploiting cloze questions for few shot text classification and natural language inference. arXiv preprint arXiv:2001.07676.
  40. Hidden technical debt in machine learning systems. In NIPS.
  41. Lloyd S. Shapley. 1988. A value for n-person games.
  42. Axiomatic attribution for deep networks. ArXiv, abs/1703.01365.
  43. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, page 3104–3112, Cambridge, MA, USA. MIT Press.
  44. Intriguing properties of neural networks. CoRR, abs/1312.6199.
  45. A hierarchical framework for relation extraction with reinforcement learning. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 7072–7079.
  46. Zheng Tang and Mihai Surdeanu. 2023. It takes two flints to make a fire: Multitask learning of neural relation and explanation classifiers. Computational Linguistics. Accepted on 2022.
  47. Erico Tjoa and Cuntai Guan. 2019. A survey on explainable artificial intelligence (xai): Toward medical xai. IEEE Transactions on Neural Networks and Learning Systems, 32:4793–4813.
  48. Patternrank: Jointly ranking patterns and extractions for relation extraction using graph-based algorithms. In Proceedings of the First Workshop on Pattern-based Approaches to NLP in the Age of Deep Learning, pages 1–10.
  49. From examples to rules: Neural guided rule synthesis for information extraction.
  50. Odinson: A fast rule-based information extraction framework. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2183–2191, Marseille, France. European Language Resources Association.
  51. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA. Curran Associates Inc.
  52. Ethical and social risks of harm from language models. ArXiv, abs/2112.04359.
  53. Show, attend and tell: Neural image caption generation with visual attention. In ICML.
  54. Luke: Deep contextualized entity representations with entity-aware self-attention. In Conference on Empirical Methods in Natural Language Processing.
  55. Hierarchical attention networks for document classification. In North American Chapter of the Association for Computational Linguistics.
  56. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP 2017), pages 35–45.
  57. Zexuan Zhong and Danqi Chen. 2020. A frustratingly easy approach for entity and relation extraction. ArXiv, abs/2010.12812.
  58. Wenxuan Zhou and Muhao Chen. 2021. An improved baseline for sentence-level relation extraction. arXiv preprint arXiv:2102.01373.
  59. Nero: A neural rule grounding framework for label-efficient relation extraction. The Web Conference ’20 in arXiv: Computation and Language.
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now