Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
102 tokens/sec
GPT-4o
59 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
50 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Counterfactual Reasoning with Knowledge Graph Embeddings (2403.06936v1)

Published 11 Mar 2024 in cs.LG, cs.AI, and cs.CL

Abstract: Knowledge graph embeddings (KGEs) were originally developed to infer true but missing facts in incomplete knowledge repositories. In this paper, we link knowledge graph completion and counterfactual reasoning via our new task CFKGR. We model the original world state as a knowledge graph, hypothetical scenarios as edges added to the graph, and plausible changes to the graph as inferences from logical rules. We create corresponding benchmark datasets, which contain diverse hypothetical scenarios with plausible changes to the original knowledge graph and facts that should be retained. We develop COULDD, a general method for adapting existing knowledge graph embeddings given a hypothetical premise, and evaluate it on our benchmark. Our results indicate that KGEs learn patterns in the graph without explicit training. We further observe that KGEs adapted with COULDD solidly detect plausible counterfactual changes to the graph that follow these patterns. An evaluation on human-annotated data reveals that KGEs adapted with COULDD are mostly unable to recognize changes to the graph that do not follow learned inference rules. In contrast, ChatGPT mostly outperforms KGEs in detecting plausible changes to the graph but has poor knowledge retention. In summary, CFKGR connects two previously distinct areas, namely KG completion and counterfactual reasoning.

View on arXiv
Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Towards a unified framework for fair and stable graph representation learning. In Uncertainty in Artificial Intelligence, pages 2114–2124. PMLR.
  2. A next-generation hyperparameter optimization framework. In Proceedings of ACM SIGKDD, pages 2623–2631.
  3. Robert J Aumann. 1995. Backward induction and common knowledge of rationality. Games and economic Behavior, 8(1):6–19.
  4. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5185–5194, Hong Kong, China. Association for Computational Linguistics.
  5. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc.
  6. LibKGE - A knowledge graph embedding library for reproducible research. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 165–174.
  7. Are missing links predictable? an inferential benchmark for knowledge graph completion. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 6855–6865, Online. Association for Computational Linguistics.
  8. Knowledge graph completion with counterfactual augmentation. In Proceedings of the ACM Web Conference 2023, pages 2611–2620.
  9. Convolutional 2d knowledge graph embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
  10. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(61):2121–2159.
  11. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9:1012–1031.
  12. Jörg Frohberg and Frank Binder. 2022. CRASS: A novel data set and benchmark to test counterfactual reasoning of large language models. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2126–2140, Marseille, France. European Language Resources Association.
  13. Counterfactual learning on graphs: A survey. arXiv preprint arXiv:2304.01391.
  14. Joseph Y Halpern. 1999. Hypothetical knowledge and counterfactual reasoning. International Journal of Game Theory, 28:315–330.
  15. Andrew F Hayes and Klaus Krippendorff. 2007. Answering the call for a standard reliability measure for coding data. Communication methods and measures, 1(1):77–89.
  16. Knowledge graphs. ACM Computing Surveys (CSUR), 54(4):1–37.
  17. On the instrumental value of hypothetical and counterfactual thought. In Proceedings of the 40th Annual Meeting of the Cognitive Science Society, CogSci 2018, Madison, WI, USA, July 25-28, 2018. cognitivesciencesociety.org.
  18. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2):494–514.
  19. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. International Conference on Learning Representations.
  20. Bhushan Kotnis and Vivi Nastase. 2017. Analysis of the impact of negative sampling on link prediction in knowledge graphs. arXiv preprint arXiv:1708.06816.
  21. I could have done otherwise: Availability of counterfactual comparisons informs the sense of agency. Consciousness and cognition, 49:237–244.
  22. Fast and exact rule mining with amie 3. In The Semantic Web, pages 36–52, Cham. Springer International Publishing.
  23. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics, pages 159–174.
  24. Counterfactual reasoning: Testing language models’ understanding of hypothetical scenarios. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 804–815, Toronto, Canada. Association for Computational Linguistics.
  25. Revisiting inferential benchmarks for knowledge graph completion. arXiv preprint arXiv:2306.04814.
  26. A causal-based symbolic reasoning framework for uncertain knowledge graphs. Computers and Electrical Engineering, 105:108541.
  27. Ana Lucic. 2022. Ter hoeve, gabriele tolomei, maarten de rijke, and fabrizio silvestri. cf-gnnexplainer: Counterfactual explanations for graph neural networks. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics, pages 4499–4511.
  28. Learning fair node representations with graph counterfactual fairness. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 695–703.
  29. John McCarthy. 2000. Free will-even for robots. Journal of experimental & theoretical artificial intelligence, 12(3):341–352.
  30. Anytime bottom-up rule learning for knowledge graph completion. In IJCAI, pages 3137–3143.
  31. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, page 809–816, Madison, WI, USA. Omnipress.
  32. Danilo Numeroso and Davide Bacciu. 2021. Meg: Generating molecular counterfactual explanations for deep graph networks. In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–8. IEEE.
  33. Counterfactual vision-and-language navigation: Unravelling the unseen. Advances in Neural Information Processing Systems, 33:5296–5307.
  34. Ensemble approaches for graph counterfactual explanations.
  35. Counterfactual story reasoning and generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5043–5053, Hong Kong, China. Association for Computational Linguistics.
  36. Eva Rafetseder and Josef Perner. 2014. Counterfactual reasoning: Sharpening conceptual distinctions in developmental studies. Child development perspectives, 8(1):54–58.
  37. Raymond Reiter. 1978. On Closed World Data Bases, pages 55–76. Springer US, Boston, MA.
  38. Drum: End-to-end differentiable rule mining on knowledge graphs. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
  39. Drum: End-to-end differentiable rule mining on knowledge graphs. Advances in Neural Information Processing Systems, 32.
  40. Tara Safavi and Danai Koutra. 2020. CoDEx: A Comprehensive Knowledge Graph Completion Benchmark. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8328–8350, Online. Association for Computational Linguistics.
  41. Counterfactual-guided and curiosity-driven multi-hop reasoning over knowledge graph. In Database Systems for Advanced Applications: 27th International Conference, DASFAA 2022, Virtual Event, April 11–14, 2022, Proceedings, Part I, page 171–179, Berlin, Heidelberg. Springer-Verlag.
  42. Complex embeddings for simple link prediction. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 2071–2080, New York, New York, USA. PMLR.
  43. Ledyard R Tucker. 1966. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311.
  44. Cognitive neuroscience of human counterfactual reasoning. Frontiers in human neuroscience, 9:420.
  45. Denny Vrandečić and Markus Krötzsch. 2014. Wikidata: A free collaborative knowledgebase. Commun. ACM, 57(10):78–85.
  46. Incorporating prior knowledge from counterfactuals into knowledge graph reasoning. Knowledge-Based Systems, 223:107035.
  47. Counterfactual and factual reasoning over hypergraphs for interpretable clinical predictions on ehr. In Machine Learning for Health, pages 259–278. PMLR.
  48. A multi-view confidence-calibrated framework for fair and stable graph representation learning. In 2021 IEEE International Conference on Data Mining (ICDM), pages 1493–1498. IEEE.
  49. Learning from counterfactual links for link prediction. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 26911–26926. PMLR.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (3)
  1. Lena Zellinger (3 papers)
  2. Andreas Stephan (11 papers)
  3. Benjamin Roth (48 papers)

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now