GreenKGC: A Lightweight Knowledge Graph Completion Method (2208.09137v2)
Abstract: Knowledge graph completion (KGC) aims to discover missing relationships between entities in knowledge graphs (KGs). Most prior KGC work focuses on learning embeddings for entities and relations through a simple scoring function. Yet, a higher-dimensional embedding space is usually required for a better reasoning capability, which leads to a larger model size and hinders applicability to real-world problems (e.g., large-scale KGs or mobile/edge computing). A lightweight modularized KGC solution, called GreenKGC, is proposed in this work to address this issue. GreenKGC consists of three modules: representation learning, feature pruning, and decision learning, to extract discriminant KG features and make accurate predictions on missing relationships using classifiers and negative sampling. Experimental results demonstrate that, in low dimensions, GreenKGC can outperform SOTA methods in most datasets. In addition, low-dimensional GreenKGC can achieve competitive or even better performance against high-dimensional models with a much smaller model size.
- TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5185–5194, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1522. URL https://aclanthology.org/D19-1522.
- Multi-relational poincaré graph embeddings. Advances in Neural Information Processing Systems, 32, 2019.
- Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250, 2008.
- Translating embeddings for modeling multi-relational data. Advances in neural information processing systems, 26, 2013.
- A semantic matching energy function for learning with multi-relational data. Machine Learning, 94(2):233–259, 2014.
- Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001.
- Classification and regression trees. Routledge, 2017.
- Low-dimensional hyperbolic knowledge graph embeddings. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6901–6914, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.617. URL https://aclanthology.org/2020.acl-main.617.
- PairRE: Knowledge graph embeddings via paired relation vectors. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4360–4369, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.336. URL https://aclanthology.org/2021.acl-long.336.
- XGBoost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785–794, 2016.
- Improving efficiency and accuracy in multilingual entity extraction. In Proceedings of the 9th international conference on semantic systems, pages 121–124, 2013.
- Convolutional 2D knowledge graph embeddings. In Thirty-second AAAI conference on artificial intelligence, 2018.
- Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop, 2015. URL http://arxiv.org/abs/1503.02531.
- Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 541–550, 2011.
- Open graph benchmark: Datasets for machine learning on graphs. Advances in neural information processing systems, 33:22118–22133, 2020.
- Knowledge graph embedding based question answering. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 105–113, 2019.
- Type-constrained representation learning in knowledge graphs. In International semantic web conference, pages 640–655. Springer, 2015.
- Green learning: Introduction, examples and outlook. Journal of Visual Communication and Image Representation, page 103685, 2022.
- Learning entity and relation embeddings for knowledge graph completion. In Twenty-ninth AAAI conference on artificial intelligence, 2015.
- YAGO3: A knowledge base from multilingual wikipedias. In 7th biennial conference on innovative data systems research. CIDR Conference, 2014.
- George A Miller. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41, 1995.
- A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 327–333, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-2053. URL https://aclanthology.org/N18-2053.
- Improving multi-hop question answering over knowledge graphs using knowledge base embeddings. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 4498–4507, 2020.
- Reasoning with neural tensor networks for knowledge base completion. Advances in neural information processing systems, 26, 2013.
- RotatE: Knowledge graph embedding by relational rotation in complex space. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=HkgEQnRqYQ.
- A re-evaluation of knowledge graph completion methods. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5516–5522, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl-main.489. URL https://aclanthology.org/2020.acl-main.489.
- Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pages 57–66, 2015.
- Complex embeddings for simple link prediction. In International conference on machine learning, pages 2071–2080. PMLR, 2016.
- Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78–85, 2014.
- Mulde: Multi-teacher knowledge distillation for low-dimensional knowledge graph embeddings. In Proceedings of the Web Conference 2021, pages 1716–1726, 2021.
- Explainable reasoning over knowledge graphs for recommendation. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 5329–5336, 2019.
- KGBoost: A classification-based knowledge base completion method with negative sampling. Pattern Recognition Letters, 157:104–111, 2022.
- Knowledge graph embedding by translating on hyperplanes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 28, 2014.
- Reinforcement knowledge graph reasoning for explainable recommendation. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pages 285–294, 2019.
- TransA: An adaptive approach for knowledge graph embedding. CoRR, abs/1509.05490, 2015. URL http://arxiv.org/abs/1509.05490.
- On supervised feature selection from high dimensional feature spaces. APSIPA Transactions on Signal and Information Processing, 11(1), 2022.
- Knowledge graph embedding by reflection transformation. Knowledge-Based Systems, 238:107861, 2022.
- AutoSF: Searching scoring functions for knowledge graph embedding. In 2020 IEEE 36th International Conference on Data Engineering (ICDE), pages 433–444. IEEE, 2020.
- Dualde: Dually distilling knowledge graph embedding for faster and cheaper reasoning. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, pages 1516–1524, 2022.
- Yun-Cheng Wang (17 papers)
- Xiou Ge (13 papers)
- Bin Wang (750 papers)
- C. -C. Jay Kuo (176 papers)