Unsupervised Relation Extraction from Language Models using Constrained Cloze Completion

Abstract: We show that state-of-the-art self-supervised LLMs can be readily used to extract relations from a corpus without the need to train a fine-tuned extractive head. We introduce RE-Flex, a simple framework that performs constrained cloze completion over pretrained LLMs to perform unsupervised relation extraction. RE-Flex uses contextual matching to ensure that LLM predictions matches supporting evidence from the input corpus that is relevant to a target relation. We perform an extensive experimental study over multiple relation extraction benchmarks and demonstrate that RE-Flex outperforms competing unsupervised relation extraction methods based on pretrained LLMs by up to 27.8 $F_1$ points compared to the next-best method. Our results show that constrained inference queries against a LLM can enable accurate unsupervised relation extraction.