2000 character limit reached
DP-CRE: Continual Relation Extraction via Decoupled Contrastive Learning and Memory Structure Preservation (2403.02718v1)
Published 5 Mar 2024 in cs.CL
Abstract: Continuous Relation Extraction (CRE) aims to incrementally learn relation knowledge from a non-stationary stream of data. Since the introduction of new relational tasks can overshadow previously learned information, catastrophic forgetting becomes a significant challenge in this domain. Current replay-based training paradigms prioritize all data uniformly and train memory samples through multiple rounds, which would result in overfitting old tasks and pronounced bias towards new tasks because of the imbalances of the replay set. To handle the problem, we introduce the DecouPled CRE (DP-CRE) framework that decouples the process of prior information preservation and new knowledge acquisition. This framework examines alterations in the embedding space as new relation classes emerge, distinctly managing the preservation and acquisition of knowledge. Extensive experiments show that DP-CRE significantly outperforms other CRE baselines across two datasets.
- Refining sample embeddings with relation prototypes to enhance continual relation extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 232–243.
- Pathnet: Evolution channels gradient descent in super neural networks.
- Continual relation learning via episodic memory activation and reconsolidation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6429–6440.
- Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
- Improving continual relation extraction through prototypical contrastive learning. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1885–1895.
- Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, volume 1, page 2.
- Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673.
- Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526.
- Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947.
- David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30.
- Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3589–3599.
- Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7765–7773.
- icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010.
- Ozan Sener and Vladlen Koltun. 2018. Multi-task learning as multi-objective optimization. Advances in neural information processing systems, 31.
- Sentence embedding alignment for lifelong relation extraction. In Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics (ACL).
- Less is more: Rethinking state-of-the-art continual relation extraction models with a frustratingly easy but effective approach. arXiv preprint arXiv:2209.00243.
- Learning robust representations for continual relation extraction via adversarial class augmentation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6264–6278.
- Curriculum-meta learning for order-robust continual relation extraction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10363–10369.
- Enhancing continual relation extraction via classifier decomposition.
- Learning adaptive embedding considering incremental class. IEEE Trans. Knowl. Data Eng., 35(3):2736–2749.
- Adaptive deep models for incremental learning: Considering capacity scalability and sustainability. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 74–82, Anchorage, AK.
- Prompt-based prototypical framework for continual relation extraction. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:2801–2813.
- Position-aware attention and supervised data improve slot filling. In Conference on Empirical Methods in Natural Language Processing.
- Consistent representation learning for continual relation extraction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 3402–3411.
- Improving continual relation extraction by distinguishing analogous semantics. In Association for Computational Linguistics: ACL 2023.
- A survey on the construction methods and applications of sci-tech big data knowledge graph. Sci. Sin. Inf, 50(7):957.
- Mengyi Huang (1 paper)
- Meng Xiao (114 papers)
- Ludi Wang (3 papers)
- Yi Du (67 papers)