Fine-Grained Evaluation for Implicit Discourse Relation Recognition

Abstract: Implicit discourse relation recognition is a challenging task in discourse analysis due to the absence of explicit discourse connectives between spans of text. Recent pre-trained LLMs have achieved great success on this task. However, there is no fine-grained analysis of the performance of these pre-trained LLMs for this task. Therefore, the difficulty and possible directions of this task is unclear. In this paper, we deeply analyze the model prediction, attempting to find out the difficulty for the pre-trained LLMs and the possible directions of this task. In addition to having an in-depth analysis for this task by using pre-trained LLMs, we semi-manually annotate data to add relatively high-quality data for the relations with few annotated examples in PDTB 3.0. The annotated data significantly help improve implicit discourse relation recognition for level-2 senses.