InsertGNN: Can Graph Neural Networks Outperform Humans in TOEFL Sentence Insertion Problem?

Abstract: Sentence insertion is an interesting NLP problem but received insufficient attention. Existing approaches in sentence ordering, text coherence, and question answering are neither suitable nor good enough at solving it. To bridge this gap, we propose InsertGNN, a simple yet effective model that represents the problem as a graph and adopts a hierarchical graph neural network (GNN) to learn the connection between sentences. We evaluate our method in our newly collected TOEFL dataset and further verify its effectiveness on the larger arXiv dataset using cross-domain learning. Extensive experiments demonstrate that InsertGNN outperforms all baselines by a large margin with an accuracy of 70\%, rivaling the average human test scores.