Sentence Centrality Revisited for Unsupervised Summarization (1906.03508v1)

Abstract: Single document summarization has enjoyed renewed interests in recent years thanks to the popularity of neural network models and the availability of large-scale datasets. In this paper we develop an unsupervised approach arguing that it is unrealistic to expect large-scale and high-quality training data to be available or created for different types of summaries, domains, or languages. We revisit a popular graph-based ranking algorithm and modify how node (aka sentence) centrality is computed in two ways: (a)~we employ BERT, a state-of-the-art neural representation learning model to better capture sentential meaning and (b)~we build graphs with directed edges arguing that the contribution of any two nodes to their respective centrality is influenced by their relative position in a document. Experimental results on three news summarization datasets representative of different languages and writing styles show that our approach outperforms strong baselines by a wide margin.

• The paper introduces PacSum, an unsupervised summarization model that enhances sentence centrality using BERT and directed graph structures.
• The methodology refines centrality computation by weighting sentence positions and directional relationships, overcoming limitations of traditional extractive baselines.
• Experimental results across English and Chinese datasets demonstrate PacSum’s competitive performance and potential for robust, multilingual applications.

Unsupervised Summarization Using Directed Sentence Centrality

This research paper, authored by Hao Zheng and Mirella Lapata, presents a novel approach to unsupervised single-document summarization, addressing the limitations inherent in current supervised methodologies. The authors argue that relying on large-scale, high-quality training datasets is often impractical across diverse languages, domains, and summarization styles. The paper revisits the concept of sentence centrality in graph-based summarization models, proposing enhancements to improve the effectiveness of centrality computation.

Methodology and Improvements

1. BERT for Sentence Representation: The authors leverage BERT, a state-of-the-art neural representation model, to enhance the semantic understanding of sentences. BERT's capability to create robust contextualized word embeddings allows for more precise sentence representations. This paradigm shift from traditional symbol-based representations such as TF-IDF or skip-thought vectors aims to improve the accuracy of sentence similarity computations.
2. Directed Graph Structure: A key innovation of this paper is the introduction of directed edges in the sentence similarity graph. Unlike traditional models that employ undirected edges, this methodology takes into account the relative positioning of sentences within a document, reflecting their potential influence on each other's centrality. This approach draws motivation from theoretical discourse structures such as Rhetorical Structure Theory, which recognizes the importance of textual units in conveying the central meaning.
3. Position Information in Centrality Computation: The authors propose a refined computation of centrality by assigning different weights to connections based on their direction (forward- or backward-looking). They posit that sentences occurring earlier in documents, particularly in news articles, are generally more pivotal, thus utilizing position information to modulate centrality scores accordingly.

Experimental Results and Implications

The authors conduct extensive experiments on three datasets—CNN/Daily Mail (English), NYT (English), and TTNews (Chinese)—to evaluate their approach's efficacy. The results demonstrate that the proposed model, named PacSum, outperforms traditional extractive baselines like TextRank and even comes close to the performance of supervised systems with significantly lesser data requirements.

• CNN/Daily Mail: Even on this dataset, notorious for the strength of the Lead-3 baseline, PacSum exhibits competitive performance.
• NYT Dataset: PacSum yields a marked improvement over baselines, signifying its proficiency in dealing with documents where important information is more uniformly distributed.
• TTNews: The model successfully transfers across languages, showcasing adaptability to different writing styles and summary lengths. This suggests robustness of the directed centrality and merits further investigation into multilingual applications.

Future Directions

The introduction of BERT as an encoding mechanism and positional considerations into unsupervised summarization frameworks sets an exciting precedent for future developments. The authors insinuate potential applications in refining supervised systems through the integration of their techniques—especially where data is scarce or costly to annotate. Looking forward, extending these findings to multi-document summarization and further optimizing the use of negative weights in graph structures are promising research avenues.

In conclusion, this paper presents substantial advancements in unsupervised summarization by revisiting and refining centrality measures. The investigation into directed graphs and sentence embeddings paves the way for more adaptable and resource-efficient summarization models that can operate across a variety of paradigms.