A Deep Reinforced Model for Abstractive Summarization (1705.04304v3)

Abstract: Attentional, RNN-based encoder-decoder models for abstractive summarization have achieved good performance on short input and output sequences. For longer documents and summaries however these models often include repetitive and incoherent phrases. We introduce a neural network model with a novel intra-attention that attends over the input and continuously generated output separately, and a new training method that combines standard supervised word prediction and reinforcement learning (RL). Models trained only with supervised learning often exhibit "exposure bias" - they assume ground truth is provided at each step during training. However, when standard word prediction is combined with the global sequence prediction training of RL the resulting summaries become more readable. We evaluate this model on the CNN/Daily Mail and New York Times datasets. Our model obtains a 41.16 ROUGE-1 score on the CNN/Daily Mail dataset, an improvement over previous state-of-the-art models. Human evaluation also shows that our model produces higher quality summaries.

A Deep Reinforced Model for Abstractive Summarization

In this paper, Paulus, Xiong, and Socher from Salesforce Research introduce a novel neural network model designed for abstractive summarization of longer documents. Their main contributions lie in the enhancement of traditional attentional, RNN-based encoder-decoder models with a unique intra-attention mechanism and a hybrid training approach combining supervised learning and reinforcement learning (RL). This hybrid method addresses significant limitations such as exposure bias and repetitive generation in long-sequence summarization tasks.

Background

Text summarization algorithms can be broadly divided into extractive and abstractive methods. While extractive summarization systems generate summaries by copying parts of the input text, abstractive summarization aims to create new phrases that might not exist in the original content. The paper focuses on addressing challenges in abstractive summarization, particularly those associated with longer input and output sequences.

Previous works in the field, such as those by Nallapati et al., have illustrated the limitations of RNN-based encoder-decoder models employed in longer documents. These models often suffer from exposure bias and repetition, generating unnatural summaries with repeated phrases.

Model Architecture

The authors propose a novel model with two significant innovations:

1. Intra-Attention Mechanism: The model integrates intra-temporal attention in the encoder and sequential intra-attention in the decoder. This dual attention approach ensures that the model attends over different parts of the input and the generated output, thereby minimizing the repetition of phrases. Intra-temporal attention helps focus on distinct parts of the input across decoding steps, while intra-decoder attention introduces more information about previously generated tokens, allowing the model to make structured predictions.
2. Hybrid Training Objective: To tackle the exposure bias and improve the summary quality, the authors incorporate a hybrid training method. This combines maximum-likelihood supervised loss with RL-based global sequence prediction. The self-critical policy gradient algorithm used in RL helps optimize the ROUGE metric, directly addressing the sequence-level evaluation discrepancy.

Experimental Setup and Results

The model is evaluated on two extensive datasets: CNN/Daily Mail and the New York Times. The CNN/Daily Mail dataset comprises 287,113 training examples and the New York Times dataset consists of 589,284 training examples. The proposed model substantially improves upon previous state-of-the-art results, achieving a ROUGE-1 score of 41.16 on the CNN/Daily Mail dataset. It also proves considerably effective on the New York Times dataset.

Quantitative results demonstrate the effectiveness of intra-attention, particularly for longer documents, as the improvement in ROUGE scores is more pronounced with longer summaries. Additionally, the hybrid RL and supervised learning objective yield higher scores than models trained with traditional maximum-likelihood objectives alone.

Further analysis through human evaluation confirms the increase in readability and relevance of the summaries generated by the hybrid model, highlighting the benefits of combining RL with supervised learning to achieve more coherent and human-like summaries.

Implications and Future Directions

The results and techniques proposed in this paper have significant implications for the field of natural language processing and text summarization. The intra-attention mechanisms introduced can be extended to other sequence-to-sequence tasks with long inputs and outputs, enhancing model performance and summary quality. The hybrid training methodology not only addresses specific issues in summarization but also opens the path for more sophisticated training paradigms that better align with discrete evaluation metrics.

Future work might explore further enhancements in attention mechanisms and different combinations of supervised learning with reinforcement strategies. Additionally, as text summarization applications expand, exploring more complex datasets and varied domains could validate and potentially extend the adaptability of the proposed methods.

Conclusion

Paulus, Xiong, and Socher have presented a robust model and training framework that addresses longstanding challenges in abstractive summarization. By introducing intra-attention mechanisms and a novel hybrid training approach, the model achieves improved performance and output quality on challenging, long-sequence summarization tasks. These contributions present a meaningful progression in developing more effective text summarizers, significantly enriching subsequent practical implementations and theoretical advancements in natural language processing.