A Sentiment Consolidation Framework for Meta-Review Generation (2402.18005v2)
Abstract: Modern natural language generation systems with LLMs exhibit the capability to generate a plausible summary of multiple documents; however, it is uncertain if they truly possess the capability of information consolidation to generate summaries, especially on documents with opinionated information. We focus on meta-review generation, a form of sentiment summarisation for the scientific domain. To make scientific sentiment summarization more grounded, we hypothesize that human meta-reviewers follow a three-layer framework of sentiment consolidation to write meta-reviews. Based on the framework, we propose novel prompting methods for LLMs to generate meta-reviews and evaluation metrics to assess the quality of generated meta-reviews. Our framework is validated empirically as we find that prompting LLMs based on the framework -- compared with prompting them with simple instructions -- generates better meta-reviews.
- Unsupervised opinion summarization with content planning. In AAAI, pages 12489–12497.
- Metagen: An academic meta-review generation system. In SIGIR, pages 1653–1656.
- Automatic text summarization: A comprehensive survey. Expert Systems with Applications, 165:113679.
- Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. JAIR, 77:103–166.
- Exploring sentiments in summarization: Sentitextrank, an emotional variant of textrank. In Proceedings of the 9th Italian Conference on Computational Linguistics, volume 3596.
- Comprehensive review of opinion summarization.
- Summarizing multiple documents with conversational structure for meta-review generation. In Findings of EMNLP.
- Compressed heterogeneous graph for abstractive multi-document summarization. In AAAI.
- Chin-Yew Lin and Eduard H. Hovy. 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In HLT-NAACL, pages 71–78.
- G-eval: NLG evaluation using GPT-4 with better human alignment. CoRR, abs/2303.16634.
- Summarization is (almost) dead. CoRR, abs/2309.09558.
- Incorporating peer reviews and rebuttal counter-arguments for meta-review generation. In CIKM, pages 2189–2198.
- Bertscore: Evaluating text generation with BERT. In ICLR.
- A survey of large language models. CoRR, abs/2303.18223.
- Towards a unified multi-dimensional evaluator for text generation. In EMNLP, pages 2023–2038.
- Miao Li (156 papers)
- Jey Han Lau (67 papers)
- Eduard Hovy (115 papers)