Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
133 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
46 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Challenges in Domain-Specific Abstractive Summarization and How to Overcome them (2307.00963v1)

Published 3 Jul 2023 in cs.CL and cs.AI

Abstract: LLMs work quite well with general-purpose data and many tasks in Natural Language Processing. However, they show several limitations when used for a task such as domain-specific abstractive text summarization. This paper identifies three of those limitations as research problems in the context of abstractive text summarization: 1) Quadratic complexity of transformer-based models with respect to the input text length; 2) Model Hallucination, which is a model's ability to generate factually incorrect text; and 3) Domain Shift, which happens when the distribution of the model's training and test corpus is not the same. Along with a discussion of the open research questions, this paper also provides an assessment of existing state-of-the-art techniques relevant to domain-specific text summarization to address the research gaps.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (39)
  1. Text summarization techniques: A brief survey.
  2. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.
  3. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
  4. CLIFF: Contrastive learning for improving faithfulness and factuality in abstractive summarization. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  5. Improving faithfulness in abstractive summarization with contrast candidate generation and selection. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online. Association for Computational Linguistics.
  6. Rethinking attention with performers. arXiv preprint arXiv:2009.14794.
  7. Neural path hunter: Reducing hallucination in dialogue systems via path grounding. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2197–2214, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  8. Ranking generated summaries by correctness: An interesting but challenging application for natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy. Association for Computational Linguistics.
  9. GO FIGURE: A meta evaluation of factuality in summarization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 478–487, Online. Association for Computational Linguistics.
  10. The reversible residual network: backpropagation without storing activations. Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 2211–2221.
  11. Assessing the factual accuracy of generated text. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery &; Data Mining, KDD ’19, page 166–175, New York, NY, USA. Association for Computing Machinery.
  12. Mind the facts: Knowledge-boosted coherent abstractive text summarization. NeurIPS, Knowledge Representation & Reasoning Meets Machine Learning (KR2ML workshop), abs/2006.15435.
  13. Abstractive summarization: An overview of the state of the art. Expert Systems with Applications, 121:49–65.
  14. Long short-term memory. Neural Computation, 9(8):1735–1780.
  15. The factual inconsistency problem in abstractive text summarization: A survey. CoRR, abs/2104.14839.
  16. UDALM: Unsupervised domain adaptation through language modeling. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2579–2590, Online. Association for Computational Linguistics.
  17. Reformer: The efficient transformer. CoRR, abs/2001.04451.
  18. Automatic text summarization: A state-of-the-art review. ICEIS (1), pages 648–655.
  19. Explainable automated fact-checking for public health claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Online. Association for Computational Linguistics.
  20. Evaluating the factual consistency of abstractive text summarization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9332–9346, Online. Association for Computational Linguistics.
  21. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. CoRR, abs/1808.06226.
  22. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. CoRR, abs/1910.13461.
  23. Lin, C.-Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  24. Roberta: A robustly optimized bert pretraining approach.
  25. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics.
  26. Summarization of biomedical articles using domain-specific word embeddings and graph ranking. Journal of Biomedical Informatics, 107:103452.
  27. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, page 311–318, USA. Association for Computational Linguistics.
  28. Efficient domain adaptation of language models via adaptive tokenization. Proceedings of the Second Workshop on Simple and Efficient Natural Language Processing, pages 155–165.
  29. A survey of evaluation metrics used for nlg systems. ACM Computing Surveys (CSUR), 55(2):1–39.
  30. Joint parsing and generation for abstractive summarization. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8894–8901.
  31. Linear-time wordpiece tokenization. CoRR, abs/2012.15524.
  32. Efficient transformers: A survey. CoRR, abs/2009.06732.
  33. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, pages 6000–6010.
  34. Big bird: Transformers for longer sequences. CoRR, abs/2007.14062.
  35. Automated fact-checking: A survey. Language and Linguistics Compass, 15(10):e12438.
  36. PEGASUS: pre-training with extracted gap-sentences for abstractive summarization. CoRR, abs/1912.08777.
  37. Bertscore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
  38. Mtl-das: Automatic text summarization for domain adaptation. Intell. Neuroscience, 2022.
  39. Detecting hallucinated content in conditional neural sequence generation. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now