Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
117 tokens/sec
GPT-4o
8 tokens/sec
Gemini 2.5 Pro Pro
47 tokens/sec
o3 Pro
5 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

The Limits of ChatGPT in Extracting Aspect-Category-Opinion-Sentiment Quadruples: A Comparative Analysis (2310.06502v1)

Published 10 Oct 2023 in cs.CL

Abstract: Recently, ChatGPT has attracted great attention from both industry and academia due to its surprising abilities in natural language understanding and generation. We are particularly curious about whether it can achieve promising performance on one of the most complex tasks in aspect-based sentiment analysis, i.e., extracting aspect-category-opinion-sentiment quadruples from texts. To this end, in this paper we develop a specialized prompt template that enables ChatGPT to effectively tackle this complex quadruple extraction task. Further, we propose a selection method on few-shot examples to fully exploit the in-context learning ability of ChatGPT and uplift its effectiveness on this complex task. Finally, we provide a comparative evaluation on ChatGPT against existing state-of-the-art quadruple extraction models based on four public datasets and highlight some important findings regarding the capability boundaries of ChatGPT in the quadruple extraction.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (41)
  1. Aspect-based sentiment analysis with opinion tree generation. In Proceedings of the 31st International Joint Conference on Artificial Intelligence, pages 4044–4050, Vienna, Austria.
  2. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, pages 1877–1901. Curran Associates, Inc., Vancouver, Canada.
  3. Aspect-category based sentiment analysis with hierarchical graph convolutional network. In Proceedings of the 28th International Conference on Computational Linguistics, pages 833–843, Barcelona, Spain.
  4. Aspect-category-opinion-sentiment quadruple extraction with implicit aspects and opinions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 340–350, Online.
  5. Open-domain aspect-opinion co-mining with double-layer pan extraction. In Proceedings of the 28th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 66–75, Washington, DC.
  6. Synchronous double-channel recurrent network for aspect-opinion pair extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6515–6524, Online.
  7. A span-level bidirectional network for aspect sentiment triplet extraction. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 4300–4309, Abu Dhabi.
  8. Zhuang Chen and Tieyun Qian. 2020. Relation-aware collaborative learning for unified aspect-based sentiment analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3685–3694, Online.
  9. A multi-task incremental learning framework with category name embedding for aspect-category sentiment analysis. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 6955–6965, Online.
  10. BERT: Pre-training of deep bidirectional transformers for language understanding. The 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186.
  11. Question-driven span labeling model for aspect-opinion pair extraction. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 12875–12883, Online.
  12. LEGO-ABSA: A prompt-based task assemblable unified generative framework for multi-task aspect-based sentiment analysis. In Proceedings of the 29th International Conference on Computational Linguistics, pages 7002–7012, Gyeongju, Korea.
  13. Unleashing the power of ChatGPT for translation: An empirical study. arXiv preprint arXiv:2304.02182.
  14. An interactive multi-task learning network for end-to-end aspect-based sentiment analysis. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 504–515, Florence, Italy.
  15. CAN: Constrained attention networks for multi-aspect sentiment analysis. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 4601–4610, Hong Kong, China.
  16. A unified model for opinion target extraction and target sentiment prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 6714–6721, Honolulu, HI.
  17. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5876–5883, New Orleans, LA.
  18. Seq2path: Generating sentiment tuples as paths of a tree. In Findings of the Association for Computational Linguistics: ACL 2022, pages 2215–2225, Dublin, Ireland.
  19. A joint training dual-MRC framework for aspect based sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 13543–13551, Online.
  20. Rethinking the role of demonstrations: What makes in-context learning work? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, Abu Dhabi.
  21. Training language models to follow instructions with human feedback. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems. Curran Associates, Inc., New Orleans, LA.
  22. Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 8600–8607, New York City, NY.
  23. SemEval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th International Workshop on Semantic Evaluation, pages 19–30, San Diego, CA.
  24. SemEval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation, pages 486–495, Denver, CO.
  25. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1–67.
  26. Aspect-based extraction of implicit opinions using opinion co-occurrence algorithm. In Proceedings of the 5th International Seminar on Research of Information Technology and Intelligent Systems, pages 781–786, Yogyakarta, Indonesia.
  27. Instruction tuning for few-shot aspect-based sentiment analysis. arXiv preprint arXiv:2210.06629.
  28. Target-apect-sentiment joint detection for aspect-based sentiment analysis. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 9122–9129, New York City, NY.
  29. UnifiedABSA: A unified ABSA framework based on multi-task instruction tuning. arXiv preprint arXiv:2211.10986.
  30. Is ChatGPT a good sentiment analyzer? a preliminary study. arXiv preprint arXiv:2304.04339.
  31. Zero-shot information extraction via chatting with ChatGPT. arXiv preprint arXiv:2302.10205.
  32. Grid tagging scheme for aspect-oriented fine-grained opinion extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2576–2585, Online.
  33. Learning span-level interactions for aspect sentiment triplet extraction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 4755–4766, Online.
  34. Position-aware tagging for aspect sentiment triplet extraction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, pages 2339–2349, Online.
  35. A unified generative framework for aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 2416–2429, Online.
  36. Zero-shot temporal relation extraction with ChatGPT. arXiv preprint arXiv:2304.05454.
  37. Aspect sentiment quad prediction as paraphrase generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9209–9219, Punta Cana, Dominican.
  38. Towards generative aspect-based sentiment analysis. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 504–510, Online.
  39. SpanMlt: A span-based multi-task learning framework for pair-wise aspect and opinion terms extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3239–3248, Online.
  40. Can ChatGPT understand too? a comparative study on ChatGPT and fine-tuned BERT. arXiv preprint arXiv:2302.10198.
  41. Large language models are human-level prompt engineers. In Proceedings of the 11th International Conference on Learning Representations, Kigali, Rwanda.
Citations (4)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now