Can ChatGPT's Responses Boost Traditional Natural Language Processing? (2307.04648v1)
Abstract: The employment of foundation models is steadily expanding, especially with the launch of ChatGPT and the release of other foundation models. These models have shown the potential of emerging capabilities to solve problems, without being particularly trained to solve. A previous work demonstrated these emerging capabilities in affective computing tasks; the performance quality was similar to traditional NLP techniques, but falling short of specialised trained models, like fine-tuning of the RoBERTa LLM. In this work, we extend this by exploring if ChatGPT has novel knowledge that would enhance existing specialised models when they are fused together. We achieve this by investigating the utility of verbose responses from ChatGPT about solving a downstream task, in addition to studying the utility of fusing that with existing NLP methods. The study is conducted on three affective computing problems, namely sentiment analysis, suicide tendency detection, and big-five personality assessment. The results conclude that ChatGPT has indeed novel knowledge that can improve existing NLP techniques by way of fusion, be it early or late fusion.
- R. Bommasani et al., “On the Opportunities and Risks of Foundation Models,” arXiv:2108.07258, 2021.
- C. Zhou et al., “A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT,” arXiv:2302.09419, 2023.
- J. Wei et al., “Emergent Abilities of Large Language Models,” arXiv:2206.07682, 2022.
- L. Ouyang et al., “Training language models to follow instructions with human feedback,” arXiv:2203.02155, 2022.
- OpenAI, “GPT-4 Technical Report,” arXiv:2303.08774, 2023.
- H. Touvron et al., “LLaMA: Open and Efficient Foundation Language Models,” arXiv:2302.13971, 2023.
- Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv:1907.11692, 2019.
- M. M. Amin, E. Cambria, and B. W. Schuller, “Will Affective Computing Emerge from Foundation Models and General AI? A First Evaluation on ChatGPT,” IEEE Intelligent Systems, vol. 38, no. 2, 2023, pp. 15–23.
- A. Hendy et al., “How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation,” arXiv:2302.09210, 2023.
- G. Wang et al., “Voyager: An Open-Ended Embodied Agent with Large Language Models,” arXiv:2305.16291, 2023.
- W. Zhang et al., “Sentiment Analysis in the Era of Large Language Models: A Reality Check,” arXiv:2305.15005, 2023.
- S. Bubeck et al., “Sparks of Artificial General Intelligence: Early experiments with GPT-4,” arXiv:2303.12712, 2023.
- E. Cambria, “Affective Computing and Sentiment Analysis,” IEEE Intelligent Systems, vol. 31, no. 2, 2016, pp. 102–107.
- T. Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality,” Advances in Neural Information Processing Systems, Curran Associates, Inc., Lake Tahoe, NV, USA, 2013.
- Y. Chen, V. Shah, and A. Ritter, “Better Low-Resource Entity Recognition Through Translation and Annotation Fusion,” arXiv:2305.13582, 2023.
- J. Li et al., “Prompt ChatGPT In MNER: Improved multimodal named entity recognition method based on auxiliary refining knowledge from ChatGPT,” arXiv:2305.12212, 2023.
- J. Kocoń et al., “ChatGPT: Jack of all trades, master of none,” Information Fusion, vol. 99, 2023, p. 101861.
- Q. Zhong et al., “Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT,” arXiv:2302.10198, 2023.
- C. Qin et al., “Is ChatGPT a General-Purpose Natural Language Processing Task Solver?” arXiv:2302.06476, 2023.
- V. Desu et al., “Suicide and Depression Detection in Social Media Forums,” Smart Intelligent Computing and Applications, Volume 2, Springer Nature Singapore, Singapore, Singapore, 2022, pp. 263–270.
- V. Ponce-López et al., “Chalearn lap 2016: First Round Challenge on First Impressions - Dataset and Results,” European conference on computer vision, Springer International Publishing, Cham, Switzerland, 2016, pp. 400–418.
- H. J. Escalante et al., “Design of an Explainable Machine Learning Challenge for Video Interviews,” International Joint Conference on Neural Networks (IJCNN), IEEE, Anchorage, AK, USA, 2017, pp. 3688–3695.
- H. Kaya, F. Gurpinar, and A. Ali Salah, “Multi-Modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening From Video CVs,” Conference on Computer Vision and Pattern Recognition Workshops, IEEE, Honolulu, HI, USA, 2017, pp. 1–9.
- D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” 3rd International Conference on Learning Representations, Conference Track Proceedings, ICLR, San Diego, CA, USA, 2015.
- M. Lindauer et al., “SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization,” Journal of Machine Learning Research, vol. 23, no. 54, 2022, pp. 1–9.
- B. Schuller et al., “The INTERSPEECH 2013 Computational Paralinguistics Challenge: Social Signals, Conflict, Emotion, Autism,” Proceedings INTERSPEECH, ISCA, Lyon, France, 2013, pp. 148–152.
- Mostafa M. Amin (6 papers)
- Erik Cambria (136 papers)
- Björn W. Schuller (153 papers)