2000 character limit reached
Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models (2403.00794v2)
Published 23 Feb 2024 in cs.CL, cs.AI, and cs.LG
Abstract: Humor is a fundamental facet of human cognition and interaction. Yet, despite recent advances in natural language processing, humor detection remains a challenging task that is complicated by the scarcity of datasets that pair humorous texts with similar non-humorous counterparts. In our work, we investigate whether LLMs, can generate synthetic data for humor detection via editing texts. We benchmark LLMs on an existing human dataset and show that current LLMs display an impressive ability to 'unfun' jokes, as judged by humans and as measured on the downstream task of humor detection. We extend our approach to a code-mixed English-Hindi humor dataset, where we find that GPT-4's synthetic data is highly rated by bilingual annotators and provides challenging adversarial examples for humor classifiers.
- Mega: Multilingual evaluation of generative ai. ArXiv, abs/2303.12528.
- You told me that joke twice: A systematic investigation of transferability and robustness of humor detection models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13701–13715, Singapore. Association for Computational Linguistics.
- K.H. Basso. 1979. Portraits of ’the Whiteman’: Linguistic Play and Cultural Symbols among the Western Apache. Cambridge University Press.
- David Benatar. 1999. Prejudice in jest: When racial and gender humor harms. Public Affairs Quarterly, 13(2):191–203.
- Language models are few-shot learners.
- Unsupervised cross-lingual representation learning at scale.
- Qlora: Efficient finetuning of quantized llms.
- Ira Glass. 2008. Tough room.
- Is gpt-4 good enough to evaluate jokes?
- Do androids laugh at electric sheep? humor “understanding” benchmarks from the new yorker caption contest. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 688–714, Toronto, Canada. Association for Computational Linguistics.
- Context-driven satirical news generation. In Proceedings of the Second Workshop on Figurative Language Processing, pages 40–50, Online. Association for Computational Linguistics.
- “president vows to cut <taxes> hair”: Dataset and analysis of creative text editing for humorous headlines. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 133–142, Minneapolis, Minnesota. Association for Computational Linguistics.
- Stimulating creativity with funlines: A case study of humor generation in headlines.
- F. Hutcheson. 1750. Reflections Upon Laughter: And Remarks Upon the Fable of the Bees. Garland Publishing.
- Sophie Jentzsch and Kristian Kersting. 2023. Chatgpt is fun, but it is not funny! humor is still challenging large language models.
- Mistral 7b.
- Is chatgpt a good translator? yes with gpt-4 as the engine.
- Exploiting asymmetry for synthetic training data generation: SynthIE and the case of information extraction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1555–1574, Singapore. Association for Computational Linguistics.
- Humor detection in english-hindi code-mixed social media content : Corpus and baseline system.
- Synthetic data generation with large language models for text classification: Potential and limitations. ArXiv, abs/2310.07849.
- Roberta: A robustly optimized bert pretraining approach.
- Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization.
- Is a prompt and a few samples all you need? using gpt-4 for data augmentation in low-resource classification tasks. ArXiv, abs/2304.13861.
- John Morreall. 2023. Philosophy of Humor. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy, Summer 2023 edition. Metaphysics Research Lab, Stanford University.
- Ravindra Nayak and Raviraj Joshi. 2022. L3Cube-HingCorpus and HingBERT: A code mixed Hindi-English dataset and BERT language models. In Proceedings of the WILDRE-6 Workshop within the 13th Language Resources and Evaluation Conference, pages 7–12, Marseille, France. European Language Resources Association.
- Gpt-4 technical report.
- OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
- Laughing heads: Can transformers detect what makes a sentence funny?
- NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776–10787, Singapore. Association for Computational Linguistics.
- Generating faithful synthetic data with large language models: A case study in computational social science. ArXiv, abs/2305.15041.
- What makes things funny? an integrative review of the antecedents of laughter and amusement. Personality and Social Psychology Review, 25(1):41–65. PMID: 33342368.
- Robert West and Eric Horvitz. 2019. Reverse-engineering satire, or “paper on computational humor accepted despite making serious advances”. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence.