One Joke to Rule them All? On the (Im)possibility of Generalizing Humor (2508.19402v1)

Abstract: Humor is a broad and complex form of communication that remains challenging for machines. Despite its broadness, most existing research on computational humor traditionally focused on modeling a specific type of humor. In this work, we wish to understand whether competence on one or more specific humor tasks confers any ability to transfer to novel, unseen types; in other words, is this fragmentation inevitable? This question is especially timely as new humor types continuously emerge in online and social media contexts (e.g., memes, anti-humor, AI fails). If LLMs are to keep up with this evolving landscape, they must be able to generalize across humor types by capturing deeper, transferable mechanisms. To investigate this, we conduct a series of transfer learning experiments across four datasets, representing different humor tasks. We train LLMs under varied diversity settings (1-3 datasets in training, testing on a novel task). Experiments reveal that models are capable of some transfer, and can reach up to 75% accuracy on unseen datasets; training on diverse sources improves transferability (1.88-4.05%) with minimal-to-no drop in in-domain performance. Further analysis suggests relations between humor types, with Dad Jokes surprisingly emerging as the best enabler of transfer (but is difficult to transfer to). We release data and code.

• The paper demonstrates that LLMs achieve up to 75% accuracy on unseen datasets by training on a diverse set of humor types.
• LLMs show strong bidirectional transfer with humor forms like Dad Jokes and One Liners due to their structural similarities.
• Moderate diversity in training (using two datasets) nearly maximizes generalization gains without significant loss in in-domain performance.

On the (Im)possibility of Generalizing Humor

Introduction

The paper "One Joke to Rule them All? On the (Im)possibility of Generalizing Humor" investigates whether LLMs can generalize humor across different types by leveraging transfer learning techniques. The key focus is on understanding if competence gained in specific humor tasks can confer broader abilities to handle novel, unseen forms of humor. This study is inspired by the continuously evolving humor landscape in digital media, where LLMs must adapt to diverse humor styles to maintain relevance.

Experimental Setup

The researchers conducted transfer learning experiments using four distinct humor datasets, each representing a unique humor type: Amazon Questions, Reddit Dad Jokes, Sarcasm Headlines, and One Liners. The experiments involved training LLMs using different combinations of these datasets to assess cross-type transferability.

The experiments were structured as follows:

• Single Dataset Training: Models were trained on one dataset and tested across all other datasets to assess basic transfer capabilities.
• Double Dataset Training: Models were trained on two datasets concurrently and tested on the remaining datasets to study multi-task learning effects.
• Triple Dataset Training: Models utilized three datasets for training, examining how increased dataset diversity affects generalization.

  Figure 1: Overview of the experimental setups, detailing different dataset combinations for training and testing.

Results and Analysis

The experiments revealed several insights into humor transferability:

1. Humor Transfer Capabilities (RQ1): LLMs can transfer humor knowledge across datasets, with models achieving up to 75% accuracy on unseen datasets. Training with diverse sources improved transferability by 1.88-4.05% with minimal drop in in-domain performance.
2. Linking Humor Types (RQ2): Certain humor types, such as Dad Jokes, emerged as effective enablers of transfer, though challenging to transfer to. Conversely, One Liners and Sarcasm Headlines showed strong bidirectional transfer, likely due to their structural and stylistic similarities.
3. Impact of Data Diversity (RQ3): Greater training diversity generally led to improved transfer, especially for simpler humor types. The most substantial gains occurred when moving from single to double dataset training, indicating that moderate diversity is nearly as effective as high diversity.

   Figure 2: Increasing the training data diversity improves transfer. Mistral results show consistent improvement across experiments, with gains diminishing as data diversity increases.

Implementation and Implications

The practical implementation of humor transfer learning involves careful selection of humor datasets to maximize generalization while maintaining in-domain performance. The results indicate that humor types with broader content and diverse structures, like Amazon Questions and Dad Jokes, are most effective for enabling transfer.

For developers aiming to implement humor understanding in LLMs, focusing on training with a diverse set of humor sources is crucial. The findings suggest that a balance between dataset diversity and quantity will optimize transferability and preserve in-domain accuracy.

Conclusion

The paper highlights the nuanced nature of humor transferability in LLMs, underscoring that while transfer learning can facilitate some generalization across humor types, successful generalization is asymmetric and dependent on humor complexity. Future research should extend these findings to include multimodal humor and cross-cultural settings, aiming to refine the understanding of humor's transfer mechanisms in both AI and cognitive contexts.

This study contributes to ongoing efforts to enhance LLMs' adaptive capabilities in intricate communicative domains like humor, paving the way for improved AI performance in natural language processing tasks.