---
title: 'Oogiri-Corpus: A Humor Dataset'
url: https://www.emergentmind.com/topics/oogiri-corpus
type: topic
---

# Oogiri-Corpus: A Humor Dataset

Searching arXiv for the cited Oogiri papers to ground the article in current preprints.
I’m going to look up the relevant arXiv entries for Oogiri-Corpus and adjacent Oogiri humor benchmarks.
Oogiri-Corpus is a dataset resource for studying humor through **Oogiri**, a Japanese creative-response game in which a participant is given a **topic/prompt** and must produce a humorous response or punchline. In the 2025 literature, the term is used in two closely related senses. In "Assessing the Capabilities of LLMs in Humor: A Multi-dimensional Analysis of Oogiri Generation and Evaluation" it denotes an annotated corpus built from existing Oogiri data, newly collected Oogiri data, and **LLM-generated responses**, with six-dimensional ratings for humor generation and evaluation [2511.09133]. In "Oogiri-Master: Benchmarking Humor Understanding via Oogiri" it denotes the underlying dataset used for linguistic analysis and for constructing a humor-understanding benchmark from **Oogiri Sogo** data [2512.21494]. This suggests a terminological ambiguity: **Oogiri-Corpus** is best understood as a family of Oogiri-centered research resources rather than a single immutable artifact.

## 1. Oogiri as a computational humor substrate

Oogiri is described as a Japanese improvisational comedy format, and in its modern form as **Tonchi** (頓智), where participants are given an **image**, **text**, or **both** and must produce an **unexpected, witty, and humorous response**. This makes it a prompt-conditioned humor task rather than a generic joke-generation problem. The format is especially valuable for computational humor because a good response must typically be **on-topic**, **clear**, **novel**, show **intelligence**, feel **relatable / empathetic**, and be **funny overall** [2511.09133][2312.02439].

The Oogiri setting is used to argue for two complementary views of humor. One is a **multidimensional** view, in which humor is not reducible to a binary “funny/not funny” judgment. The other is a creative-reasoning view, in which Oogiri requires **remote association**, **conceptual leaps**, and **thinking outside the box**, rather than only step-by-step logical chaining. The Oogiri-GO work frames this as **Leap-of-Thought (LoT)**, a non-sequential associative reasoning mode distinct from **Chain-of-Thought (CoT)** [2312.02439]. The six-dimensional Oogiri-Corpus work frames it in terms of **incongruity-resolution**, arguing that LLMs can often produce the “incongruity” part via novelty but struggle with the human-like grounding needed for humorous resolution, especially **Empathy** [2511.09133].

A common misconception is that Oogiri datasets are merely humor corpora in the narrow sense of isolated joke texts. The literature instead treats them as structured resources for examining how humor depends on the relation between prompt and response, on shared social context, and on human judgments collected under specific voting or annotation protocols [2511.09133][2512.21494].

## 2. Corpus lineages and dataset construction

The main Oogiri-Corpus lineages are closely tied to earlier Oogiri dataset construction efforts. The precursor dataset is **Oogiri-GO**, introduced as a **multimodal and multilingual Oogiri-GO dataset** containing **more than 130,000 high-quality samples** after filtering, with inputs in **English, Chinese, and Japanese** and task types **I2T**, **T2T**, and **IT2T** [2312.02439]. That resource was built from **Bokete**, **Twitter**, and **Weibo**, beginning with **over 200,000 raw samples**, then applying **Qwen-VL** screening, **NudeNet**-based NSFW filtering, and two rounds of iterative manual screening [2312.02439].

The six-dimensional Oogiri-Corpus extends this lineage by combining **existing Oogiri data**, **newly collected Oogiri data**, and **LLM-generated responses**. Its two named human sources are **Oogiri-GO**, taken from **Bokete**, and **Oogiri-Chaya**, collected from another online Oogiri platform. The filtered source sizes are **1,329 topics** for Oogiri-GO, consisting of **313 text-based** and **1,016 image-based** topics, and **551 topics** for Oogiri-Chaya, consisting of **425 text-based** and **126 image-based** topics. For Oogiri-Chaya, the authors collected **all samples between 21 Nov 2021 and 17 Jul 2024**, used **GPT-4o** to detect potentially discriminatory, violent, sexual, or otherwise socially inappropriate content, manually reviewed flagged items, and removed inappropriate ones [2511.09133].

The Oogiri-Master usage of Oogiri-Corpus follows a different construction path. It was built by web crawling from the public Japanese Oogiri competition platform **Oogiri Sogo**. The authors collected **2,165 prompts**, then **excluded prompts with fewer than 100 total votes**, leaving **908 prompts** and **82,536 total prompt–response pairs**, with **about 95.9 candidate responses per prompt on average** and **about 171.6 votes per prompt on average** [2512.21494].

| Resource | Data basis | Key properties |
|---|---|---|
| **Oogiri-GO** [2312.02439] | Bokete, Twitter, Weibo | More than 130,000 high-quality samples; I2T, T2T, IT2T; English, Chinese, Japanese |
| **Oogiri-Corpus** [2511.09133] | Oogiri-GO, Oogiri-Chaya, LLM-generated responses | Oogiri-GO: 1,329 topics; Oogiri-Chaya: 551 topics; benchmark selects 200 topics and 8 response types |
| **Oogiri-Corpus** [2512.21494] | Oogiri Sogo | 908 prompts; 82,536 prompt–response pairs; about 95.9 responses and about 171.6 votes per prompt |

A central issue across these resources is **popularity bias**. The six-dimensional paper emphasizes that **Bokete / Oogiri-GO** allows users to see other responses and vote counts, which can create **first-mover advantage** and **conformity / bandwagon bias**, whereas **Oogiri-Chaya** hides responses during submission and hides vote counts during evaluation, making voting closer to “pure funniness” [2511.09133]. The Oogiri-Master paper makes a parallel argument for **Oogiri Sogo**, where vote counts are not displayed during voting and voters cannot see others’ ratings / vote counts while voting [2512.21494].

## 3. Annotation regimes and humor dimensions

The most explicit annotation protocol appears in the six-dimensional Oogiri-Corpus. It uses **six 5-point absolute rating dimensions**, described as **0–4 integer ratings** where **0 = not at all / worst** and **4 = excellent / best**. The dimensions are **Novelty**, **Clarity**, **Relevance**, **Intelligence**, **Empathy**, and **Overall Funniness** [2511.09133]. Formally, the rubric is

$$
\text{Score} \in \{0,1,2,3,4\}
$$

for each of

$$
\{\text{Novelty}, \text{Clarity}, \text{Relevance}, \text{Intelligence}, \text{Empathy}, \text{Overall Funniness}\}.
$$

The human annotation setup recruits annotators via **Lancers**. Each response is rated by **four native Japanese speakers**, and the authors create **eight distinct subsets** so each annotator evaluates only **one response type per topic**. The stated purpose is to keep the rating fair and avoid direct comparisons within the same topic set. The same models used for generation—**GPT-4.1**, **Gemini 2.5 Pro**, and **Claude Sonnet 4**—are also used as judges, with the same rubric and instructions as human annotators [2511.09133].

The Oogiri-Master usage of Oogiri-Corpus does not rely on a separate annotation campaign for the main corpus. Instead, it uses the platform’s **naturally collected vote counts** as the funniness signal. Each response has a **vote count** indicating perceived funniness, and those votes are collected **independently** in the platform’s voting phase. For analysis, responses are stratified by vote rank within each prompt: **High-rated responses** are the **top 3 responses** by vote count, and **Low-rated responses** are the **bottom 3 responses**. This yields **5,448 responses total**, i.e. **908 prompts × 6 responses** [2512.21494].

The Oogiri-Master paper explicitly notes that it does not report a standard inter-rater reliability statistic such as Cohen’s kappa or Krippendorff’s alpha for the corpus votes, because the votes are collected through the platform rather than a controlled multi-rater annotation study. Its main quality-control measures are filtering out prompts with fewer than 100 votes, hiding vote visibility, and using top-vs-bottom response selection for analysis. For the separate human baseline in Oogiri-Master, each item is answered by **21 crowdworkers**, with **attention checks with unambiguous answers**, and final labels are determined by majority vote among workers who passed the checks [2512.21494].

## 4. Benchmark tasks and experimental formulations

The six-dimensional Oogiri-Corpus supports two benchmark tasks. **Task 1: Oogiri generation** asks whether an LLM can generate humorous Oogiri responses that humans judge as funny. The model is given a topic and asked to generate a humorous response in Japanese, and the generated outputs are then rated by humans on the six dimensions. **Task 2: Six-dimensional Oogiri evaluation** asks whether an LLM can judge humorous responses in a way that aligns with human evaluation. Here, each model is given a **topic-response pair** and asked to score it on the same six dimensions; the output is compared against human ratings using **Spearman correlation** [2511.09133].

For the benchmark evaluation in that work, the authors select **200 topics** total, split into **100 text-based** and **100 image-based** topics. For each topic they prepare **8 response types**: **High-voted human response**, **Mid-voted human response**, **Low-voted human response**, **Unrelated response**, **GPT-4.1 serious**, **GPT-4.1 humorous**, **Gemini 2.5 Pro humorous**, and **Claude Sonnet 4 humorous**. The models evaluated are **GPT-4.1** (`gpt-4.1-2025-04-14`), **Gemini 2.5 Pro** (`gemini-2.5-pro-preview-05-06`), and **Claude Sonnet 4** (`claude-sonnet-4-20250514-v1:0`). Generation and evaluation are both prompted in Japanese using a base prompt template, use **default settings**, and report **no additional training** [2511.09133].

Oogiri-Master transforms Oogiri-Corpus into a benchmark for humor understanding rather than generation. It contains **600 items** total, with **400 items** for relative judgment tasks and **200 items** for binary classification. The five tasks are **Binary_same**, **Binary_diff**, **Triple**, **Quad**, and an absolute binary classification task that decides whether a response to a prompt is **funny** or **not funny**. The paper emphasizes **Binary_diff**, which compares a high-rated response for the prompt with a high-rated response from a different prompt, because it tests whether the model actually understands prompt-conditioned humor rather than generic funny style [2512.21494].

The Oogiri-GO literature introduces a different task family oriented to creative reasoning. It includes **choice questions** such as **2T1**, **3T1**, **4T1**, and **5T2**, evaluated by **Accuracy**, as well as ranking questions evaluated by **Top-1 accuracy** and **NDCG**. In that setting, Oogiri is used to test **associative generation**, **associative discrimination**, and **creative humor generation**, and to support the **CLoT** framework for improving **LoT** ability [2312.02439].

## 5. Empirical findings on generation and judgment

The most prominent finding from the six-dimensional Oogiri-Corpus is that LLMs generate Oogiri responses at a level **between low-tier and mid-tier humans**. The best model is **Gemini 2.5 Pro**. In Table 1, mean **Overall Funniness** is **2.563** for **Human High**, **1.913** for **Human Mid**, **1.232** for **Human Low**, **1.621** for **GPT-4.1**, **1.803** for **Gemini 2.5 Pro**, and **1.504** for **Claude Sonnet 4** [2511.09133].

| Response source | Mean Overall Funniness |
|---|---:|
| Human High | 2.563 |
| Human Mid | 1.913 |
| Human Low | 1.232 |
| GPT-4.1 | 1.621 |
| Gemini 2.5 Pro | 1.803 |
| Claude Sonnet 4 | 1.504 |

Dimension-level analysis shows that LLMs are relatively decent at **Clarity**, **Relevance**, and partly **Novelty**, but notably weaker on **Empathy**. The paper identifies **Empathy** as the biggest gap between humans and LLMs, and further argues that this deficit helps explain why LLMs fail to replicate human humor assessment [2511.09133].

Agreement between model judgments and human judgments is **modest to low** on **Overall Funniness**. The reported **Spearman** coefficients are **0.224** for **GPT-4.1**, **0.169** for **Gemini 2.5 Pro**, and **0.266** for **Claude Sonnet 4**. A key qualitative result is a difference in evaluation logic: **Humans prioritize Empathy**, whereas **LLMs prioritize Novelty**. The paper also notes that LLMs show stronger agreement on more “objective” dimensions like **Clarity** and **Relevance**, but that this does not carry over to the more subjective **Overall Funniness** dimension [2511.09133].

The paper’s qualitative error analysis names several biases. **Positivity bias** refers to the tendency to rate many responses too highly, including unrelated responses, weaker human responses, and sometimes even serious responses. **Self-preference bias** refers to LLMs often rating **their own generated responses** highly. **Context negligence** refers to the tendency to score unrelated responses surprisingly positively when humans sharply penalize them. The broader explanation is a **Novelty-over-Empathy mismatch**, in which models overvalue unexpectedness, originality, and surface-level cleverness, while humans care more about empathy, situatedness, and emotional resonance [2511.09133].

A concrete example illustrates this contrast. For the topic **“Tear a coupon to pieces and say one line.”**, the human high-rated response **“A funeral director shouldn't say things like, ‘See you next time!’”** received a human **Empathy** score of **3.00** and a human **Funniness** score of **2.00**, whereas the GPT-4.1 response **“So, this makes it zero calories now, right?”** received **1.25** for **Empathy** and **0.75** for **Funniness** [2511.09133].

## 6. Linguistic analysis, benchmark performance, and research significance

The Oogiri-Master paper uses Oogiri-Corpus as a basis for quantitative analysis of what linguistic features correlate with funniness. It compares high-rated and low-rated responses using an **independent two-sample Student’s t-test** and **Cohen’s d**, with the interpretation that \(d = 0.2, 0.5, 0.8\) correspond to small, medium, and large effects. The feature families are **basic linguistic features**, **semantic distance and textual entailment**, **surprisal and nPMI**, and **LLM-scored higher-order features** [2512.21494].

The higher-order features are rated on a 1–5 scale for **ambiguity exploitation**, **associative distance**, **benign violation**, **coherence**, **expectedness**, **incongruity resolution**, **metaphor use**, and **perspective shift**. The main findings are that funny responses are significantly shorter, have fewer unique characters and lower lexical novelty, and are especially associated with higher-order traits such as **ambiguity exploitation**, **associative distance**, **benign violation**, **incongruity resolution**, **metaphor use**, and **perspective shift**. Among these, **perspective shift** and **ambiguity** show the largest effects, while **semantic distance** and **surprisal** are weaker signals [2512.21494].

The reported table-level results include: **length: high 14.12 vs low 16.40, \(d=-0.28\)**; **lexical novelty: high 0.80 vs low 0.93, \(d=-0.21\)**; **ambiguity exploitation: high 2.10 vs low 1.61, \(d=0.42\)**; **associative distance: high 4.38 vs low 3.90, \(d=0.33\)**; **incongruity resolution: high 3.71 vs low 3.35, \(d=0.36\)**; and **perspective shift: high 2.40 vs low 1.87, \(d=0.50\)** [2512.21494]. These results provide an empirical complement to the six-dimensional finding that humor judgments depend on more than novelty alone.

In benchmark performance, **Claude-Opus-4** and the **human baseline** both achieve the best average accuracy, **68.7%**, while **GPT-5** reaches **67.6%**, **Gemini-2.5-Pro** reaches **53.4%**, and **LLM-jp-3.1-13b\(_{ja}\)** reaches **49.8%**. The paper’s **insight-augmented prompting** incorporates features such as length, unique character count, prompt–response length ratio, symbol ratio, katakana ratio, ambiguity exploitation, associative distance, benign violation, incongruity resolution, metaphor use, and perspective shift. For **GPT-5**, performance rises from **67.6%** to **70.7%**, an improvement of **+3.1 points** [2512.21494].

Several broader implications follow. First, Oogiri-Corpus supports the study of humor as a **multidimensional cognitive and social phenomenon** rather than a single scalar label [2511.09133]. Second, it connects humor evaluation to creative-reasoning research, including **LoT** and **CLoT**, where Oogiri serves as a benchmark for associative generation and discrimination [2312.02439]. Third, it highlights persistent limitations: some humor is **culture-specific**, the Oogiri-Master benchmark is limited to **understanding/judging humor** rather than generation or explanation, the current work is **text-to-text only** in that benchmark, and human evaluation could be improved by accounting for annotator attributes [2512.21494]. A further misconception is that higher novelty alone should predict humor; the combined evidence instead points to a more structured interaction among prompt relevance, incongruity, perspective shift, and especially humanly grounded **Empathy** [2511.09133][2512.21494].

Source: https://www.emergentmind.com/topics/oogiri-corpus