CreativityPrism: Multidimensional LLM Creativity Framework

Updated 4 July 2026

CreativityPrism is a benchmark framework measuring LLM creativity as a multidimensional construct with distinct tasks and standardized metrics.
It evaluates three domains—divergent thinking, creative writing, and logical reasoning—using 9 tasks and 20 metrics organized by quality, novelty, and diversity.
Empirical findings show notable performance gaps between proprietary and open-source models, highlighting that excellence in one creative dimension does not ensure overall transferability.

Searching arXiv for the core paper and directly related "Prism"/creativity references to support the article. CreativityPrism is a benchmark and evaluation-analysis framework for LLM creativity that treats creativity as a multidimensional construct rather than a single scalar capability. It spans 9 tasks, 3 domains—divergent thinking, creative writing, and logical reasoning—and 20 evaluation metrics, organized into the three dimensions of quality, novelty, and diversity. Its central hypothesis is that “creativity is not one fixed idea,” so strong performance on one task or dimension should not be assumed to transfer across others; accordingly, the framework evaluates 17 state-of-the-art proprietary and open-sourced LLMs and emphasizes cross-task, cross-domain, and cross-dimension analysis rather than a single leaderboard number (Hou et al., 23 Oct 2025).

1. Definition and conceptual basis

CreativityPrism was proposed in response to two linked problems in LLM creativity evaluation. First, creativity is hard to evaluate automatically at scale, and prior work often depends on expensive human judgment. Second, creativity is defined inconsistently across tasks and domains, so apparently similar claims about “creative” performance may rest on incompatible metrics. The framework therefore organizes task-specific evaluation under a common taxonomy of quality, novelty, and diversity, while preserving the fact that these dimensions are instantiated differently across tasks (Hou et al., 23 Oct 2025).

In this taxonomy, quality measures whether a generation satisfies a task’s core functional requirements; novelty measures how rare or original a response is relative to existing or conventional content; and diversity measures variation among generated outputs. The framework explicitly treats quality as the convergent side of creativity and novelty-plus-diversity as divergent aspects. This decomposition is intended to avoid the failure mode in which one metric—often lexical variability or corpus rarity—is overinterpreted as a general creativity proxy (Hou et al., 23 Oct 2025).

The “prism” metaphor is methodological rather than decorative. Just as a prism decomposes white light into multiple colors, CreativityPrism decomposes model creativity into distinct evaluative aspects across contexts. A plausible implication is that the framework should be read less as a universal definition of creativity than as an operational scheme for comparing heterogeneous creativity tasks without collapsing them into one undifferentiated criterion.

2. Benchmark structure and task inventory

CreativityPrism distributes its tasks across three domains. The divergent-thinking domain includes Alternative Uses Test (AUT), Divergent Association Task (DAT), and Torrance Tests of Creative Thinking (TTCT). The creative-writing domain includes Torrance Test of Creative Writing (TTCW), Creative Short Story, Creativity Index, and CS4. The logical-reasoning domain includes NeoCoder and Creative Math (Hou et al., 23 Oct 2025).

The framework is intentionally heterogeneous. AUT asks for unconventional yet feasible uses of common objects; DAT asks for 10 nouns that are as semantically unrelated as possible; TTCT evaluates open-ended responses across seven task types; TTCW asks for long-form story generation from plot summaries; Creative Short Story constrains generation with three required words and a “boring theme” to avoid; Creativity Index evaluates originality of continuations relative to traceable web text; CS4 evaluates story revision under increasing constraints; NeoCoder measures coding under forbidden-technique constraints; and Creative Math measures correctness plus distinctness from reference solutions (Hou et al., 23 Oct 2025).

Domain	Tasks	Typical dimension coverage
Divergent thinking	AUT, DAT, TTCT	novelty, diversity, and TTCT quality
Creative writing	TTCW, Creative Short Story, Creativity Index, CS4	all three, but task-specific
Logical reasoning	NeoCoder, Creative Math	quality and novelty

The metric inventory is similarly task-specific. AUT contributes an AUT score; DAT contributes a DAT score; TTCT contributes Fluency, Flexibility, Originality, and Elaboration; TTCW contributes Narrative Ending, Understandability and Coherence, Emotional Flexibility, and World Building and Setting; Creative Short Story contributes Novelty Score, Surprise-ness, and N-gram Diversity; Creativity Index contributes L-uniqueness; CS4 contributes QUC, RCS transformed as $1-\mathrm{RCS}$ , and Dist-N; NeoCoder contributes Convergence@k and Divergent@k; and Creative Math contributes Correctness Ratio and Novelty Ratio (Hou et al., 23 Oct 2025).

An important ambiguity is acknowledged in the benchmark description itself: a naive count of the listed metrics yields 21, whereas the paper consistently states 20 metrics and does not explicitly resolve the discrepancy (Hou et al., 23 Oct 2025).

3. Scoring formalism and evaluation protocol

CreativityPrism normalizes all raw metric scores before aggregation. For model $i$ and metric $m$ , the normalized score is

$\hat S_{i,m}=\frac{S_{i,m}-\min_m}{\max_m-\min_m}.$

If a task $t$ has multiple metrics within the same creativity dimension, those normalized metric scores are averaged first:

$\bar S_{i,t}=\frac{1}{k_t}\sum_{m\in M_t}\hat S_{i,m}.$

For dimension $d \in \{\text{qual},\text{nov},\text{div}\}$ , the dimension-level score is then

$D_i^{(d)}=\frac{1}{|T_d|}\sum_{t\in T_d}\bar S_{i,t}.$

The overall creativity score is the mean of the three dimension scores:

$C_i=\frac{D_i^{(\text{qual})}+D_i^{(\text{nov})}+D_i^{(\text{div})}}{3}.$

The authors explicitly note that this overall score is mainly a convenience for comparison; in practice, the separate dimension scores are preferred (Hou et al., 23 Oct 2025).

The evaluation covers 17 models, including open-source families such as Mistral, Qwen, OLMo, and Llama, as well as API-accessed systems such as GPT-4.1, GPT-4.1-mini, Claude 3.7 Sonnet, Claude 3.5 Haiku, Gemini 2.0 Flash, DeepSeek-R1, and DeepSeek-V3. Open-source models were run with vLLM v0.7.2, while proprietary systems and DeepSeek models were accessed through APIs. Task settings vary, but the benchmark largely follows the original papers’ prompting and decoding protocols for each task (Hou et al., 23 Oct 2025).

Six of the nine tasks use LLM judges: AUT, TTCT, TTCW, CS4, Creative Math, and NeoCoder for technique detection. The default judge is Qwen2.5-72B; Creative Math correctness uses Claude-3.7-Sonnet; and Creative Math novelty uses a majority vote of Gemini-2.0-Flash, GPT-4.1, and Claude-3.7-Sonnet. Reported judge-alignment figures include 0.7 human–LLM Pearson correlation for AUT from prior work using the same setup, 0.94 recall for NeoCoder technique detection, 0.55 Pearson correlation for CS4 with $p<0.01$ , 0.78 novelty accuracy and 0.94 correctness accuracy for Creative Math, and TTCT proxy judge correlations ranging from 0.5033 to 0.6884 (Hou et al., 23 Oct 2025).

4. Empirical findings

The benchmark reports a substantial gap between proprietary/API-accessed systems and open-source models. The best open-source model is Qwen2.5-72B with an overall score of 0.596, while the best proprietary/API model is DeepSeek-V3 with 0.739; other top proprietary/API systems include GPT-4.1 at 0.721, Claude 3.7 Sonnet at 0.697, GPT-4.1-mini at 0.695, Gemini 2.0 Flash at 0.677, and DeepSeek-R1 at 0.638 (Hou et al., 23 Oct 2025).

This gap is not uniform. It is largest in logical reasoning, then creative writing, and smallest in divergent thinking. Across creativity dimensions, the proprietary/API lead is larger for quality and diversity than for novelty. The paper interprets this as consistent with stronger industrial optimization for coding, math, and writing, while novelty remains harder to operationalize and less transferable (Hou et al., 23 Oct 2025).

A second major result concerns metric correlations. Metrics within the same task correlate strongly, and tasks within the same domain often correlate more than tasks across domains. The paper highlights TTCW and TTCT as especially cohesive, with metric-to-metric correlations greater than 0.85 for all metrics within those tasks. By contrast, cross-domain correlations are weaker, which supports the claim that “strong performance in one creativity task or dimension does not necessarily generalize to others” (Hou et al., 23 Oct 2025).

Among the three dimensions, quality and diversity show strong correlations, whereas novelty is far less coherent. The framework presents novelty as the most heterogeneous dimension because it refers to different things in different tasks: uncommon uses in AUT, low corpus overlap in Creativity Index, technique-level divergence in NeoCoder, method-level distinctness in Creative Math, and sentence-to-sentence semantic surprise in Creative Short Story. The paper even notes a negative correlation of $i$ 0 between Surprises in Creative Short Story and Divergence@0 in NeoCoder, underscoring that novelty does not behave like a single latent factor across tasks (Hou et al., 23 Oct 2025).

5. Interpretation, limitations, and controversies

CreativityPrism is explicit that it does not measure a single, unified creativity faculty. Its strongest interpretive claim is that LLM creativity is multidimensional, domain-dependent, and only partially transferable across tasks. A direct consequence is that overall benchmark scores should be treated cautiously; the separate dimension and task scores are often more informative than the aggregate $i$ 1 (Hou et al., 23 Oct 2025).

The benchmark also states several limitations. It is English-only, which matters because creativity can be culturally grounded. It covers only text tasks, not multimodal settings, and its task selection is biased toward benchmarks with scalable automatic evaluation. This bias is especially consequential for novelty, because high-level novelty often still requires expert human judgment and is not well captured by currently scalable metrics (Hou et al., 23 Oct 2025).

A second limitation concerns LLM-as-a-judge. Six of the nine tasks involve judge models, so evaluator bias remains unavoidable even when alignment checks are reported. The benchmark explicitly notes concerns about evaluator preference bias, family-related bias, and the general instability of LLM judgment. Prompt dependence is another limitation: several tasks, especially TTCT and AUT, are sensitive to prompting and decoding setup, even though CreativityPrism standardizes these settings where possible (Hou et al., 23 Oct 2025).

A third limitation concerns the meaning of novelty metrics themselves. Creativity Index depends on visible corpora and can misjudge paraphrases; later model release dates and private training corpora may confer advantages; and novelty metrics that reward difference from prior text do not necessarily reward meaningful originality. This suggests that CreativityPrism is best understood as a benchmark of operationalized creativity proxies rather than a final theory of creative cognition.

6. Position in the broader creativity-evaluation landscape

CreativityPrism occupies a distinctive position because it is holistic across text tasks but still product-centric and benchmark-oriented. By contrast, CrIB targets a narrower facet—goal-driven combinational p-creativity—through 2000 problems across five domains, emphasizing novelty relative to an agent’s initial knowledge rather than a broad task suite (Guzdial et al., 2018). In visual creativity, CREward decomposes image creativity into geometry, material, and texture, while the drawing-assessment framework in “Simple Lines, Big Ideas” decomposes creativity into content and style; both therefore pursue factorized evaluation, but on image artifacts rather than LLM-generated text (Han et al., 25 Nov 2025, Lin et al., 17 Nov 2025).

A similar pattern appears in domain-specific design evaluation. The Rowen Test of Creativity in Visualisation Design evaluates low-fidelity visualization sketches along Quantity, Correctness, Novelty, and Feasibility, again showing that creativity assessment often becomes more interpretable when decomposed into separable criteria (Owen et al., 2024). This suggests that CreativityPrism belongs to a broader research movement toward structured, multi-axis creativity evaluation rather than single-score judgments.

At the same time, CreativityPrism is not itself a creativity-generation method. It contrasts sharply with PRISM—Pluralistic Reasoning via In-context Structure Modeling—which intervenes at inference time to increase novelty and distributional diversity by constructing dynamic on-the-fly epistemic graphs for LLMs (Tu et al., 24 Feb 2026). CreativityPrism evaluates creativity across tasks and dimensions; PRISM attempts to induce more pluralistic outputs.

Finally, CreativityPrism should not be conflated with process theories of creativity such as Gabora’s accounts of creative inklings, potentiality, and honing, which emphasize reconstructive memory, context-sensitive emergence, and movement from “half-baked” ideas toward more definite form (Gabora, 2013, Gabora, 2015). A plausible implication is that CreativityPrism is strongest as an evaluation framework for creative products and task performance, whereas those theories address the microgenesis of creative thought itself.