---
title: 'CreativityPrism: Multidimensional LLM Creativity Framework'
url: https://www.emergentmind.com/topics/creativityprism
type: topic
---

# CreativityPrism: Multidimensional LLM Creativity Framework

Searching arXiv for the core paper and directly related "Prism"/creativity references to support the article.
CreativityPrism is a benchmark and evaluation-analysis framework for large language model creativity that treats creativity as a multidimensional construct rather than a single scalar capability. It spans **9 tasks**, **3 domains**—**divergent thinking**, **creative writing**, and **logical reasoning**—and **20 evaluation metrics**, organized into the three dimensions of **quality**, **novelty**, and **diversity**. Its central hypothesis is that “creativity is not one fixed idea,” so strong performance on one task or dimension should not be assumed to transfer across others; accordingly, the framework evaluates **17 state-of-the-art proprietary and open-sourced LLMs** and emphasizes cross-task, cross-domain, and cross-dimension analysis rather than a single leaderboard number [2510.20091].

## 1. Definition and conceptual basis

CreativityPrism was proposed in response to two linked problems in LLM creativity evaluation. First, creativity is hard to evaluate automatically at scale, and prior work often depends on expensive human judgment. Second, creativity is defined inconsistently across tasks and domains, so apparently similar claims about “creative” performance may rest on incompatible metrics. The framework therefore organizes task-specific evaluation under a common taxonomy of **quality**, **novelty**, and **diversity**, while preserving the fact that these dimensions are instantiated differently across tasks [2510.20091].

In this taxonomy, **quality** measures whether a generation satisfies a task’s core functional requirements; **novelty** measures how rare or original a response is relative to existing or conventional content; and **diversity** measures variation among generated outputs. The framework explicitly treats quality as the convergent side of creativity and novelty-plus-diversity as divergent aspects. This decomposition is intended to avoid the failure mode in which one metric—often lexical variability or corpus rarity—is overinterpreted as a general creativity proxy [2510.20091].

The “prism” metaphor is methodological rather than decorative. Just as a prism decomposes white light into multiple colors, CreativityPrism decomposes model creativity into distinct evaluative aspects across contexts. A plausible implication is that the framework should be read less as a universal definition of creativity than as an operational scheme for comparing heterogeneous creativity tasks without collapsing them into one undifferentiated criterion.

## 2. Benchmark structure and task inventory

CreativityPrism distributes its tasks across three domains. The divergent-thinking domain includes **Alternative Uses Test (AUT)**, **Divergent Association Task (DAT)**, and **Torrance Tests of Creative Thinking (TTCT)**. The creative-writing domain includes **Torrance Test of Creative Writing (TTCW)**, **Creative Short Story**, **Creativity Index**, and **CS4**. The logical-reasoning domain includes **NeoCoder** and **Creative Math** [2510.20091].

The framework is intentionally heterogeneous. AUT asks for unconventional yet feasible uses of common objects; DAT asks for 10 nouns that are as semantically unrelated as possible; TTCT evaluates open-ended responses across seven task types; TTCW asks for long-form story generation from plot summaries; Creative Short Story constrains generation with three required words and a “boring theme” to avoid; Creativity Index evaluates originality of continuations relative to traceable web text; CS4 evaluates story revision under increasing constraints; NeoCoder measures coding under forbidden-technique constraints; and Creative Math measures correctness plus distinctness from reference solutions [2510.20091].

| Domain | Tasks | Typical dimension coverage |
|---|---|---|
| Divergent thinking | AUT, DAT, TTCT | novelty, diversity, and TTCT quality |
| Creative writing | TTCW, Creative Short Story, Creativity Index, CS4 | all three, but task-specific |
| Logical reasoning | NeoCoder, Creative Math | quality and novelty |

The metric inventory is similarly task-specific. AUT contributes an AUT score; DAT contributes a DAT score; TTCT contributes **Fluency**, **Flexibility**, **Originality**, and **Elaboration**; TTCW contributes **Narrative Ending**, **Understandability and Coherence**, **Emotional Flexibility**, and **World Building and Setting**; Creative Short Story contributes **Novelty Score**, **Surprise-ness**, and **N-gram Diversity**; Creativity Index contributes **L-uniqueness**; CS4 contributes **QUC**, **RCS** transformed as \(1-\mathrm{RCS}\), and **Dist-N**; NeoCoder contributes **Convergence@k** and **Divergent@k**; and Creative Math contributes **Correctness Ratio** and **Novelty Ratio** [2510.20091].

An important ambiguity is acknowledged in the benchmark description itself: a naive count of the listed metrics yields **21**, whereas the paper consistently states **20 metrics** and does not explicitly resolve the discrepancy [2510.20091].

## 3. Scoring formalism and evaluation protocol

CreativityPrism normalizes all raw metric scores before aggregation. For model \(i\) and metric \(m\), the normalized score is

$$
\hat S_{i,m}=\frac{S_{i,m}-\min_m}{\max_m-\min_m}.
$$

If a task \(t\) has multiple metrics within the same creativity dimension, those normalized metric scores are averaged first:

$$
\bar S_{i,t}=\frac{1}{k_t}\sum_{m\in M_t}\hat S_{i,m}.
$$

For dimension \(d \in \{\text{qual},\text{nov},\text{div}\}\), the dimension-level score is then

$$
D_i^{(d)}=\frac{1}{|T_d|}\sum_{t\in T_d}\bar S_{i,t}.
$$

The overall creativity score is the mean of the three dimension scores:

$$
C_i=\frac{D_i^{(\text{qual})}+D_i^{(\text{nov})}+D_i^{(\text{div})}}{3}.
$$

The authors explicitly note that this overall score is mainly a convenience for comparison; in practice, the separate dimension scores are preferred [2510.20091].

The evaluation covers **17** models, including open-source families such as **Mistral**, **Qwen**, **OLMo**, and **Llama**, as well as API-accessed systems such as **GPT-4.1**, **GPT-4.1-mini**, **Claude 3.7 Sonnet**, **Claude 3.5 Haiku**, **Gemini 2.0 Flash**, **DeepSeek-R1**, and **DeepSeek-V3**. Open-source models were run with **vLLM v0.7.2**, while proprietary systems and DeepSeek models were accessed through APIs. Task settings vary, but the benchmark largely follows the original papers’ prompting and decoding protocols for each task [2510.20091].

Six of the nine tasks use LLM judges: **AUT**, **TTCT**, **TTCW**, **CS4**, **Creative Math**, and **NeoCoder** for technique detection. The default judge is **Qwen2.5-72B**; **Creative Math correctness** uses **Claude-3.7-Sonnet**; and **Creative Math novelty** uses a majority vote of **Gemini-2.0-Flash**, **GPT-4.1**, and **Claude-3.7-Sonnet**. Reported judge-alignment figures include **0.7** human–LLM Pearson correlation for AUT from prior work using the same setup, **0.94** recall for NeoCoder technique detection, **0.55** Pearson correlation for CS4 with \(p<0.01\), **0.78** novelty accuracy and **0.94** correctness accuracy for Creative Math, and TTCT proxy judge correlations ranging from **0.5033** to **0.6884** [2510.20091].

## 4. Empirical findings

The benchmark reports a substantial gap between proprietary/API-accessed systems and open-source models. The best open-source model is **Qwen2.5-72B** with an overall score of **0.596**, while the best proprietary/API model is **DeepSeek-V3** with **0.739**; other top proprietary/API systems include **GPT-4.1** at **0.721**, **Claude 3.7 Sonnet** at **0.697**, **GPT-4.1-mini** at **0.695**, **Gemini 2.0 Flash** at **0.677**, and **DeepSeek-R1** at **0.638** [2510.20091].

This gap is not uniform. It is largest in **logical reasoning**, then **creative writing**, and smallest in **divergent thinking**. Across creativity dimensions, the proprietary/API lead is larger for **quality** and **diversity** than for **novelty**. The paper interprets this as consistent with stronger industrial optimization for coding, math, and writing, while novelty remains harder to operationalize and less transferable [2510.20091].

A second major result concerns metric correlations. Metrics within the same task correlate strongly, and tasks within the same domain often correlate more than tasks across domains. The paper highlights **TTCW** and **TTCT** as especially cohesive, with metric-to-metric correlations greater than **0.85** for all metrics within those tasks. By contrast, cross-domain correlations are weaker, which supports the claim that “strong performance in one creativity task or dimension does not necessarily generalize to others” [2510.20091].

Among the three dimensions, **quality** and **diversity** show strong correlations, whereas **novelty** is far less coherent. The framework presents novelty as the most heterogeneous dimension because it refers to different things in different tasks: uncommon uses in AUT, low corpus overlap in Creativity Index, technique-level divergence in NeoCoder, method-level distinctness in Creative Math, and sentence-to-sentence semantic surprise in Creative Short Story. The paper even notes a negative correlation of **\(-0.25\)** between **Surprises** in Creative Short Story and **Divergence@0** in NeoCoder, underscoring that novelty does not behave like a single latent factor across tasks [2510.20091].

## 5. Interpretation, limitations, and controversies

CreativityPrism is explicit that it does not measure a single, unified creativity faculty. Its strongest interpretive claim is that LLM creativity is **multidimensional**, **domain-dependent**, and only **partially transferable** across tasks. A direct consequence is that overall benchmark scores should be treated cautiously; the separate dimension and task scores are often more informative than the aggregate \(C_i\) [2510.20091].

The benchmark also states several limitations. It is **English-only**, which matters because creativity can be culturally grounded. It covers only **text tasks**, not multimodal settings, and its task selection is biased toward benchmarks with scalable automatic evaluation. This bias is especially consequential for novelty, because high-level novelty often still requires expert human judgment and is not well captured by currently scalable metrics [2510.20091].

A second limitation concerns **LLM-as-a-judge**. Six of the nine tasks involve judge models, so evaluator bias remains unavoidable even when alignment checks are reported. The benchmark explicitly notes concerns about evaluator preference bias, family-related bias, and the general instability of LLM judgment. Prompt dependence is another limitation: several tasks, especially **TTCT** and **AUT**, are sensitive to prompting and decoding setup, even though CreativityPrism standardizes these settings where possible [2510.20091].

A third limitation concerns the meaning of novelty metrics themselves. **Creativity Index** depends on visible corpora and can misjudge paraphrases; later model release dates and private training corpora may confer advantages; and novelty metrics that reward difference from prior text do not necessarily reward meaningful originality. This suggests that CreativityPrism is best understood as a benchmark of operationalized creativity proxies rather than a final theory of creative cognition.

## 6. Position in the broader creativity-evaluation landscape

CreativityPrism occupies a distinctive position because it is holistic across text tasks but still product-centric and benchmark-oriented. By contrast, **CrIB** targets a narrower facet—**goal-driven combinational p-creativity**—through **2000 problems** across five domains, emphasizing novelty relative to an agent’s initial knowledge rather than a broad task suite [1805.03720]. In visual creativity, **CREward** decomposes image creativity into **geometry**, **material**, and **texture**, while the drawing-assessment framework in “Simple Lines, Big Ideas” decomposes creativity into **content** and **style**; both therefore pursue factorized evaluation, but on image artifacts rather than LLM-generated text [2511.19995] [2511.12880].

A similar pattern appears in domain-specific design evaluation. The **Rowen Test of Creativity in Visualisation Design** evaluates low-fidelity visualization sketches along **Quantity**, **Correctness**, **Novelty**, and **Feasibility**, again showing that creativity assessment often becomes more interpretable when decomposed into separable criteria [2409.02036]. This suggests that CreativityPrism belongs to a broader research movement toward structured, multi-axis creativity evaluation rather than single-score judgments.

At the same time, CreativityPrism is not itself a creativity-generation method. It contrasts sharply with **PRISM**—**Pluralistic Reasoning via In-context Structure Modeling**—which intervenes at inference time to increase novelty and distributional diversity by constructing dynamic on-the-fly epistemic graphs for LLMs [2602.21317]. CreativityPrism evaluates creativity across tasks and dimensions; PRISM attempts to induce more pluralistic outputs.

Finally, CreativityPrism should not be conflated with process theories of creativity such as Gabora’s accounts of **creative inklings**, **potentiality**, and **honing**, which emphasize reconstructive memory, context-sensitive emergence, and movement from “half-baked” ideas toward more definite form [1310.0736] [1501.04406]. A plausible implication is that CreativityPrism is strongest as an evaluation framework for creative products and task performance, whereas those theories address the microgenesis of creative thought itself.

Source: https://www.emergentmind.com/topics/creativityprism