Creativity Stress Test: Evaluating Creativity Under Pressure

• Creativity stress tests evaluate agents’ ability to produce innovative and constraint-adherent outputs as task complexity increases.
• They employ controlled escalation techniques—such as increasing constraint counts and prompt specificity—to challenge both human and AI creativity.
• Empirical findings reveal performance decay under pressure, highlighting differences in creative robustness and guiding improvements in evaluation methods.

A creativity stress test is an evaluation paradigm that probes whether a human or artificial agent can sustain originality, usefulness, coherence, or other creativity-relevant properties as task demands intensify. In the literature, the term refers both to explicit benchmark designs and to a broader testing philosophy: increasing constraint count, requiring divergent yet valid outputs, or forcing navigation of large solution spaces so that rote recall, template matching, or high-probability continuation becomes insufficient. Across symbolic, textual, visual, multimodal, and code-generation settings, these tests operationalize creativity as performance under pressure rather than as unconstrained aesthetic production (Riedl, 2014, Atmakuru et al., 2024, Wang et al., 12 Mar 2026).

1. Historical origin and conceptual framing

A major antecedent is the Lovelace 2.0 Test, introduced by Mark O. Riedl as both a test of computational creativity and, by extension, a stress test of human-level intelligence in artificial agents. Where the classic Turing Test relies on deception and conversational indistinguishability, the Lovelace 2.0 Test asks an agent to originate genuine artifacts under human-specified constraints. The test proceeds with three parties—an artificial agent $a$, a human evaluator $h$, and a human referee $r$—and requires the evaluator to select an artifact type $t$ and a finite set of constraints $C=\{c_1,c_2,\dots,c_k\}$, each requirement being expressible in natural language. The agent must produce an artifact $o$ of type $t$, after which the evaluator judges whether $o$ is a valid instance of $t$ and satisfies every constraint, while the referee vetoes any combination $(t,C)$ deemed “unrealistic for an average human” (Riedl, 2014).

This formulation was explicitly designed to resist mere lookup or rote template matching. The theoretical basis given for that resistance is that creative generation of many artifact types, notably fictional stories, entails commonsense reasoning, narrative planning, theory of mind, affective reasoning, and discourse structuring. The paper also states that the test builds directly on Bringsjord, Bello, and Ferrucci’s original Lovelace Test, incorporates Margaret Boden’s notions of creativity as producing artifacts that are valuable, novel, and surprising, and sidesteps Turing’s reliance on imitation and deception by demanding origination of concept under instruction (Riedl, 2014).

A later line of work generalizes the stress-test idea beyond single artifacts to large benchmark ecosystems. AGC-Bench introduces an artificial general creativity benchmark covering 78 datasets spanning brainstorming, problem solving, STEM, narrative, figurative language, and humor, and reports a single creativity factor $h$0 that explains $h$1 of variance across 83 LLMs, while remaining related to but separable from general knowledge and reasoning. This suggests that the stress-test concept has evolved from one-off challenge settings into psychometric infrastructures aimed at domain-spanning measurement (Beaty et al., 1 Jul 2026).

2. Core design principle: increasing pressure until routine generation fails

The central design principle is controlled escalation. In the Lovelace 2.0 protocol, evaluators administer a sequence of tests indexed by $h$2 until failure, with exactly $h$3 constraints in the $h$4-th test. If $h$5 is the largest $h$6 for which an agent succeeds under evaluator $h$7, the creativity score is the mean number of constraints handled before failure:

$h$8

No absolute threshold is specified for intelligence; higher $h$9 indicates greater creative capability under human-devised stresses (Riedl, 2014).

CS$r$0 applies the same escalation principle to story writing by synthesizing atomic constraints $r$1 and parameterizing prompt specificity by $r$2, with $r$3. As $r$4 grows, the overlap between the prompt and any single training-set story is intended to shrink, compelling the model to invent new narrative glue rather than regurgitate. The benchmark bundles coherence and constraint adherence into “Quality Under $r$5 Constraints,”

$r$6

and defines the Relative Creativity Score between two levels $r$7 as

$r$8

A smaller $r$9 indicates a model whose quality decays more slowly as constraints accumulate (Atmakuru et al., 2024).

Other stress tests vary the pressure variable rather than the literal number of constraints. CREATE evaluates associative creativity by asking models to generate multiple paths connecting concepts in a knowledge graph and rewards sets of answers that are simultaneously high in specificity and diversity. Creative utility is computed over an unordered set of paths $t$0 with a greedy ordering and a patience discount $t$1, so that later answers are penalized if they are too similar to earlier ones (Wadhwa et al., 10 Mar 2026). CreativeBench, in turn, distinguishes combinatorial and exploratory creativity in code generation and uses progressively harder automatically generated tasks, including self-play constraint stacking for constrained exploration (Wang et al., 12 Mar 2026).

A common implication across these frameworks is that creativity is tested not by whether a model can produce a single plausible answer, but by whether quality persists when the search space becomes sparse, recall is no longer sufficient, and multiple interacting requirements must be satisfied. This implication is explicit in CS$t$2 and CreativeBench and is already latent in the Lovelace 2.0 formulation (Riedl, 2014, Atmakuru et al., 2024, Wang et al., 12 Mar 2026).

3. Task families and benchmark realizations

The literature operationalizes creativity stress testing through heterogeneous task families rather than a single canonical benchmark.

Paradigm	Stress variable	Modality
Lovelace 2.0	Number and complexity of natural-language constraints	Artifact generation
CS$t$3	Number of story-writing constraints	Story generation
LLM Discussion	Multi-agent divergence, debate, and convergence	Open-ended text tasks
CREATE	Specificity and diversity of associative paths	Knowledge-graph reasoning
CreativeBench	Reverse-engineered fusion and self-play constraint stacking	Code generation
AGC-Bench	Cross-domain aggregation across 78 datasets	General text and multimodal creativity

In text generation, LLM Discussion organizes multiple agents into three phases—Divergence, Debate, and Convergence—over tasks such as the Alternative Uses Test, Similarities Test, Instances Test, and Scientific Creativity Test. The method fixes $t$4 and typically uses $t$5 rounds, with role-play prompts such as Visionary Millionaire, Social Entrepreneur, Creative Professional, Futurist, or Six Thinking Hats to combat homogeneity of LLMs. The framework is evaluated on originality, elaboration, fluency, and flexibility, and the reported table shows originality improvements over single-LLM baselines on all four benchmarks (Lu et al., 2024).

In code generation, CreativeBench comprises two subsets. CreativeBench-Combo targets combinatorial creativity by reverse-engineering problems from fused executable solutions, while CreativeBench-Explore targets exploratory creativity through dynamic constraint stacking that starts from a seed problem and repeatedly forbids the current core technique. Difficulty filtering rejects trivially solved tasks and retains levels with Pass@1 below $t$6 for strong LLMs, with a reported human audit validity of approximately $t$7 on a random sample (Wang et al., 12 Mar 2026).

In visual and multimodal settings, the Rowen Test of Creativity in Visualization Design uses a sketch-based format with four normalized dimensions—Quantity, Correctness, Novelty, and Feasibility—each on a $t$8 Likert scale, producing a total score $t$9 with $C=\{c_1,c_2,\dots,c_k\}$0. CreBench evaluates multimodal LLMs across Creative Idea, Creative Process, and Creative Product, each decomposed into rubric-based sub-tasks such as Originality, Divergence, Elaboration, Effectiveness, Aesthetic, Novelty, Manufacturability, and Systemic Complexity (Owen et al., 2024, Xue et al., 17 Nov 2025).

Psychometric item-generation work treats the construction of the test itself as an iterative creativity stress problem. The Creative Psychometric Item Generator uses LLM-based item generators and evaluators to iteratively develop new prompts for creative problem-solving scenarios, selecting exemplars that maximize response originality while controlling pool diversity, and reports that mean originality of synthetic responses more than doubles from round 1 to round 5 across open-source LLMs (Jr. et al., 2024).

4. Measurement frameworks and scoring mathematics

The defining feature of creativity stress tests is not only task difficulty but also formalized scoring that separates mere novelty from successful, constraint-compatible novelty.

Lovelace 2.0 uses a human-centered success criterion and a graded constraint-handling score $C=\{c_1,c_2,\dots,c_k\}$1. CS$C=\{c_1,c_2,\dots,c_k\}$2 decomposes story quality into a constraint-satisfaction ratio,

$C=\{c_1,c_2,\dots,c_k\}$3

a normalized coherence score derived from pairwise comparison against a fixed baseline story, and diversity measures including self-perplexity and

$C=\{c_1,c_2,\dots,c_k\}$4

These are then combined through $C=\{c_1,c_2,\dots,c_k\}$5 and $C=\{c_1,c_2,\dots,c_k\}$6 to quantify degradation under rising prompt specificity (Atmakuru et al., 2024).

CreativeBench defines creativity as the product of quality and novelty. For each generated code solution $C=\{c_1,c_2,\dots,c_k\}$7, quality is $C=\{c_1,c_2,\dots,c_k\}$8, novelty is a hybrid distance $C=\{c_1,c_2,\dots,c_k\}$9, and the creativity score is

$o$0

Novelty combines embedding-space distance and character 4-gram dissimilarity. The multiplication rather than addition is intended to penalize outputs that are highly novel but incorrect, or correct but routine (Wang et al., 12 Mar 2026).

CREATE formalizes an analogous trade-off for associative reasoning. A path quality function $o$1 is determined by factuality and the minimum specificity over the triples in the path, while pairwise path diversity is measured by a transformed distance $o$2 from embedding cosine dissimilarity. The overall creative utility of a response set $o$3 is then an ordered sum of discounted quality times marginal diversity, reported as $o$4 and $o$5 for patience values $o$6 and $o$7 (Wadhwa et al., 10 Mar 2026).

Some frameworks retain classical divergent-thinking metrics. LLM Discussion adopts Torrance TTCT dimensions—Originality, Elaboration, Fluency, Flexibility—and gives explicit formulas for the Alternative Uses Test and Instances Test:

$o$8

For the Scientific Creativity Test, it uses Analytical Creativity $o$9 and Synthetic Creativity $t$0 with

$t$1

These metrics support head-to-head comparison between single-LLM and multi-agent discussion protocols (Lu et al., 2024).

A different measurement tradition uses pairwise comparison rather than direct rating. CreataSet and CrEval frame creativity evaluation as a three-way classification over response pairs $t$2 under a shared instruction $t$3, with labels $t$4. A single response can then be scored by its average win rate against other candidates under the same instruction (Cao et al., 25 May 2025). AGC-Bench extends psychometric calibration further through Judge Response Theory, fitting a Bayesian graded response model to correct judge leniency and severity before aggregating benchmark-level creativity scores and extracting the latent factor $t$5 (Beaty et al., 1 Jul 2026).

5. Empirical findings: what these tests reveal

The broad empirical pattern is performance decay under pressure. In CS$t$6, all tested models’ constraint-satisfaction ratios and coherence scores fall as the number of constraints rises. The paper reports that LLaMA-2 drops approximately $t$7 percentage points in satisfaction from $t$8 to $t$9, while Gemma loses only approximately $o$0 points. Gemma and Mistral lead in $o$1 at approximately $o$2, LLaMA follows at approximately $o$3, and OLMo Base trails at approximately $o$4. The smallest $o$5 belongs to Gemma, which the paper interprets as the most creative under stress (Atmakuru et al., 2024).

In LLM Discussion, a discussion framework with four agents, five rounds, and temperature $o$6 outperforms single-LLM baselines on originality across four benchmarks: AUT improves from $o$7 to $o$8 $o$9, Instances from $t$0 to $t$1 $t$2, Similarities from $t$3 to $t$4 $t$5, and Scientific from $t$6 to $t$7 $t$8. Elaboration gains are reported as $t$9–$(t,C)$0 (Lu et al., 2024).

The human-versus-model comparison remains mixed. In the Alternative Uses Test study of GPT-3, humans outperform GPT-3 on originality ratings $(t,C)$1, surprise ratings $(t,C)$2, and semantic distance $(t,C)$3, while GPT-3 outperforms humans on usefulness ratings $(t,C)$4. Flexible responding is also higher on average for humans, although GPT-3’s flexibility shows greater variance (Stevenson et al., 2022).

Several studies identify a recurrent failure mode: apparent novelty without deep originality. The advertising study framed as a Galton-style regression to the mean reports that creative features such as metaphors, emotions, and visual cues disappear early under staged contraction, while factual content remains. In plain expansion from highly compressed inputs, cosine similarity remains approximately $(t,C)$5–$(t,C)$6, METEOR approximately $(t,C)$7–$(t,C)$8, entropy rises above the original, and 4-gram uniqueness reaches approximately $(t,C)$9–$h$00, indicating lexical variety without recovery of distinctive ideas. Marker-driven expansion improves alignment but still relies on familiar tropes (Keon et al., 30 Sep 2025).

The Circles Exercise study reaches a related conclusion under the label “narrow creativity.” Humans and GenAI both concentrate heavily on small subsets of categories. Humans use on average $h$01 of 10 categories, with approximately $h$02 of sketches in frequent categories; GenAI CoT uses $h$03 categories but still retains approximately $h$04 frequent-category concentration. The paper states that Chain-of-Thought prompting mitigates narrow creativity issues but still falls short of substantially broadening the creative scope of humans and GenAI (Duan et al., 11 Feb 2025).

6. Debates, limitations, and methodological frontiers

A central debate concerns whether human creativity tests are valid measures of machine creativity. A large-scale 2026 study argues that the validity of human creativity tests for LLMs has not been established and that these tests already have limited validity as predictors of human creativity. It finds that the Divergent Association Task and Conditional DAT are the best predictors of creative writing and divergent thinking respectively, but that no single test predicts all constructs well, and that no existing test reliably predicts scientific ideation ability except the newly introduced Divergent Remote Association Test (Schapiro et al., 13 May 2026).

Another debate concerns whether creativity is domain-specific or psychometrically general. AGC-Bench reports a single creativity factor $h$05 with domain loadings of approximately $h$06–$h$07, first eigenvalue $h$08, and Cronbach $h$09, yet also reports only partial correlations with a pure-$h$10 probe $h$11 and MMLU-Pro $h$12. The paper’s conclusion is that creativity is related to but separable from general intelligence and knowledge (Beaty et al., 1 Jul 2026).

Evaluation reliability is a recurring methodological problem. Lovelace 2.0 depends on evaluator choice and on a referee who can identify demands exceeding “average human” capability. The Rowen Test currently relies on self-ratings and notes that inter-rater reliability is “not yet applicable” in the pilot, with future studies planned to introduce external raters and compute Cronbach’s $h$13. CS$h$14 and related recent benchmarks respond to these concerns by using LLM-as-judge pipelines, pairwise comparison protocols, or psychometric calibration, but also retain human calibration for edge cases (Riedl, 2014, Owen et al., 2024, Atmakuru et al., 2024).

A further limitation is that many benchmarks operationalize creativity as one or two measurable proxies and therefore omit other dimensions. Lovelace 2.0 does not judge aesthetic quality. CS$h$15 does not claim to measure all aspects of story quality and focuses on constraint adherence, coherence, and diversity. CreativeBench defines creativity as quality times novelty, which objectively distinguishes creativity from hallucination in executable code, but this definition is specialized to artifacts with sandbox-verifiable quality signals (Riedl, 2014, Atmakuru et al., 2024, Wang et al., 12 Mar 2026).

Future directions in the literature converge on several themes. These include multilevel difficulty rather than a single benchmark level, automated yet calibrated evaluators, planned-missing multi-judge designs, adaptive testing, cross-domain aggregation, and inference-time steering or search methods that explicitly counter model homogeneity and routine generation. A plausible implication is that the field is moving from isolated creativity tasks toward standardized stress infrastructures in which the main experimental variable is not simply whether a model can generate an unusual artifact, but how gracefully quality degrades as novelty pressure, specificity, and combinatorial burden increase (Lu et al., 2024, Beaty et al., 1 Jul 2026, Wang et al., 12 Mar 2026).