Divergent Remote Association Test (DRAT)

• DRAT is defined as a vocabulary-space test that hybridizes divergent and convergent thinking by requiring generated words to be both semantically relevant to multiple anchors and sufficiently diverse.
• Its methodology uses calibrated relevance thresholds and pairwise cosine distance scoring with Sentence-BERT to ensure a balanced evaluation of creativity.
• Empirical results show that DRAT significantly predicts scientific ideation with higher validity and specificity compared to DAT, RAT, CDAT, and PACE.

Searching arXiv for the cited DRAT/DAT and related association-reasoning papers to ground the article. arxiv_search(query="Divergent Remote Association Test DRAT creativity LLM", max_results=10) The Divergent Remote Association Test (DRAT) is a vocabulary-space test that jointly measures divergent and convergent thinking. It was introduced to address a specific gap in LLM creativity evaluation: although tests such as the Divergent Association Task (DAT), Conditional DAT (CDAT), PACE, and the Remote Associates Test (RAT) are informative for some targets, none of them reliably predicts scientific ideation ability in LLMs. DRAT operationalizes a hybrid requirement: generated words must be semantically relevant to a set of remote anchors, yet also maximally different from one another. In the reported study, it is the first and only creativity test for LLMs that is a significant predictor of scientific ideation ability, and its advantage is not recoverable from any linear combination of DAT and RAT (Schapiro et al., 13 May 2026).

1. Definition and conceptual placement

DRAT is defined around an anchor set $A = (a_1,\dots,a_k)$ of $k$ remote anchor words, a word-to-embedding map $\phi : \text{word} \to \mathbb{R}^d$, and a model response $W$ consisting of $\sim 10$ nouns. The test is designed to combine two constructs that are usually separated in creativity testing. Its convergent component comes from requiring each candidate word to be semantically useful relative to the anchor set; its divergent component comes from measuring semantic spread among the surviving words. The resulting instrument is therefore neither a pure divergent-association task nor a pure remote-association task, but a hybrid of both (Schapiro et al., 13 May 2026).

This placement is clearest when DRAT is contrasted with adjacent tests. DAT asks participants or models to generate unrelated nouns and scores the average pairwise semantic distance among them; it is an objective measure of divergent thinking and creativity (Chen et al., 2023). RAT, by contrast, is a classic convergent creativity or insight test in which three cue words require one connecting word. CDAT adds a single cue word to DAT-style divergence, and PACE evaluates associative chains, but neither yields significant validity or specificity for scientific ideation in the reported large-scale LLM study (Schapiro et al., 13 May 2026).

A broader cognitive interpretation is available from work on associative versus analytic thought. Creativity has been characterized as involving shifts between an associative or divergent mode, which is conducive to remote or subtle associations, and an analytic or convergent mode, which is conducive to rule-based operations and solution refinement. DRAT aligns naturally with that framework because it requires remote associative breadth together with constraint satisfaction, rather than either capacity in isolation (Gabora, 2013).

2. Task structure and scoring procedure

A DRAT item consists of an anchor set and a prompt instructing the model to generate 10 nouns that are maximally different from each other and each of which could be metaphorically applied to all of the anchors. In the main configuration, the study uses 30 anchor sets with $k=4$ anchors each, mostly scientific terms. An example anchor set is “heartbeat”, “oscillator”, “pipeline”, “topology” (Schapiro et al., 13 May 2026).

Scoring proceeds in four steps. First, each response word is assigned a utility relative to the anchors:

$$u(w;\,A) \;=\; \max_{i \in \{1,\dots,k\}} \cos\!\bigl(\phi(w),\, \phi(a_i)\bigr).$$

Second, an anchor-specific relevance threshold is calibrated from a large pool $\mathcal{P}$ of random nouns:

$$\tau_A \;=\; \mathrm{quantile}_q\!\Bigl(\bigl\{ u(p;\,A) : p \in \mathcal{P} \bigr\}\Bigr),$$

with $q = 0.90$. Third, only useful words are retained:

$k$0

Fourth, the score is the DAT-style mean pairwise cosine distance among the surviving words, provided that at least $k$1 survive:

$k$2

Scores lie in the same $k$3 range as DAT. The main results use Sentence-BERT all-mpnet-base-v2 for similarity computations (Schapiro et al., 13 May 2026).

The scoring design makes two failure modes explicit. In diversity collapse, words are relevant to the anchors but occupy a narrow semantic band; the paper’s example “rhythm, pulse, flow, network, passage, current, circuit, structure, cascade, vibration” scores 71.31. In relevance collapse, words are diverse but mostly unrelated to the anchors; the example “sunrise, thunderbolt, maze, symphony, sculpture, ocean, meteor, rainbow, mosaic, quasar” yields zero survivors and therefore a score of 0. A balanced response such as “river, symphony, skeleton, breath, labyrinth, garden, engine, web, clockwork, fabric” scores 81.99 (Schapiro et al., 13 May 2026).

3. Relation to DAT, RAT, CDAT, and PACE

The distinctive feature of DRAT is that it embeds convergent and divergent demands within the same item rather than combining separate tests post hoc. DAT computes mean pairwise semantic distance among generated words and is intended to measure divergent semantic association. In LLM evaluation, DAT revealed that under greedy decoding GPT-4 achieved a DAT of 89.1 and outperformed 96.1% of humans, while GPT-3.5-turbo achieved 80.8 and exceeded the average human level; these findings established that advanced LLMs can exhibit strong divergent semantic association (Chen et al., 2023). DRAT inherits DAT’s distance-based machinery but conditions it on anchor-relevance filtering.

The contrast with nearby instruments is summarized below.

Test	Core requirement	Limitation relative to DRAT
DAT	Generate 10 unrelated nouns; score mean pairwise semantic distance	No relevance constraint
CDAT	Diversity plus relevance to a single cue word	Single-cue utility only
RAT	Produce one connecting word for remote cues	Convergent only
PACE	Associative chains	No explicit multi-anchor gate
DRAT	Generate 10 nouns that are maximally different and metaphorically applicable to multiple anchors	Hybrid convergent-divergent design

The empirical consequence is central. Regressing scientific-ideation performance on DAT and RAT together gives $k$4 with $k$5, essentially no predictive power. Adding DRAT raises $k$6 to 0.293, with $k$7, $k$8, and $k$9. The reverse analysis shows the asymmetry: DRAT alone gives $\phi : \text{word} \to \mathbb{R}^d$0, and adding DAT and RAT raises this only slightly to 0.293, with $\phi : \text{word} \to \mathbb{R}^d$1, $\phi : \text{word} \to \mathbb{R}^d$2, and $\phi : \text{word} \to \mathbb{R}^d$3 (Schapiro et al., 13 May 2026). This directly contradicts the common simplification that DRAT is merely “DAT plus RAT.”

4. Empirical validity for scientific ideation

The principal evaluation target is LiveIdeaBench, a benchmark for scientific idea generation scored on originality, flexibility, feasibility, clarity, and fluency. For overall LiveIdeaBench score, DRAT achieves validity $\phi : \text{word} \to \mathbb{R}^d$4 with $\phi : \text{word} \to \mathbb{R}^d$5, and specificity $\phi : \text{word} \to \mathbb{R}^d$6 with $\phi : \text{word} \to \mathbb{R}^d$7, where the capability controls are Arena Overall Elo and MMLU-Pro accuracy (Schapiro et al., 13 May 2026).

Facet-level correlations are uneven rather than uniform. DRAT correlates strongly with flexibility, $\phi : \text{word} \to \mathbb{R}^d$8 and $\phi : \text{word} \to \mathbb{R}^d$9, and with clarity, $W$0 and $W$1. It also correlates with originality, $W$2, although the reported specificity for originality is positive but non-significant at the study’s sample size. By contrast, feasibility and fluency are not reliably predicted: feasibility yields $W$3 and non-significant specificity, while fluency yields $W$4 and non-significant specificity (Schapiro et al., 13 May 2026).

The comparison with alternative tests is decisive. On overall LiveIdeaBench, DAT gives validity $W$5 and specificity $W$6; CDAT gives $W$7 and $W$8; PACE gives $W$9 and $\sim 10$0; RAT gives $\sim 10$1 and $\sim 10$2. All of these are non-significant in the reported evaluation, whereas DRAT is significant on both axes (Schapiro et al., 13 May 2026).

The paper also frames these results against a validity–specificity frontier. If $\sim 10$3 is the correlation between a benchmark and its capability-based prediction, then any test with validity $\sim 10$4 satisfies

$\sim 10$5

The point of this bound is methodological rather than merely formal: specificity is structurally limited when a downstream benchmark is tightly coupled to general model capability (Schapiro et al., 13 May 2026).

5. Design choices, robustness, and adjacent benchmarks

Several ablations indicate that DRAT’s predictive behavior depends on concrete design choices rather than on arbitrary aggregation. With scientific-term anchors, predictive power increases monotonically with the number of anchors: for $\sim 10$6, validity is $\sim 10$7 and specificity $\sim 10$8; for $\sim 10$9, validity is $k=4$0 and specificity $k=4$1; for $k=4$2, validity is $k=4$3 and specificity $k=4$4. Replacing scientific anchors with ConceptNet anchors weakens performance: ConceptNet with $k=4$5 yields validity $k=4$6 and specificity $k=4$7, both non-significant. Utility aggregation also matters: the $k=4$8 utility used in the main definition outperforms $k=4$9 and average variants (Schapiro et al., 13 May 2026).

DRAT sits within a broader family of association benchmarks. MM-OPERA extends remote-association evaluation into an open-ended multimodal setting with 11,497 instances across Remote-Item Association (RIA) and In-Context Association (ICA). Although it does not define DRAT by name, it explicitly aims to resemble the spirit of divergent thinking and convergent associative reasoning through free-form responses and explicit reasoning paths. Its scoring framework includes holistic 0–4 ratings, the divergence indicator $$u(w;\,A) \;=\; \max_{i \in \{1,\dots,k\}} \cos\!\bigl(\phi(w),\, \phi(a_i)\bigr).$$0, and process-level measures based on Reasonableness, Distinctiveness, and Knowledgeability (Huang et al., 30 Oct 2025). This suggests a wider landscape in which DRAT occupies the vocabulary-space end of a continuum that also includes open-ended, multi-hop, and multimodal association reasoning.

Later LLM creativity work strengthens the importance of divergent-association tests while also emphasizing their limits. CreativityNeuro reports that on DAT it improves performance by up to 14 human percentile points and reduces measures of mode collapse, while also improving human-rated originality, surprise, and creativity on AUT and Task Task. Activation steering achieves comparable performance to CreativityNeuro on the DAT, but it does not transfer to the AUT and Task Task (Schapiro et al., 1 Jul 2026). A plausible implication is that DRAT, because it structurally combines divergence with constraint satisfaction, can serve as a more informative target than DAT alone when the downstream concern is scientific ideation rather than vocabulary dispersion per se.

6. Scope, limitations, and interpretation

DRAT is not presented as a universal measure of creativity. It predicts flexibility, clarity, and to a lesser extent originality in scientific ideation, but it does not reliably track feasibility. The paper therefore treats it as a proxy for specific cognitive components of scientific creativity rather than as a full replacement for open-world evaluation (Schapiro et al., 13 May 2026).

Several limitations follow directly from the design. First, performance is substantially better when anchors are scientific terms, which is appropriate for a scientific-ideation test but also means that DRAT is somewhat specialized to scientific concept vocabularies. Second, embedding choice and anchor vocabulary affect scores, so results are not representation-independent. Third, because the test is short-form and vocabulary-based, it does not directly evaluate long-horizon workflows, iterative refinement, or experimental design. Fourth, language and cultural biases in embeddings and anchor lists are not yet fully explored (Schapiro et al., 13 May 2026).

A related misconception is that high performance on divergent-association tasks alone suffices for broad claims about machine creativity. DAT work already cautions against this by emphasizing that divergent semantic association is a fundamental process underlying creativity, not the entirety of creativity itself (Chen et al., 2023). DRAT sharpens that caution: it shows that simultaneous assessment of divergent and convergent thinking in one instrument is essential to reliably predicting scientific ideation ability, but it still leaves open dimensions such as feasibility, execution, and long-form knowledge integration (Schapiro et al., 13 May 2026).

In that sense, DRAT is best understood as a carefully engineered hybrid test. It filters words based on calibrated semantic relevance to multiple anchors, computes semantic spread among the survivors, and thereby measures balanced creativity under multi-anchor constraints. Its reported significance for scientific ideation, together with the failure of DAT, CDAT, PACE, and RAT to achieve the same result in the same study, makes it a specialized but consequential instrument for evaluating scientific creativity in LLMs (Schapiro et al., 13 May 2026).