---
title: Creative Homogeneity in LLMs
url: https://www.emergentmind.com/topics/creative-homogeneity-in-llms
type: topic
---

# Creative Homogeneity in LLMs

Creative homogeneity in large language models (LLMs) denotes the systemic tendency of these models to generate outputs that are semantically, structurally, or stylistically similar across repeated samples, tasks, and even across distinct models. This phenomenon arises from the statistical, architectural, and alignment-centric foundations of current LLMs, and is evidenced by reduced lexical, conceptual, and perspectival diversity relative to human population baselines. Empirical studies confirm that creative homogeneity pervades everything from divergent‐thinking tests and narrative generation to open-ended ideation and social group portrayals, raising concerns for language diversity, cognitive pluralism, fairness, and the future trajectory of both AI-generated and human communication.

## 1. Formal Definitions and Core Metrics

Creative homogeneity is instantiated through several forms—semantic, lexical, structural—that are quantified via clustering, pairwise similarity, entropy, and divergence metrics:

- **Semantic Homogeneity:** Let $S = \{\mathcal{P}_1, \dots, \mathcal{P}_K\}$ be $K$ model-generated texts for a fixed prompt. Embedding vectors $e(\mathcal{P}_k)\in\mathbb{R}^d$ yield mean pairwise cosine similarity:
  \[
  S_\text{intra}(S) = \frac{2}{K(K-1)}\sum_{i<j}\cos\left(e(\mathcal{P}_i), e(\mathcal{P}_j)\right)
  \]
  Higher values manifest greater homogeneity; low diversity is equally evidenced by high self-BLEU and low Distinct-$n$ scores [2501.19361, 2510.22954, 2505.18949].
- **Population Variability:** Measured by embedding-based spread or pairwise sentence similarity among all responses in a population (human or LLM). Wenger & Kenett compute [2501.19361]:
  \[
  \mathcal{V}_t(\mathcal{P}) = \left\{1 - \cos\big(\mathcal{S}(\mathbf{R}_i^p), \mathcal{S}(\mathbf{R}_j^p)\big)\ |\ i \neq j, p\right\}
  \]
  LLM–LLM distances are significantly lower than human–human on divergent-thinking tasks.
- **Entropy & Distributional Divergence:** Shannon entropy $H(P) = -\sum_{i}p_i \log p_i$ and Kullback-Leibler divergence $D_{KL}(P\parallel Q)$ measure collapse toward central distributions, as in content, style, or perspective [2508.01491, 2509.16794].
- **Structural Homogeneity:** Narrative functions $f_i$ in LLM fiction are distributed with sharply peaked histograms and high BERTScore among outputs, compared to more dispersed human texts [2603.14430].

In multi-model or multi-sample contexts, inter-model and intra-model similarities are also central: S_intra (within model), S_inter (between models) [2510.22954].

## 2. Empirical Manifestations and Scope

Creative homogeneity is robustly observed across models, tasks, and evaluation settings:

- **Population-Level Collapse:** On divergent-thinking tests (Alternative Uses, Forward Flow, Divergent Association), LLM outputs are significantly more similar to each other (AUT: $\mathcal{V}_\text{AUT}^\text{LLM}=0.459$, $\mathcal{V}_\text{AUT}^\text{human}=0.738$) despite comparable individual originality scores [2501.19361].
- **Instruction and Chat Tuning:** Fully structured prompts (system/user/assistant tokens) anchor models to restrictive output spaces; “diversity collapse” persists even at high temperature, and removing chat tokens increases output diversity by 2x–3x [2505.18949].
- **Narrative Homogenization:** In both English and Chinese story generation, LLMs exhibit fixed, repetitive plot paradigms with low narrative‐function entropy ($H(P_m)$) and high Jensen-Shannon divergence from human distributions. Even high-capacity models like Deepseek and Qwen default to “hero-struggle-victory” templates [2603.14430].
- **Open-Ended Ideation:** On the Infinity-Chat corpus, 79% of open-ended prompts yield S_intra > 0.8; human population variability dwarfs that of both intra- and inter-model LLM generations [2510.22954].
- **Cross-Model Agreement:** LLMs used as creative judges agree with each other at Spearman $\rho$ > 0.7 and even match expert-constructed oracles ($\rho\sim0.95$), reinforcing output homogeneity in both creation and assessment [2411.15560].

## 3. Mechanistic Sources and Theoretical Frameworks

Multiple, orthogonal mechanisms underlie creative homogeneity:

- **Statistical Centrality and Averaging:** Next-token prediction implicitly rewards frequent, high-probability continuations, pulling outputs toward “average” rhetorical, stylistic, or conceptual modes (“kitsch” metaphor) [2509.16794]. 
- **Format-Induced Collapse:** Instruction-finetuning on chat-style (multi-turn) templates instills fixed generation priors, rapidly restricting output variability after just a few decoding steps [2505.18949].
- **Alignment and Uncertainty Minimization:** Alignment via RLHF or SFT steers outputs toward low-uncertainty, “safe” continuations, suppressing constructive ambiguity and leading to bland, risk-free narratives [2602.16162].
- **Cognitive Process Compression:** Without explicit divergent–convergent phase separation, LLMs conflate idea generation with immediate constraint satisfaction, pruning uncommon ideas and drift toward a hive-mind [2512.23601].
- **Training Data and Social Bias:** Differential coverage and stereotype alignment in corpora cause subordinate social groups to be depicted with markedly higher narrative homogeneity, mirroring the human out-group homogeneity effect [2401.08495].
- **Reward Models and Model Judges:** LM judges and reward models, trained to replicate aggregate “quality,” demonstrate strong cross-model calibration but fail to account for the pluralism apparent in human preferences [2510.22954, 2411.15560].

## 4. Social and Cultural Implications

The homogenizing tendencies of LLMs propagate beyond algorithmic evaluations into broader social and epistemic contexts:

- **Linguistic and Cultural Flattening:** Lexical, syntactic, and stylistic diversity in LLM-assisted texts is systematically lower (TTR: −15–20%, Shannon entropy suppressed by comparable margins) [2508.01491, 2509.16794].
- **Perspective and Reasoning Narrowing:** LLMs collapse perspectival spread (Jaccard similarity: 0.75 LLM vs 0.45 human), erasing rare beliefs or reasoning strategies [2508.01491].
- **Reinforcement Loops:** Recursive use of LLM-generated content in communications and knowledge bases turns statistically dominant patterns into cultural norms, amplifying convergence and diminishing pluralism over time [2508.01491].
- **Creative Erasure and Fairness:** In domains such as narrative generation about socially subordinate groups, homogeneity compounds social bias, narrowing both individuating detail and trait diversity [2401.08495].
- **Educational and Ideational Impact:** Homogeneous LLM-generated educational materials propagate redundant or repetitive problem structures, impinging on divergent thinking and novelty for learners [2512.23601].

## 5. Mitigation Strategies and Interventions

Approaches for ameliorating creative homogeneity span data, model, and user-facing solutions:

- **Prompt and Decoding Engineering:**
  - Structure-free or “natural instruction” prompts maximize output diversity [2505.18949].
  - Decoupling divergent and convergent phases at inference (e.g., CreativeDC) yields higher Vendi, lexical and semantic diversity, and effective distinct problem counts [2512.23601].
  - Higher temperatures and nucleus sampling have only limited impact unless format priors are also weakened [2505.18949, 2510.22954].
- **Architectural and Objective Modifications:**
  - Training with diversity-promoting objectives or multi-persona/debate-driven architectures [2508.01491].
  - Fine-tuning with anti-causal and structural-contrastive losses; explicit narrative-function supervision for fiction domains [2603.14430].
  - Uncertainty-aware alignment (ambiguity-permissive reward models, credal-set planning) to re-inject productive unpredictability [2602.16162].
- **Data and Evaluation Practices:**
  - Augmenting training data with culturally and narratively diverse exemplars, especially for underrepresented groups or non-Western perspectives [2401.08495, 2508.01491].
  - Automated and human-in-the-loop diversity audits, cross-prompt and cross-group regularization [2401.08495, 2509.16794].
  - Human-in-the-loop interventions and transparency tools that expose model-driven linguistic convergence [2508.01491].
- **Critical and Regulatory Approaches:** 
  - “Kitsch studies” and critical rhetoric frameworks to educate users and guide regulatory audits for style and content diversity [2509.16794].
  - Diversity-focused curriculum and alignment testing within foundation model development [2510.22954].

## 6. Persistent Challenges and Open Problems

Despite methodological innovations and some gains, significant obstacles remain:

- **Superficial vs. Deep Diversity:** Many interventions (e.g., temperature scaling, anti-kitsch prompting) boost superficial variation without restoring deep cultural or reasoning heterogeneity [2508.01491].
- **Scalability of Diversity Curation:** Ensuring pluralistic representations at web scale requires scalable data collection and labeling solutions.
- **Evaluation in Unscored Domains:** Formal measurement of creativity and diversity lags in poetry, humor, argumentation, and long-form storytelling. LM-based judges, though consistent, cannot presently capture the pluralism observable in human populations [2510.22954, 2411.15560].
- **Societal Feedback Loops:** As LLMs proliferate as both content generators and evaluators, their structural bias towards consensus and similarity risks accelerating epistemic and cultural convergence beyond language modeling per se [2508.01491, 2510.22954].

## 7. Summary Table: Major Sources, Metrics, and Empirical Findings

| Paper (arXiv ID)     | Core Metric(s)                | Primary Homogeneity Finding                                      |
|:---------------------|:-----------------------------|:-----------------------------------------------------------------|
| 2501.19361           | $\mathcal{V}_t$ (embed dist) | LLM–LLM similarity $\gg$ Human–Human, despite equal originality |
| 2510.22954           | S_intra, S_inter (embed sim)  | Intra- and inter-model S > 0.8 on 50+% of open-ended tasks      |
| 2505.18949           | D_sem, D_topic, self-BLEU     | Format induces 2–3x collapse in diversity, persists at T=1.0    |
| 2602.16162           | PPL, NLL, PMI, CPMI          | LLM uncertainty 2–8x lower than humans, correlates to triteness |
| 2512.23601           | SemDiv, LexDiv, Vendi Score   | Two-phase prompts (CreativeDC) increase effective distinctivity  |
| 2401.08495           | Mean standardized embedding   | Subordinate groups are more homogeneous (race, gender effects)   |
| 2603.14430           | JSD, BERTScore, Entropy       | Narrative function collapse, fixed high-frequency plot templates |
| 2508.01491           | TTR, entropy, Jaccard, KL     | Lexical, syntactic, and reasoning diversity all shrink in LLMs   |
| 2411.15560           | Spearman $\rho$ (AUT judge)   | LLM–LLM $\rho$ > 0.7 for creativity evaluation                  |

The creative homogeneity of LLMs is a multidimensional, system-level phenomenon resulting from the statistical, algorithmic, and socio-cultural processes governing current models and their deployment. It is grounded in robust empirical findings, has measurable expression across creative and non-creative tasks, and presents a suite of technical and societal challenges that require algorithmic, evaluative, and cultural mitigation strategies [2501.19361, 2508.01491, 2602.16162, 2603.14430, 2510.22954, 2505.18949, 2411.15560, 2401.08495, 2512.23601, 2509.16794].

Source: https://www.emergentmind.com/topics/creative-homogeneity-in-llms