AI as a Tool for Simulation-Based Experiments in Literary Studies

Abstract: Generative AI systems open new possibilities for experimentation in literary studies via controlled, grounded, large-scale, low-cost simulations of cultural production. Current systems have not yet been shown to produce high-quality, book-length narrative texts that reliably reflect arbitrarily specified cultural constraints or stylistic features. But there exists substantial relevant research on each of the components required for literary-historical simulation. These include the use and validation of AI systems as proxies for differentiable human populations; the narrative and stylistic properties of AI-generated texts; the stability and coherence of multiagent, multiturn AI simulations of human actors; and technical methods through which to alter in predictable ways the knowledge and behavior of generative systems. Together, these areas could provide a starting point for more ambitious AI-based modeling of cultural systems of literary production. We describe the possibilities and challenges of simulation-based experiments in literary studies, summarize the current state of the art in relevant fields, and explain key technical aspects of the work. To provide an example directly relevant to literary scholars, we present the results of experiments on literary text generation, including comparisons to high-status, human-authored novels. Our results include the first demonstration of (limited) in-distribution outputs by AI models in this domain. We conclude with a description of future work on full counterfactual literary-historical simulations using AI.

• The paper demonstrates that engineered prompts enable AI to generate counterfactual literary corpora closely mimicking high-status human texts.
• It employs methods like high-dimensional embedding analysis and Fightin’ Words to validate genre structures and lexical distinctions.
• Findings indicate that prompt complexity, rather than model architecture, is crucial for achieving authentic simulation fidelity in literary outputs.

AI-Driven Simulation-Based Experiments in Literary Studies

Introduction

This paper establishes a comprehensive methodological and empirical framework for the use of generative AI in simulation-based experiments for literary studies, specifically leveraging LLMs as proxies for the cultural processes that shape literary production. Unlike traditional literary history, which is hampered by the inability to experimentally manipulate historical conditions, this approach enables the systematic generation of counterfactual literary corpora. The paper synthesizes relevant advances in model-based simulation, text generation, model editing/unlearning, and validation, culminating in large-scale experiments evaluating the plausibility and controllability of AI-generated literature relative to high-status human-authored texts.

Methodology and Experimental Setup

The experiments utilize a substantial reference corpus—CONLIT—composed of contemporary English-language novels and narrative nonfiction, enriched with granular metadata (e.g., genre, year, author demographics). Generation pipelines employ OpenAI GPT-5 under two main prompting conditions: complex prompts—incorporating detailed synthetic biographies and historical or genre instructions—and basic prompts with minimal conditioning. Evaluation of generated and reference texts is grounded in high-dimensional document embeddings (Qwen3-Embedding-8B), facilitating both unsupervised clustering and pointwise similarity analysis. Lexical divergence is interrogated via the Fightin' Words method, sensitive to the most distinctive tokens between corpora.

Embedding Space and Genre Structure

The dimensionality-reduced embedding analysis reveals that AI-generated fiction, when robustly prompted, occupies similar regions to human-authored genre texts and preserves the hierarchical structure of reception genres (Figure 1).

Figure 1: Human-authored narratives by genre visualized in embedding-reduced space, validating genre separability in human texts.

Complex-prompted AI texts, particularly in high-status genres, are more proximate in embedding space to human counterparts than to off-target genres, and the rank ordering by reception status is largely preserved, mirroring the structural dynamics of contemporary Anglophone literature. Intriguingly, internally, AI-generated genre clusters demonstrate greater diversity (lower within-genre similarity) than human-authored counterparts. This finding subverts the narrative of LLM output homogeneity documented in prior work [xuEchoesAIQuantifying2025, zhangNoveltyBenchEvaluatingLanguage2025].

Prompt Complexity and Simulation Fidelity

The experimental matrix demonstrates that simulation fidelity—how closely AI outputs resemble human-authored texts—depends critically on prompt design. Complex prompting, leveraging synthetic biographies and explicit task constraints, reduces the over-clustering characteristic of basic instructional prompts and shifts the output distribution toward higher correspondence with human literary texts (Figure 2).

Figure 2: Comparison of human-authored texts in selected genres to AI generations under complex and basic prompting strategies; complex prompts yield outputs closer to human distributions.

Basic prompts produce hyper-homogeneous, centrally clustered outputs, whereas complex prompts yield broader, more human-like dispersion and increased embedding proximity to human genre exemplars. This result foregrounds prompt design—not model architecture per se—as the dominant variable for simulation-based studies.

Temporal and Counterfactual Conditioning

Temporal conditioning (year directives in prompts) fails to induce significant diachronic variation in text embeddings. The null result implies that current LLMs, even with precise prompt-level instructions, cannot reliably mimic distributional shifts in literary production over short timescales. This is attributed to both the slow secular progression in real-world literary style and inherent model limitations in temporal steerability, corroborating findings by Underwood et al. [underwoodCanLanguageModels2025].

The implication is that authentic counterfactual modeling—such as simulating the narrative field in the absence of a particular author or movement—demands intervention at the level of training regimen (domain-specific pretraining), targeted model editing (factual and stylistic association alteration [mengLocatingEditingFactual2023, mengMassEditingMemoryTransformer2023]), or unlearning techniques [eldanWhosHarryPotter2023, mainiTOFUTaskFictitious2024], each with substantial technical friction.

Lexical Distinctiveness and Narrative Register

Lexical analysis shows that the most distinctive features distinguishing AI and human fiction remain subtle and primarily manifest as differences in pronoun usage, narrative tense, and function words (Figure 3):

Figure 3: Fightin' Words analysis of the most distinctive words in human- and AI-authored PW fiction, highlighting register, perspective, and tense divergences.

Human prizewinning fiction preferentially employs third-person pronouns and past tense, while AI generations—despite sophisticated prompting—show a present-tense bias, higher incidence of logical connectives, metafictional tokens, and first-person plural register. These shallow register mismatches align with prior studies on narratological limitations in LLM-generated texts [walshDoesChatGPTHave2024, hicke2025zerobodyproblemprobing].

Implications for Simulation-Based Literary History

This research affirms the viability—but not yet the comprehensiveness—of LLM-driven literary simulation as an analytic tool. While genre structure and high-level reception categories are recoverable, some aspects of deep narrative form, diachronic drift, and cultural/historical fidelity remain out of reach—pinpointing the limitation to the most recent, well-represented Anglophone traditions. The findings demonstrate that simulation-based approaches can serve as a principal means of interrogating counterfactuals, particularly for aggregate or distributional questions (e.g., the central tendency of a genre, the impact of specific contextual cues), but are less reliable for capturing extreme or innovative outlier phenomena without advances in output steerability.

The outlined research program directly informs the construction of counterfactual experiments, such as withholding authorial influence, rerunning historical junctures, and imposing controlled cultural contexts. However, realization of robust historical/cultural simulation will require solutions for model editing, period-locked pretraining [IntroducingTalkie13B, dgoettlichHistoryllmsRanke4b], cultural knowledge enrichment [veselovskyLocalizedCulturalKnowledge2025], and new validation metrics transcending embedding-level similarity [hamilton-etal-2026-narrabench, russellStoryScopeInvestigatingIdiosyncrasies2026].

Limitations and Future Directions

Key limitations include: the restriction to first chapters rather than entire long-form works, exclusive reliance on a single SOTA LLM, and embedding-based analysis that, while sensitive to topic and surface form, cannot probe structural narrative quality or interpretive complexity. Additionally, generate-evaluate paradigms here are not yet equipped to inform true causal inference, though the groundwork for such studies is established [federCausalInferenceNatural2022a, keithTextCausalInference2020].

Future directions demand systematic evaluation across models, full-length generation, more granular causal intervention in training/editing, and new evaluative frameworks that better capture narratological and cultural nuance—particularly outside the contemporary Anglophone mainstream.

Conclusion

This work systematically demonstrates that LLMs, under methodologically controlled prompt engineering, yield synthetic outputs that match aggregate distributional properties of human literary production in genre structure and diversity, though with persistent gaps in stylistic register and historical fidelity. The major advances required for the field are technical improvements in counterfactual conditioning, cultural localization, and deep narrative validation. The simulation-based paradigm offers literary scholars and computational researchers a rigorous, interpretable scaffold for engaging with counterfactual inference in literary history, opening a pathway for experimental literary scholarship and computational cultural studies at scale.