- The paper presents a detailed stylometric analysis comparing GPT-generated SOTU speeches with those of recent U.S. Presidents.
- It examines linguistic features like vocabulary richness, lemma frequency, and sentence complexity using statistical measures.
- The findings reveal that GPT speeches use noun-heavy structures and a neutral, optimistic tone, diverging from human rhetorical nuance.
Stylometric Analysis of GPT-Generated Political Speeches
In the paper titled "GPT as Ghostwriter at the White House," the author Jacques Savoy presents a detailed stylometric analysis comparing the State of the Union (SOTU) addresses generated by ChatGPT-3.5 with those delivered by four recent U.S. Presidents: Reagan, Clinton, Bush, and Obama. The research offers a critical examination of the stylistic distinctions and similarities between authentic presidential speeches and those produced by a LLM.
The investigation provides substantial insights into the linguistic choices and rhetorical positions adopted by GPT, contrasting these with human-composed speeches across several dimensions. This is crucial for understanding the potential and limitations of LLMs in tasks traditionally associated with human creativity and political communication.
Methodology and Findings
To generate the corpus, the researchers employed ChatGPT-3.5 to produce SOTU addresses for each target president, guiding the model with non-SOTU samples of their speech extracted from publicly available archives. The analysis covers several linguistic features, including lemma frequency, part-of-speech distribution, vocabulary richness, and sentence complexity.
A key finding of the paper is that GPT-generated speeches manifest distinctive stylistic markers, such as a higher frequency of the pronoun "we," longer word and sentence lengths, and an elevated usage of nouns and adjectives over verbs. These characteristics result in speeches with a more didactic and neutral tone, divergent from the stylistically richer and contextually embedded language of human-authored SOTUs.
Table 5 in the paper quantifies these differences, highlighting GPT’s reliance on noun-heavy sentence structures that often lack the temporal and spatial anchoring present in authentic political discourse. This tendency is further corroborated by the repeated statistical significance found in differences between GPT and presidential documents in term frequency and phraseology.
Rhetorical Analysis
The paper explores the rhetorical tone projected by GPT, noting a consistent inclination towards positive expressions and a clear aversion to negativity or blame, as delineated in Table 7. This approach imbues GPT's output with elements of optimism and patriotism, aligning with its artificial objective of nonpartisan communication.
However, while GPT successfully mimics some rhetorical elements of presidential discourse, its attempts at imitation fall short in terms of emotional engagement and specificity, as it tends to avoid divisive issues or detailed examples crucial for effective political rhetoric.
Intertextual Distance and Stylometric Implications
Significant attention is given to intertextual distance—a metric used to quantify stylistic proximity—demonstrating a clear divergence between machine-generated texts and those crafted by presidential speechwriters. Figure 1 conveys this separation visually, with GPT-generated content clustering distinctly from authentic presidential addresses.
Despite the apparent stylistic gaps, attempts to imitate specific presidential styles reveal the model’s ability to shift slightly in tone and vocabulary when directed, although it still favors a generalized style characteristic of LLM outputs.
Future Implications
The implications of Savoy's findings for LLM development are considerable. The paper sets a foundational understanding of the capabilities and current limitations of AI in generating politically themed texts, pointing towards areas for enhancement, such as incorporating more personal pronouns and contextual anchors to reduce the perceptual difference from authentic speeches.
As LLMs like GPT continue to evolve, ongoing research could address their capacity to generate content that fully integrates the complex, human-like stylistic and rhetorical nuances characteristic of effective political discourse. This paper lays groundwork for future exploration in refining AI's ability to emulate human authors convincingly, particularly in more stylistically and contextually demanding applications.