Show, Don't Tell: Uncovering Implicit Character Portrayal using LLMs (2412.04576v1)

Abstract: Tools for analyzing character portrayal in fiction are valuable for writers and literary scholars in developing and interpreting compelling stories. Existing tools, such as visualization tools for analyzing fictional characters, primarily rely on explicit textual indicators of character attributes. However, portrayal is often implicit, revealed through actions and behaviors rather than explicit statements. We address this gap by leveraging LLMs to uncover implicit character portrayals. We start by generating a dataset for this task with greater cross-topic similarity, lexical diversity, and narrative lengths than existing narrative text corpora such as TinyStories and WritingPrompts. We then introduce LIIPA (LLMs for Inferring Implicit Portrayal for Character Analysis), a framework for prompting LLMs to uncover character portrayals. LIIPA can be configured to use various types of intermediate computation (character attribute word lists, chain-of-thought) to infer how fictional characters are portrayed in the source text. We find that LIIPA outperforms existing approaches, and is more robust to increasing character counts (number of unique persons depicted) due to its ability to utilize full narrative context. Lastly, we investigate the sensitivity of portrayal estimates to character demographics, identifying a fairness-accuracy tradeoff among methods in our LIIPA framework -- a phenomenon familiar within the algorithmic fairness literature. Despite this tradeoff, all LIIPA variants consistently outperform non-LLM baselines in both fairness and accuracy. Our work demonstrates the potential benefits of using LLMs to analyze complex characters and to better understand how implicit portrayal biases may manifest in narrative texts.

• The paper introduces LIIPA, a new framework that extracts implicit character attributes from narratives using full contextual cues.
• The methodology compares three LIIPA variants for classifying character traits along intellect, appearance, and power, with LIIPA-direct excelling in accuracy despite fairness concerns.
• Experimental results on the ImPortPrompts dataset reveal enhanced lexical diversity and robustness, offering promising insights for computational literary analysis.

Analysis of Implicit Character Portrayal Using LLMs

"Show, Don’t Tell: Uncovering Implicit Character Portrayal using LLMs" introduces a novel methodology for extracting implicit character portrayals from narrative texts using LLMs. The primary contribution of the work is the LIIPA (LLMs for Inferring Implicit Portrayal for Character Analysis) framework, designed to address the challenge of deriving character attributes when these are implied rather than directly stated in the text.

Dataset and Framework Design

The authors generate a new benchmark dataset, ImPortPrompts, specifically curated to facilitate the task of implicit character portrayal classification. This dataset outperforms existing corpora, such as TinyStories and WritingPrompts, by featuring greater lexical diversity, broader character role representation, and stronger cross-topic similarity. The paper defines the portrayal task as a multi-label classification problem across three dimensions—intellect, appearance, and power. The labels for these dimensions are classified into "low," "neutral," or "high."

LIIPA's framework presents three variants: LIIPA-direct, LIIPA-story, and LIIPA-sentence, each utilizing different prompting approaches, such as character attribute word lists or chain-of-thought models. A key finding is that LIIPA-direct achieves the highest accuracy due to using the complete narrative context, although it exhibits a fairness concern, highlighting a fairness-accuracy tradeoff—a relevant theme within algorithmic fairness research.

Experimental Findings

The empirical evaluation demonstrates that LLMs, when equipped with the LIIPA framework, outperform the previous COMET-based approaches in character portrayal tasks, achieving superior results in both accuracy and fairness. Notably, while LIIPA-direct maximizes accuracy, it does so at the cost of fairness, showcasing the nuances of leveraging LLMs in literary analysis.

The authors also explore the implications of narrative complexity, revealing that longer stories and greater character counts present challenges. Contextual methods such as LIIPA-story and LIIPA-direct, which utilize the full narrative, show robustness against increasing character diversity but struggle with very lengthy narratives due to information overload.

Implications and Future Directions

The research offers promising enhancements to tools used by writers and scholars aiming to analyze complex fictional characters. The insights from LIIPA could be integrated into visualization tools, enabling a more nuanced understanding of implicit character biases and potentially benefiting literary studies, particularly in analyzing and revising narrative drafts.

Future work could revolve around refining LLM approaches to mitigate identified fairness issues, thereby improving ethical and unbiased character analysis. Additionally, expanding the model's capabilities to include other dimensions of character portrayal, such as emotional depth, could further advance the utility of LLMs for nuanced narrative understanding.

In conclusion, this paper contributes significantly to the field of computational literary analysis, pushing beyond explicit indicators to automate the detection of subtle narrative elements using LLMs, and thus providing avenues for future exploration of implicit biases in both machine-generated and human-written narratives.