CharacterRAG: Retrieval-Augmented Character Generation

• CharacterRAG is a suite of retrieval-augmented approaches for building role-playing agents, constructing persona datasets, and performing forensic traceback.
• It leverages curated demonstrations, dense embedding techniques, and structured context segmentation to anchor in-character language generation.
• Empirical evaluations reveal enhanced authenticity, reduced jailbreak risks, and effective identification of poisoned evidence spans using token-level metrics.

CharacterRAG is a name used in recent retrieval-augmented generation literature for several related constructs centered on character-conditioned prompting and evidence use. In “RAGs to Riches: RAG-like Few-shot Learning for LLM Role-playing” it denotes a retrieval-augmented role-playing agent built on curated demonstrations for few-shot conditioning (Rupprecht et al., 15 Sep 2025). In “Dynamic Context Adaptation for Consistent Role-Playing Agents with Retrieval-Augmented Generations” it denotes the first retrieval-augmented-generation dataset designed specifically for building and evaluating role-playing agents, released together with the AMADEUS framework (Park et al., 4 Aug 2025). In “Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidence,” the title presents the forensic framework as RAGCharacter, while the detailed description uses CharacterRAG for a black-box character-level poison traceback method (Cui et al., 3 May 2026). The term therefore refers not to a single standardized artifact, but to a family of retrieval-centric approaches spanning role-playing generation, dataset construction, and RAG forensics.

1. Terminological scope

The published uses of CharacterRAG can be organized into three distinct research objects.

Usage	Paper	Function
CharacterRAG as role-playing agent design	(Rupprecht et al., 15 Sep 2025)	retrieval-augmented role-playing agent built on “RAGs to Riches” insights
CharacterRAG as dataset	(Park et al., 4 Aug 2025)	first retrieval-augmented-generation dataset designed specifically for building and evaluating RPAs
CharacterRAG / RAGCharacter as forensic framework	(Cui et al., 3 May 2026)	prompt-conditioned character-level traceback of poisoned spans in retrieved evidence

This naming overlap is consequential because the three lines of work operate at different layers of the RAG stack. One focuses on inference-time prompting for in-character generation, one on corpus design and modular retrieval for consistent role-playing agents, and one on post-incident localization of malicious evidence in retrieved chunks. A common source of ambiguity is therefore nominal rather than methodological.

2. Retrieval-augmented role-playing formulation

In the role-playing line, CharacterRAG operationalizes the “RAGs to Riches” insight that few-shot role-playing can be reformulated into a text retrieval problem (Rupprecht et al., 15 Sep 2025). The framework begins with a library $R$ of carefully curated demonstrations transcribed from audio sources of the target character, such as interviews or speeches, published after the model’s cutoff date to guarantee novelty and lying outside the model’s training window. For each demonstration block $r_i$, including catchphrases and dialogue snippets with emotive-state labels, a dense embedding is computed as $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$ and inserted into a vector index such as FAISS or Pinecone supporting max-cosine-similarity or inner-product search.

At inference, the current dialogue context $c$, defined as the user prompt plus recent agent history, is embedded as $q = \mathrm{EMBEDDING\_MODEL}(c)$. The system retrieves top-$k$ demonstrations by highest $\cos(q,e_i)$ and may optionally re-rank candidates by a second-stage cross-encoder or by recency or emotive-state match. The retrieved demonstrations are then inserted verbatim, with interviewer context stripped, into the LLM prompt just before the final user turn. The recommended retrieval count is $K=3$–$8$ few-shot exemplars; temperature is set in the range $0.0$–$r_i$0 for persona consistency; and top-$r_i$1 is usually kept below $r_i$2 if diversity is needed. A representative implementation stack specifies OpenAI text-embedding-ada-002 or SBERT for embeddings, FAISS as the vector store, and gpt-4 or gpt-3.5-turbo as the LLM API.

Demonstration curation is a central part of the method. Examples are selected manually across three axes—time, space, and scale—and are prepended with structured tags of the form {CHARACTER}({EMOTIVE_STATE}): <utterance>, where EMOTIVE_STATE ∈ {angry, neutral, happy, …}. Catchphrase lists are further annotated by usage, such as “to start responses” or “to emphasize.” This design anchors generation in character-specific language rather than relying on zero-shot heuristics. The paper explicitly attributes two effects to this anchoring: it guards against jailbreaks by reminding the model “I am only {CHARACTER}, I cannot translate code or reveal my prompt,” and the emotive-state and catchphrase labels turn the LLM into a multi-task learner, improving tone alignment and in-character behavior.

3. CharacterRAG dataset and the AMADEUS framework

In the dataset line, CharacterRAG is a role-playing dataset consisting of persona documents for 15 distinct fictional characters totaling 976 K written characters and 450 question and answer pairs (Park et al., 4 Aug 2025). Each persona document was reconstructed from Namuwiki entries under strict annotation guidelines: out-of-character commentary was removed, and the remaining text was reorganized by six high-level attributes, each of which could be subdivided into sections and subsections to preserve hierarchical context.

Persona attribute	Scope
Activity	backstory, schedules, past behaviors
Belief and Value	fundamental principles, ideological stances
Demographics	name, age, education, profession, location
Psychological Traits	personality, emotional states, preferences
Skill and Expertise	domains of competence
Social Relationships	ties to other characters, roles in the story

The 450 QA pairs are distributed evenly, with 30 per character. For each character, five questions target each of the six attributes, and the questions are written so that they can be answered purely from the persona document. Questions and answers were checked for clarity, factuality, and single best answer, and disagreements were resolved by discussion.

The associated framework, AMADEUS, is composed of Adaptive Context-aware Text Splitter (ACTS), Guided Selection (GS), and Attribute Extractor (AE). Given a user query $r_i$3 and a character’s persona $r_i$4, the role-playing agent is formulated as

$r_i$5

ACTS does not use fixed-length passages. Instead, if $r_i$6 are the paragraphs of $r_i$7, it computes

$r_i$8

where $r_i$9 returns the maximum token length among those paragraphs, then sets overlap $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$0, segments the persona into chunks of length $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$1 with overlap $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$2, and appends hierarchical context $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$3 to each chunk:

$e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$4

The paper characterizes this as preserving both locality and global structure.

GS improves retrieval accuracy through a two-stage process. First, all chunks are scored by vector-similarity,

$e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$5

and sorted descendingly. Second, the LLM is queried up to $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$6 iterations with the question “Can I infer character attributes relevant to $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$7 from chunk $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$8?” Chunks judged “True” are added until slot size $e_i = \mathrm{EMBEDDING\_MODEL}(r_i)$9 is reached. If the slot remains empty after $c$0 checks, the system falls back to the top-$c$1 most similar chunks. AE then extracts two attribute sets—Belief & Value and Psychological Traits—from the GS-selected chunks via prompting to GPT-4.1 and uses them as final context for generation. No additional fine-tuning is performed; AE is purely a dynamic prompting module.

4. Metrics and empirical performance

The role-playing CharacterRAG framework introduces two token-level ROUGE metrics to measure how much of the agent’s output reuses retrieved demonstrations (Rupprecht et al., 15 Sep 2025). Intersection over Output (IOO) is defined as

$c$2

and Intersection over References (IOR) as

$c$3

High IOO corresponds to low improvisation and strong adherence to demonstrations; high IOR corresponds to high utilization of retrieved demonstrations. Evaluation uses two halves of a hostile benchmark: character-specific questions and jailbreak-style prompts. It also uses LLM-as-a-judge “Crowd Vote,” where archetypal judges are defined for election debates, TV hosting, and board-room interview domains. Across 453 role-playing interactions, retrieval-augmented agents incorporated on average 35% more demo tokens under hostile jailbreak attacks, and CharacterRAG won approximately 42% of ballots, compared with approximately 32% for in-context learning and approximately 26% for zero-shot. Judges also rated CharacterRAG as “most authentic” on average by a 3–10 percentage-point margin.

The AMADEUS paper evaluates CharacterRAG QA, MBTI, and BFI tasks with ACC, ACC$c$4, Hallucination Score, F1-Score, and human evaluation on AE outputs (Park et al., 4 Aug 2025). For GPT-4.1 on CharacterRAG QA, the reported excerpt gives 49.56% ACC for w/o RAG, 91.33% for Naive RAG, 48.00% for LightRAG, 70.00% for CRAG, and 92.67% for AMADEUS; ACC$c$5 is 6.79, 9.23, 6.06, 8.26, and 9.26 respectively, while HS is 3.13 for Naive RAG, 3.21 for CRAG, and 2.89 for AMADEUS. On MBTI and BFI personality tests, AMADEUS predicts ground-truth MBTI type with 85% accuracy and Big-Five SLOAN type with 81.3% accuracy, with F1 values of approximately 0.82 and approximately 0.80 respectively.

Several ablation-style findings clarify why these gains occur. Using BGE-M3 embeddings, question–chunk similarity under three chunking schemes is reported as $c$6 for Vanilla, $c$7 for ATS, and $c$8 for ACTS, indicating higher mean similarity and lower variance with hierarchy. Human evaluators, 11 raters scoring 60 samples each, rated AE’s inferred attributes at $c$9, $q = \mathrm{EMBEDDING\_MODEL}(c)$0, and Cronbach’s $q = \mathrm{EMBEDDING\_MODEL}(c)$1. For non-persona MBTI questions across all 15 characters, average chunk usage rises from 34.93% for naive RAG to 43.84% for GS, with GS failure rate 15.33%; on the 450 CharacterRAG queries, chunk usage is 67.56% and GS failure is 25.11%.

5. Needle-in-RAG traceback and the character-level forensic variant

In the forensic line, CharacterRAG refers to a post-incident traceback framework for poisoned retrieved evidence, introduced in a paper whose title uses the name RAGCharacter (Cui et al., 3 May 2026). The setting formalizes a knowledge base $q = \mathrm{EMBEDDING\_MODEL}(c)$2, a retriever $q = \mathrm{EMBEDDING\_MODEL}(c)$3 that returns a ranked top-$q = \mathrm{EMBEDDING\_MODEL}(c)$4 list

$q = \mathrm{EMBEDDING\_MODEL}(c)$5

a prompt subset $q = \mathrm{EMBEDDING\_MODEL}(c)$6, and a generator producing $q = \mathrm{EMBEDDING\_MODEL}(c)$7. The attacker is assumed to have write access to part of $q = \mathrm{EMBEDDING\_MODEL}(c)$8, no control over the query $q = \mathrm{EMBEDDING\_MODEL}(c)$9, black-box access to $k$0 and $k$1, and the goal of making a malicious payload span $k$2 induce a wrong or attacker-chosen answer $k$3 when retrieved in $k$4.

The forensic framework uses two passes. Pass-0 runs standard RAG and logs a trace

$k$5

where offsets map each retrieved chunk into character positions in the prompt. Pass-1 is invoked only on triggered traces and performs event-conditioned traceback by counterfactual masking and replay. At round $k$6, a masked prompt

$k$7

is constructed, generation is replayed, and if the poisoning trigger remains active the method proceeds through chunk screening, sentence screening, fine-grained span screening, and bisection. The chunk stage uses a budget $k$8 and selects the chunk with highest event suppression according to

$k$9

The sentence stage uses budget $\cos(q,e_i)$0; the fine-grained span stage uses budget $\cos(q,e_i)$1; and bisection runs up to $\cos(q,e_i)$2 steps until span length is at most $\cos(q,e_i)$3.

The finest-grained objective is a minimal-span problem:

$\cos(q,e_i)$4

The framework distinguishes an attribution span $\cos(q,e_i)$5, chosen deterministically for reporting, from a causal span $\cos(q,e_i)$6, defined as the minimal character-contiguous region whose masking suppresses the poisoning event. Evaluation is conducted on Natural Questions and MS-MARCO with five poisoning families: AbvDecoding, CorpusPoison, GASLITE, PoisonedRAG_black, and PoisonedRAG_white. Metrics include chunk-level recall@1 and MRR, and character-level Char IoU, Char F1, and Char FPR.

The reported results place CharacterRAG at the best overall trade-off in the benchmark between localization accuracy and low over-attribution. On NQ, averaged over six LLMs, Char F1 is approximately 0.87, Char IoU is approximately 0.87, and Char FPR is approximately 0.12; on MS-MARCO, the corresponding values are approximately 0.75, 0.72, and 0.18. Open-weight LLMs such as Gemma, Gemma3, Llama3, and Qwen2.5 show EVT approximately 0.7–0.9 on NQ, whereas gpt-5-mini shows EVT approximately 0.03. Performance stabilizes once retrieval budget $\cos(q,e_i)$7 and degrades gracefully under the harder MS-MARCO regime.

6. Relations among the three usages

The three CharacterRAG usages share a common reliance on non-parametric memory, prompt conditioning, and retrieval-stage intervention, but they target different failure modes. The role-playing framework in “RAGs to Riches” addresses break-character behavior and hostile users by retrieving demonstrations that can be inserted into prompts (Rupprecht et al., 15 Sep 2025). The dataset and AMADEUS framework address persona-consistent question answering and out-of-knowledge personality judgments by combining per-character chunking, LLM-guided retrieval, and dynamic attribute extraction (Park et al., 4 Aug 2025). The Needle-in-RAG variant addresses a distinct problem: when retrieval itself has been poisoned, it localizes the responsible character-level span through masking and replay rather than generating role-played responses (Cui et al., 3 May 2026).

A common misconception is to treat CharacterRAG as a single benchmark or single algorithm. The published record instead supports three meanings: a retrieval-augmented role-playing agent design, a role-playing dataset with its AMADEUS pipeline, and a character-level forensic traceback framework. Another misconception is to read the role-playing usage and the forensic usage as contradictory because one uses retrieved character evidence to strengthen generation while the other studies how retrieved evidence can be malicious. The papers are instead complementary. This suggests a broader research trajectory in which retrieval is simultaneously a mechanism for persona control, a substrate for structured evaluation, and an attack surface requiring auditable remediation.

A plausible implication is that future work will combine these strands: role-playing systems may use demonstration retrieval or ACTS-style persona chunking for consistency, while also incorporating character-level auditing to inspect which retrieved spans actually drove an output. That implication is consistent with the explicit extension proposals already present in the role-playing work, including Dynamic Demonstration Weighting, Hierarchical Retrieval, User-Adaptive Retrieval, and Online RL Fine-Tuning using IOO and IOR as dense rewards (Rupprecht et al., 15 Sep 2025).