- First, the paper identifies how differences in LLM prompting structures affect users' factual recall and subjective interpretation of witnessed events.
- Congruent prompting mechanisms can affect whether users reconstruct events accurately and trust the information internally or generated.
- Users employ varying strategies with LLM-powered tools, leading to differential levels of memory accuracy and interpretation, demonstrating the need for prompt engineering.
Overview and motivation
This CHI '26 paper examines how interaction design of LLM-powered writing assistants shapes both memory recall and subjective interpretation when people document one-time witnessed events. The authors situate the work in eyewitness testimony contexts, where witnesses—typically laypeople unfamiliar with legal reporting conventions—are increasingly likely to turn to LLM tools for help composing statements. Prior work has established that generative chatbots can amplify false memories in witness interviews (Chan et al., 2024) and that AI-edited media can implant false recollections (Pataranutaporn et al., 2024). This study extends that line by asking a more granular question: given the same underlying model, how do different prompting structures affect the mechanisms by which users internalize, trust, and reconstruct what they believe they remember?
The study addresses three research questions: (RQ1) how different LLM prompting designs affect recall of witnessed events; (RQ2) how prompting designs shape post-hoc perceptions of the event; and (RQ3) how users actually employ LLM tools when writing witnessed statements.
Method
Twenty-eight participants recruited from a university-affiliated social network watched a 36-second surveillance video depicting an ambiguous indoor invasion and robbery—an incident deliberately lacking clearly visible weapons or explicit actions to mirror real eyewitness ambiguity. After a 15-minute wakeful rest interval (chosen based on evidence that brief rest enhances retention), participants composed a witness statement using GPT-4 under one of two conditions:
- Natural/default condition (n = 13): an unprompted GPT-4 interface representing unstructured, everyday use.
- Guided/prompted condition (n = 15): GPT-4 configured with a system prompt derived from a standardized law-enforcement eyewitness preliminary investigation protocol. The agent elicited open-ended elaboration with neutral, non-leading follow-up questions (e.g., "What color was the vehicle?" rather than "Was the vehicle red?"), used participants' own wording, prohibited speculation, and only drafted a statement after a mandatory stop rule.
Participants were blind to condition assignment. Following statement composition, a 60-minute consolidation interval preceded a post-task survey: a 50-item forced-choice factual recognition test across five thematic categories, a 14-item Likert-scale interpretation battery (perceived intensity, emotional response, incident clarity, legitimacy judgments toward individuals involved), measures of perceived GPT accuracy and confidence, and semi-structured interviews analyzed via reflexive thematic analysis.
A deliberate design choice warrants emphasis: the study is all-LLM, with no no-GPT control group or human-led interview comparison. The authors justify this on the grounds that prior work already demonstrated LLM-induced memory distortion, so their contribution isolates relative effects between prompting regimes while holding interviewer behavior constant. A pilot no-GPT condition confirmed that recall drifts over time even without intervention, which motivated excluding it from the main study.
Quantitative findings
Factual recall accuracy did not differ significantly between conditions (guided: M = 24.20, SD = 5.17; natural: M = 24.23, SD = 3.19; t(26) = −0.017, p > .05). Structured protocol-based prompting was therefore equivalent to default interaction in supporting objective recall—a null result that frames the more consequential differences below.
The conditions diverged sharply in subjective interpretation and metacognitive calibration:
- Legitimacy judgments: default-condition participants rated the intruder significantly less favorably (M = 8.15, SD = 2.70) than guided-condition participants (M = 10.00, SD = 1.85), t(26) = 2.13, p = .043. Unstructured GPT interaction appears to amplify affective bias against the perceived perpetrator despite identical factual recall.
- Clarity–trust coupling in the default condition: self-perceived incident clarity correlated positively with perceived accuracy of GPT's output, r(12) = .66, p = .014, and negatively with perceived legitimacy of the intruder, r(12) = −.68, p = .011. Participants who felt they understood the event better trusted the model more and judged the intruder more harshly—even though their actual recall was no better than guided-condition peers.
- Calibration in the guided condition: perceived clarity correlated with actual recall performance, r(13) = .55, p = .034. Only under structured, non-leading prompting did subjective confidence track objective memory.
The implication is direct: default GPT interaction decouples metacognitive confidence from memory accuracy while coupling it instead to trust in AI output and moral judgment about event actors. Guided prompting restores confidence–accuracy alignment. For evidence-based domains, this suggests prompt engineering is not merely an interface concern but a mechanism for controlling interpretive distortion.
Qualitative findings: interaction strategies
Thematic analysis of interviews and chat logs identified five statement-composition patterns ranging from submitting fully GPT-written text to using GPT solely for linguistic polish. Within the default condition, participants spontaneously developed three notable strategies:
- Building on partial recollections: iterative co-writing in which GPT's questions scaffolded vague memories into coherent structure—the same pattern the guided condition imposed by design, but self-discovered here.
- "Tell me what I missed": explicitly requesting potentially overlooked details, e.g., asking whether an accomplice acted as a lookout.
- Recall coaching: using GPT meta-levelly for tips on how to recall rather than for content itself.
Two further findings qualify the risk picture. First, participants detected clearly identifiable fabrications—GPT inserted sound effects into a muted video, dramatized tone—and corrected them without lasting influence on factual judgments. Second, however, misinformation adjacent to core content (number of people, clothing colors) produced ambiguity: participants attributed their inability to verify such details to their own fuzzy memory rather than to the model, a pattern the authors link to DRM-style gist-based false memory errors. One participant captured the failure mode precisely: if one starts in a misguided direction, GPT "will let you go further down the road." Perceived authority also mattered: some participants inferred correctness from the agent's failure to contradict them ("it didn't say I was wrong... I just assumed that I was right"), while others restricted GPT to grammar-checking out of distrust grounded in the correct observation that the model had never seen the video.
Limitations and open questions
The authors are explicit about several constraints. The sample is small (N = 28) and drawn from a university-affiliated network, limiting statistical power and generalizability. Only ChatGPT/GPT-4 was tested; other models may behave differently. Time intervals (15 minutes post-viewing, 1 hour pre-survey) were chosen from pilots balancing consolidation against dropout; longer delays, as in classic Loftus & Palmer designs, might reveal different long-term effects. No pre-GPT baseline recognition test was administered because piloting showed the test itself contaminates memory—a trade-off between measurement and interference that leaves the trajectory of memory change during the initial wait unobserved. And because the main study omitted a no-GPT control, the absolute effect of GPT interaction relative to unaided recall cannot be quantified from these data. Finally, the guided prompt enforced neutrality; the authors note that an adversarially prompted agent (e.g., persistently probing the intruder) could steer memory and perception differently, and this manipulation remains untested.
Conclusion
This paper demonstrates that prompting structure—not just model capability—determines whether LLM-assisted witness documentation preserves or distorts the relationship between what users remember and what they believe they remember. Default GPT use inflated trust in generated content among those with high subjective clarity and amplified negative legitimacy judgments, while protocol-guided prompting aligned confidence with actual recall at no cost to accuracy. The qualitative analysis shows users bring diverse, sometimes protective strategies to these tools, but remain vulnerable to gist-consistent misinformation they cannot detect. The results argue for carefully engineered, non-leading interaction protocols in any evidence-based documentation setting—policing, medical charting, journalism—where the distinction between recalled fact and AI-suggested plausibility carries material consequences.
