Validity of think-aloud and interview verbalizations for internal-state inference

Ascertain whether participants’ verbal reports in think-aloud and interview protocols accurately reflect their internal cognitive states during the use of LLM-powered writing systems such as GhostWriter.

Background

The paper relied on think-aloud and interview protocols to gather qualitative insights about user experience and perceptions. Such methods can introduce uncertainty about the fidelity of verbalizations as proxies for internal cognitive states.

The authors explicitly acknowledge this limitation, noting uncertainty about whether participants’ quotes truly reflect internal states, thereby highlighting an unresolved methodological question relevant to interpreting findings in this and similar HCI studies.