AI Through the Human Lens: Investigating Cognitive Theories in Machine Psychology (2506.18156v1)

Abstract: We investigate whether LLMs exhibit human-like cognitive patterns under four established frameworks from psychology: Thematic Apperception Test (TAT), Framing Bias, Moral Foundations Theory (MFT), and Cognitive Dissonance. We evaluated several proprietary and open-source models using structured prompts and automated scoring. Our findings reveal that these models often produce coherent narratives, show susceptibility to positive framing, exhibit moral judgments aligned with Liberty/Oppression concerns, and demonstrate self-contradictions tempered by extensive rationalization. Such behaviors mirror human cognitive tendencies yet are shaped by their training data and alignment methods. We discuss the implications for AI transparency, ethical deployment, and future work that bridges cognitive psychology and AI safety

• The paper identifies that LLMs mimic human cognitive patterns using frameworks like TAT, framing bias, MFT, and cognitive dissonance.
• It employs systematic testing across models such as GPT-4 and LLaMA, using measures like SCORS-G to evaluate narrative and bias responses.
• The findings emphasize ethical implications and call for improved AI transparency, balanced alignment methods, and bias mitigation strategies.

AI Through the Human Lens: Investigating Cognitive Theories in Machine Psychology

The paper "AI Through the Human Lens: Investigating Cognitive Theories in Machine Psychology" (2506.18156) presents an exploration of whether LLMs exhibit human-like cognitive patterns under four psychological frameworks: Thematic Apperception Test (TAT), Framing Bias, Moral Foundations Theory (MFT), and Cognitive Dissonance. This research analyzes the behavioral alignment of LLMs with human cognitive processes, focusing on their potential implications for AI transparency and ethical deployment.

Understanding Cognitive Patterns in LLMs

The investigation utilized a systematic approach to explore several LLMs, including proprietary and open-source models like GPT-4o, LLaMA, Mixtral, among others. The paper's primary aim is to discern where these models mirror or diverge from human reasoning, specifically in narrative coherence, susceptibility to bias, moral judgment, and handling contradictions.

Thematic Apperception Test (TAT)

In assessing LLMs using the TAT, the paper employed a selection of ambiguous images requiring narrative generation. The responses were then evaluated using the Social Cognition and Object Relations Scale--Global (SCORS-G) to capture dimensions like Complexity of Representation (COM) and Emotional Investment in Values and Moral Standards (EIM). This method reveals how LLMs project personality-like patterns and relational dynamics, offering a window into their narrative capabilities.

Figure 1: The narrative generation capability of models as tested with the Thematic Apperception Test illustrates the depth of their interpretative storytelling.

Framing Bias

The paper examines whether LLMs exhibit framing biases akin to humans by testing responses to variably framed questions. The results indicate that, similar to human behavior, LLMs showed a tendency towards positive entailment despite negative framing. This pattern raises considerations regarding the alignment objectives embedded in LLM training, as framing effects can significantly influence decision-making outputs.

Moral Foundations Theory (MFT) Analysis

Exploring the moral reasoning of LLMs under MFT, the paper extends a standard dataset with additional questions reflecting the six-core moral dimensions. Notably, LLMs displayed higher sensitivity to the Liberty/Oppression foundation, suggesting an embedded bias due to alignment methods like Reinforcement Learning with Human Feedback (RLHF). This points to a potential overemphasis in specific areas of moral reasoning during the fine-tuning process.

Cognitive Dissonance Evaluation

Cognitive dissonance is tested by exposing LLMs to contradictory scenarios and measuring their rationalization complexity and internal consistency. While the paper found low occurrences of direct contradictions, it highlighted extensive rationalizations in model responses, underscoring a propensity towards justified coherence over acknowledgment of inconsistencies.

Implications and Future Research

This research highlights the nuanced effects of model architecture and training on cognitive resemblance to human-like reasoning within LLMs. It underscores the importance of psychological paradigms in AI to potentially preempt biases and ethical concerns. Future work may explore additional cognitive phenomena and biases, continuous moral evaluation, and interactive decision-making tasks to broaden understanding.

Conclusion

The exploration of cognitive theories in machine psychology through this paper provides insights into the similarities and disparities between LLM-generated and human cognitive responses. While alignment methods facilitate models in achieving a coherent structure, the underlying biases in moral and cognitive decision processes remain a focal point for further ethical and technical scrutiny. These findings provoke a call to balance AI's evolving cognitive frameworks with their real-world applications, focusing on transparency and accountability.

Figure 1: Sample TAT Image illustrating interpretative storytelling capabilities of LLMs.