Real-world effects of unrestricted interactions with biased AI language models

Determine how the outcomes of user interactions with biased AI language models—specifically changes in political opinions in the Topic Opinion Task and budget allocation decisions in the Budget Allocation Task—differ in an unregulated real-world setting without limits on the number or type of interactions, relative to the study’s controlled setting that imposed a maximum of 20 interactions per task.

Background

The paper measured immediate changes in political opinions and budgeting decisions after participants interacted with AI LLMs that were programmed to exhibit liberal or conservative bias, compared against a neutral control. To standardize the experimental environment, interactions were constrained to between three and twenty exchanges per task.

The authors note that while these constraints facilitate internal validity, they may not reflect typical real-world usage where users can engage with AI systems in an unrestricted manner. Understanding whether and how the magnitude or nature of AI-induced influence changes outside controlled conditions is important for assessing real-world risks and mitigation strategies.