Partial LLM Mediation in Research

• Partial LLM Mediation is a phenomenon where participants use AI tools for targeted tasks such as translation, idea generation, or stylistic refinement, resulting in hybrid human-AI outputs.
• The selective use of LLMs reduces response variance and introduces cultural biases, challenging traditional assumptions in behavioral data analysis.
• Mitigation strategies include explicit norm signaling, input restrictions, and adaptive detection protocols to preserve the integrity of online research data.

Partial LLM Mediation denotes the phenomenon in online behavioral research wherein human participants selectively enlist LLMs to assist with specific aspects of a paper task, without fully delegating the interaction to the LLM. This selective engagement can involve practices such as using LLMs for translation, idea generation, or stylistic refinement, resulting in paper outputs that are partially—rather than wholly—shaped by AI. The interaction of Partial LLM Mediation with Full LLM Delegation and LLM Spillover forms a spectrum of LLM-induced distortions collectively referred to as “LLM Pollution” (Rilla et al., 2 Aug 2025).

1. Mechanisms and Examples of Partial LLM Mediation

Partial LLM Mediation occurs when participants leverage the capabilities of LLMs for limited, targeted support during online studies rather than submitting entirely AI-generated responses.

Common cases include:

• Translation assistance: Participants paste survey instructions into an LLM (e.g., ChatGPT) to translate text or clarify complex language before responding.
• Stylistic or wording support: Participants request the LLM to rephrase or improve their open-ended answers, leading to increased fluency or coherence.
• Strategic or cognitive advice: Participants enlist LLMs for hints on efficient survey completion or summarization of complex passages.

This form of mediation differs from Full LLM Delegation, where participants allow an agentic LLM to autonomously complete a paper or survey (i.e., outputs are almost entirely machine-generated). In Partial LLM Mediation, the human actor retains agency but the LLM’s influence percolates into the final response through indirect textual, conceptual, or stylistic modification (Rilla et al., 2 Aug 2025).

2. Impact on Data Validity and Research Integrity

Partial LLM Mediation poses several specific methodological threats to the validity of online behavioral research:

• Reduced Variance: LLM-mediated responses tend to be highly fluent and conform to dominant linguistic or rhetorical patterns embedded in training data. This “homogenization” effect reduces the natural diversity of human answers and may lead to artificially inflated statistical central tendencies.
• Cultural Bias: Since LLMs are predominantly trained on WEIRD data (Western, Educated, Industrialized, Rich, Democratic societies), their mediation introduces systematic biases, even if participants themselves are more culturally heterogeneous. For instance, translated or refined responses may inadvertently “Westernize” linguistic forms, idioms, or reasoning styles, thus contaminating the sample with unexpected confounds.
• Epistemic Uncertainty: Researchers risk misattributing these LLM-shaped outputs as genuine indicators of human cognition, leading to erroneous inferences and the erosion of the epistemic grounding of behavioral science data.

This threat is compounded by the relative subtlety of partial mediation, which makes automated post hoc detection challenging, in contrast to the often more obvious markers of full automation (Rilla et al., 2 Aug 2025).

3. Relationship to Other LLM Pollution Phenomena

Partial LLM Mediation is distinct but interacts with two other architectures of LLM-induced research contamination:

Variant	Description	Research Impact
Partial LLM Mediation	Selective, targeted use of LLM for specific tasks; human/AI hybrid outputs	Homogenization, cultural bias, data interpretability
Full LLM Delegation	Complete outsourcing of the paper to an LLM-based agent	Removes human agency, undermines human-subject base
LLM Spillover	Participants alter behavior in anticipation of (or reaction to) LLM presence	Self-signaling, moral licensing, behavioral drift

Full LLM Delegation poses the deepest epistemic threat by removing genuine human input, while LLM Spillover denotes behavioral shifts arising even in studies where no LLM intervention is actually employed (second-order effects).

Partial Mediation and Full Delegation form a continuum of increasing automation; LLM Spillover operates orthogonally as a second-order reactivity effect (Rilla et al., 2 Aug 2025).

4. Detection and Mitigation Strategies

A multi-layered response is advocated to preserve methodological integrity in the face of Partial LLM Mediation:

• Researcher Practices:
  • Employ explicit verbal norm signaling (e.g., consent declarations such as: “I confirm that all responses I provide in this paper are my own, without the assistance of any AI tools”).
  • Impose input restrictions (disabling copying/pasting, using audio or image-based instructions) to increase the friction of LLM-mediated participation.
  • Embed comprehension or attention checks that are prone to erroneous completions by LLMs but less so by attentive humans (e.g., visual illusions, hallucination-prone tasks).
• Platform Accountability:
  • Use bot-protection and behavioral scoring services (e.g., Cloudflare, reCAPTCHA). For example, reCAPTCHA v3 assigns a score $\in [0, 1]$; a practical threshold might be $0.7$ for exclusion.
  • Insert honeypot/invisible items to detect large-scale scraping or automated interaction.
• Community and Institutional Policies:
  • Curate repositories of best practices and coordinate protocol adaptation as LLM capabilities evolve.
  • Encourage platforms to explicitly prohibit unauthorized LLM use, and to provide relevant abuse-reporting/refund structures.

These approaches are designed to address both overt and subtle mediation, adapting as generative AI technologies and participant toolsets co-evolve (Rilla et al., 2 Aug 2025).

5. Theoretical and Methodological Implications

Partial LLM Mediation challenges core assumptions in behavioral science that online responses are uncontaminated indicators of human psychology. This may necessitate a reevaluation of:

• Statistical models of data variance and central tendency, now potentially conflated by AI-induced regularity.
• The “ecological baseline” of research, especially as AI mediation becomes a normative mode for cognitive augmentation and communication.
• Epistemic and ethical baselines, as the boundaries between purely human-generated and hybrid or largely AI-generated responses are blurred.

There is a plausible implication that, under continued technological and cultural trends, methodological baselines in behavioral research will shift to accommodate routine AI augmentation, necessitating continual updates to “valid data” frameworks and new theory.

6. Future Directions and Open Challenges

Ongoing challenges are anticipated as LLMs further excel at mimicking subtle aspects of human language, reasoning, and stylistic diversity:

• Increasing detection difficulty will fuel a methodological “arms race” between researchers and technologically assisted participants.
• Standardized, scalable, and adaptable detection protocols will be essential but may still lag behind state-of-the-art LLM capabilities.
• The distinction between valid and invalid (polluted) data may become less clear, necessitating new methodological, philosophical, and community-wide approaches to human-subject research online (Rilla et al., 2 Aug 2025).

This evolutionary dynamic underscores the necessity for coordinated adaptation spanning research practice, platform governance, and community infrastructure in order to protect the scientific integrity of online behavioral data in an era of Partial LLM Mediation.