Enhancing behavioral nudges with large language model-based iterative personalization: A field experiment on electricity and hot-water conservation

Abstract: Nudging is widely used to promote behavioral change, but its effectiveness is often limited when recipients must repeatedly translate feedback into workable next steps under changing circumstances. LLMs may help reduce part of this cognitive work by generating personalized guidance and updating it iteratively across intervention rounds. We developed an LLM agent for iterative personalization and tested it in a three-arm randomized experiment among 233 university residents in China, using daily electricity and shower hot-water conservation as objectively measured cases differing in friction. LLM-personalized nudges (T2) produced the largest conservation effects, while image-enhanced conventional nudges (T1) and text-based conventional nudges (C) showed similar outcomes (omnibus p = 0.009). Relative to C, T2 reduced electricity consumption by 0.56 kWh per room-day (p = 0.014), corresponding to an 18.3 percentage-point higher adjusted saving rate. This advantage emerged within the first two intervention rounds, alongside iterative updating of personalized guidance, and persisted thereafter. Hot-water outcomes followed the same direction but were smaller, less precisely estimated, and attenuated over time, consistent with stronger friction in this domain. LLM-personalized nudges emphasized prospective and context-specific guidance and were associated with higher participant engagement. This study provides field evidence that LLM-based iterative personalization can enhance behavioral nudging, with behavioral friction as a potential boundary condition. Larger trials and extension to more behaviors are warranted.

• The paper demonstrates that LLM-based iterative personalization significantly reduces electricity consumption by 0.56 kWh/room-day compared to conventional nudges.
• The study uses a three-arm randomized controlled trial with robust statistical and machine learning analyses to evaluate the effectiveness on both electricity and hot-water savings.
• The findings reveal rapid, sustained savings for electricity, while effects on high-friction behaviors like hot-water use are less durable.

Enhancing Behavioral Nudges via LLM-Based Iterative Personalization: An Empirical Assessment

Introduction

This paper rigorously investigates the application of LLM-based iterative personalization for behavioral nudging in the domain of electricity and hot-water conservation, leveraging a naturalistic three-arm randomized controlled trial (RCT) among university dormitory residents. The study directly quantifies the incremental efficacy of LLM-personalized nudges over conventional and visually enhanced conservation nudges, dissects user engagement dynamics, and scrutinizes both average and heterogeneous treatment effects using robust statistical and machine learning pipelines. The approach operationalizes LLMs for real-world health- and environment-critical behavior modification, contributing concrete field evidence and delineating boundary conditions for AI-personalized interventions.

Experimental Design and LLM Personalization Pipeline

The trial randomized 233 participants into three arms: (C) conventional text-based nudges, (T1) image-enhanced nudges, and (T2) LLM-personalized nudges, all delivered via a WeChat-based chatbot. Both electricity and hot-water consumption were instrumented as objective, high-granularity endpoints. T2 nudges incorporated three unique LLM-generated layers: (1) individualized conservation suggestions drawn from a curated knowledge base via retrieval-augmented generation (RAG); (2) context-grounded scenarios embedded in participants' daily routines; and (3) quantitative analogies translating expected savings into intuitive terms, all iteratively updated per participant trajectory.

Figure 1: Schematic of RCT design and decomposition of intervention components, highlighting the added dimensions of LLM personalization in T2.

The LLM intervention architecture used chain-of-thought prompting across modular subprocesses for analytic feedback, profile-based suggestion selection, and quantitative estimation. Profile state was recursively updated using fresh behavioral, interaction, and feedback logs at each intervention round.

Main Effects on Conservation Behavior

LLM-personalized nudges (T2) produced statistically significant and substantial reductions in electricity consumption relative to both controls and image-enhanced formats. T2 yielded a 0.56 kWh/room-day reduction (p = 0.014) over control, equating to an 18.3 percentage point increase in adjusted saving rate. Visual enhancements (T1) alone were statistically indistinct from conventional nudges (C). For hot-water conservation, T2 demonstrated the same directional ordering, but with attenuated effect size and non-significant precision, reflecting domain-specific behavioral friction.

Figure 2: Distribution and adjusted rates of electricity and hot water savings, showing a pronounced separation for T2 (LLM-personalized) groups.

Temporal trajectory analysis revealed that T2’s advantage for electricity emerged rapidly (within two rounds) and persisted through the intervention, while hot water savings were greatest initially but regressed toward baseline—a pattern congruent with higher cost-of-change for comfort-centric behaviors.

Content Analysis, Personalization Depth, and Engagement

Topic modeling and keyword analysis quantified a structural content shift in T2 nudges: higher incidence of planning/action language and appliance-specific guidance, as opposed to descriptive, comparative, or motivational content dominating conventional nudges.

Figure 3: Topic composition, participant engagement, and survey-based evaluation of content actionability and satisfaction for each nudge arm.

Per-user content iteratively adapted to behavioral data, with actionability and satisfaction measures increasing over rounds. T2 elicited higher engagement rates (69.7% vs. 57–58% for other arms), more sustained responsiveness, and higher task-focus in chatbot interactions—suggesting that iterative personalization not only affected outcomes but also increased the intervention’s psychological reach.

Heterogeneous Effects and Behavioral Archetypes

Ensemble meta-learners estimated individual treatment effects (ITEs), revealing that T2’s superiority manifested over a broader cross-section of participants, especially for electricity. Distributional analysis mapped archetypal behavioral trajectories: T2 increased the proportion of “quick responders” and decreased “adverse responders” for both resource classes. Archetype and ITE heterogeneity was associated with baseline pro-conservation attitudes and living budgets, underlining the importance of psychological and economic priors in nudge responsiveness.

Figure 4: Estimated ITE distributions and relative prevalence of behavioral archetypes by trial arm, underscoring LLM-driven population shifts toward early and persistent savings.

Predictive Modeling of Conservation and Dynamics

XGBoost models identified baseline consumption as the dominant predictor for both resource types, yet relative importance of psychological and socio-structural predictors increased for hot-water over time, paralleling declining effect size as adaption progressed. This underscores the explanatory value of pre-existing habits and the shifting role of attitude/efficacy constructs in high-friction domains.

Figure 5: Feature importance analysis over time, showing the diverging influence of baseline behavior and psychological/socio-structural factors.

Theoretical and Practical Implications

The study demonstrates that LLM-personalized nudges can serve as scalable, lightweight coaching mechanisms, pushing nudge mechanisms toward dynamic state adaptation and cross-round content updating—features only marginally approachable by rule-based personalization. The magnitude and durability of effects, however, are highly sensitive to behavioral friction, with lower-friction domains (e.g., electricity) showing robust, persistent improvements. This aligns with the broader literature arguing for nuanced segmentation of behavioral target domains when deploying AI-personalized interventions [RN55].

Integration with existing DTR and JITAI paradigms is seamless: LLM-based nudges instantiate just-in-time, adaptive, context-sensitive intervention logic using open-ended natural language, but without the strict formal optimality constraints or limited expressivity of static templates. The potential for integrating richer, end-user state tracking or dynamic skill acquisition is evident, but raises substantial privacy, cultural, and autonomy concerns.

Limitations and Future Research Trajectories

The research is explicitly limited to a student population and two resource-conservation behaviors; generalizability to broader, multi-behavioral settings remains to be empirically established. The intervention treated LLM-personalized nudges as a bundled package, leaving the precise contribution of content, context integration, and iterative updating uncoupled—suggesting a need for systematic ablation studies. Longer-term persistence, spillover, and scaling effects, as well as cross-cultural robustness, are critical open questions. Additionally, differential efficacy among psychologically or socioeconomically distinct subpopulations mandates further investigation, especially regarding fairness and model bias concerns [RN63] [RN64] [RN65].

Conclusion

LLM-based iterative personalization significantly elevates the efficacy and engagement of behavioral nudges for electricity conservation, with moderate but less durable effects for high-friction behaviors such as hot-water use. The findings support deeper integration of AI-driven, contextually adaptive support in digital health and environmental interventions, while emphasizing the necessity for behavioral segmentation, ablation-driven mechanism elucidation, and proactive governance against unintended harms. This work sets a new benchmark for field validation of AI-personalized nudges and outlines concrete pathways for both methodological refinement and theoretical expansion in adaptive behavior change interventions.

(2604.03881)