CRMAgent: A Multi-Agent LLM System for E-Commerce CRM Message Template Generation (2507.08325v2)

Abstract: In e-commerce private-domain channels such as instant messaging and e-mail, merchants engage customers directly as part of their Customer Relationship Management (CRM) programmes to drive retention and conversion. While a few top performers excel at crafting outbound messages, most merchants struggle to write persuasive copy because they lack both expertise and scalable tools. We introduce CRMAgent, a multi-agent system built on LLMs that generates high-quality message templates and actionable writing guidance through three complementary modes. First, group-based learning enables the agent to learn from a merchant's own top-performing messages within the same audience segment and rewrite low-performing ones. Second, retrieval-and-adaptation fetches templates that share the same audience segment and exhibit high similarity in voucher type and product category, learns their successful patterns, and adapts them to the current campaign. Third, a rule-based fallback provides a lightweight zero-shot rewrite when no suitable references are available. Extensive experiments show that CRMAgent consistently outperforms merchants' original templates, delivering significant gains in both audience-match and marketing-effectiveness metrics.

• The paper presents a novel multi-agent LLM framework that generates and optimizes CRM message templates for e-commerce.
• It integrates ContentAgent, RetrievalAgent, TemplateAgent, and EvaluateAgent to analyze, retrieve, rewrite, and score marketing messages.
• Experimental results reveal a 9.09% improvement in audience fit and a 38.44% boost in marketing effectiveness.

CRMAgent: A Multi-Agent LLM System for E-Commerce CRM Message Template Generation

The paper "CRMAgent: A Multi-Agent LLM System for E-Commerce CRM Message Template Generation" explores an innovative multi-agent system designed to optimize customer relationship management (CRM) message templates in e-commerce settings (2507.08325). The system employs a modular approach driven by LLMs to rewrite underperforming message templates, enhancing audience engagement and marketing effectiveness for e-commerce merchants. This paper explores the architecture and implementation details of CRMAgent, empirical evaluations of its performance, and provides insights into its optimization of customer relationship management workflows.

Introduction

CRMAgent is positioned to address deficiencies in e-commerce Customer Relationship Management (CRM) communication by automating the generation of effective message templates. Outbound CRM messages serve as a critical medium for engaging customers and boosting conversion rates in digital marketing contexts. However, crafting these messages poses challenges for many merchants who lack specialized marketing expertise and time, leading to a reliance on generic, system-generated copy. The authors of this paper present CRMAgent as a multi-agent system integrated with LLMs to enhance the quality of CRM message templates. Through the use of historical engagement data, CRMAgent identifies low-performing messages to improve them by leveraging top-performing templates from similar contexts.

Figure 1: Framework of CRMAgent, a multi-agent system for improving underperforming CRM message templates in e-commerce.

The multi-agent system comprises ContentAgent, RetrievalAgent, TemplateAgent, and EvaluateAgent, each responsible for a specific subtask such as content analysis, template retrieval and adaptation, and evaluation. The system divides its functionality into group-based learning for in-group optimization, retrieval-and-adaptation across merchants for reference gathering, and a rule-based fallback strategy for zero-shot scenarios. The framework focuses on maintaining semantic consistency while optimizing audience alignment and marketing efficacy.

Multi-Agent System Architecture

ContentAgent

The ContentAgent plays a critical role by conducting structured comparisons between high- and low-performing CRM message templates within the same audience segment. This diagnostic process encompasses evaluating structural elements, tone, wording, and the efficacy of incentive presentations. The outcomes provide contextual insight into successful message attributes, which are integrated into the rewriting processes of the CRMAgent system.

RetrievalAgent

The RetrievalAgent is instrumental in providing good-performing CRM template examples when internal merchant templates fall short. It performs cross-merchant retrieval based on audience similarity, voucher type, and product category. This agent uses dense embeddings developed by a SentenceTransformer model for efficient similarity searches via Faiss indexing.

TemplateAgent

As the component responsible for rewriting templates, the TemplateAgent receives inputs from both ContentAgent and RetrievalAgent to form improved messages. Depending on the availability of suitable exemplars, TemplateAgent employs a tiered generation strategy, using either group-based learning, retrieval-based adaptation, or rule-based rewriting to produce effective and persuasive CRM messages. The prompts employed by TemplateAgent ensure that the original promotional intent is preserved while improving clarity, structure, and incentive framing.

EvaluateAgent

The EvaluateAgent fulfills an essential evaluation function by scoring each message for audience alignment and marketing effectiveness. It provides objective assessments in scoring mode, based on a predefined three-level scale. In comparison mode, it engages in pairwise preference judgments, electing the superior message based solely on persuasive strength and audience match. This twofold evaluation process provides a robust assessment of the agent-generated content quality.

Experimental Results

Extensive experiments were conducted to validate the CRMAgent framework's efficacy in improving CRM message template performance. Metrics reported in the experiments include audience fit, marketing effectiveness, and comparative preference. The paper applied GPT-4o as the backbone model across the agents, with variations in model configurations for ablation studies.

The compelling experimental results of CRMAgent demonstrate significant uplift across diverse audience segments, achieving a 9.09\% improvement in audience fit score and a 38.44\% boost in marketing effectiveness, as shown in Table~\ref{tab:audience_segment_summary}.

Semantics and Rewriting Fidelity

The semantic consistency between original and generated CRM messages was measured using BERTScore-F1 at a token level. The average BERTScore-F1 for generated messages is 0.87, whereas the more specific chrF metric value is 24.22 for surface overlap. These metrics corroborate that the generated templates maintain semantic alignment with the originals while improving their overall effectiveness.

Figure 2: Distribution of CRM audience segments based on plan targeting.

Experiments and Results

CRMAgent was evaluated across diverse audience segments, including potential new customers, lapsing buyers, and frequent buyers. As depicted in Table \ref{tab:audience_segment_summary}, generated templates consistently outperformed original messages, with EvaluateAgent expressing a preference for CRMAgent's output in 78.44% of cases.

Conclusion

The CRMAgent framework represents a sophisticated implementation of multi-agent LLM systems for e-commerce CRM message template generation. By combining fine-grained performance analysis, multi-source template retrieval, and strategic multi-agent collaboration, the system demonstrates significant improvements in both audience alignment and marketing effectiveness across diverse e-commerce segments, as confirmed by rigorous automatic and human evaluation. The use of a modular architecture and exemplar-based learning offers promising scalability and adaptability across marketing contexts and scenarios. Future directions could involve the integration of more nuanced personalization techniques and extensions to other domains where audience segmentation is critical. The findings from the paper suggest that the multi-agent LLM approach holds promise for scaling automated CRM solutions, potentially transforming customer engagement across digital commerce ecosystems.