MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agents (2507.05330v1)

Abstract: Recent advances in LLMs have enabled new applications in e-commerce customer service. However, their capabilities remain constrained in complex, multimodal scenarios. We present MindFlow, the first open-source multimodal LLM agent tailored for e-commerce. Built on the CoALA framework, it integrates memory, decision-making, and action modules, and adopts a modular "MLLM-as-Tool" strategy for effect visual-textual reasoning. Evaluated via online A/B testing and simulation-based ablation, MindFlow demonstrates substantial gains in handling complex queries, improving user satisfaction, and reducing operational costs, with a 93.53% relative improvement observed in real-world deployments.

• The paper introduces MindFlow, a novel multimodal LLM agent framework for e-commerce support, combining working and long-term memory to enhance context-aware responses.
• It utilizes a modular 'MLLM-as-Tool' paradigm and a 'Propose-Evaluate-Select' framework that streamlines decision-making and reduces token consumption.
• Online A/B testing and simulation-based ablation demonstrate up to a 93.53% improvement in AI contributions and significant reductions in response delays.

MindFlow: Revolutionizing E-commerce Customer Support with Multimodal LLM Agents

Introduction

The exponential growth of e-commerce has led to increased demand for sophisticated customer support systems capable of managing complex, multimodal inquiries in real-time. MindFlow introduces a novel solution by leveraging multimodal LLM agents specifically tailored for e-commerce customer service. Built on the CoALA framework, it integrates tightly coupled memory, decision-making, and action modules, employing a modular "MLLM-as-Tool" paradigm to enhance visual-textual reasoning efficiency.

Figure 1: MindFlow Architecture.

Memory and Decision-Making Modules

MindFlow's memory module is bifurcated into working and long-term memory components. The working memory accounts for the historical dialogue context, which aids in inferencing buyer intent during unclear queries, while long-term memory stores domain-specific knowledge such as platform policies and buyer-specific information. Both components are essential for context-aware reasoning across different e-commerce scenarios, ensuring accurate and adaptive responses.

The decision-making module employs a "Propose-Evaluate-Select" framework. This involves generating multiple candidate action plans, evaluating them for alignment with buyer intent, and selecting the most suitable option. This framework promotes the consideration of diverse possibilities, enhancing the agent's decision-making confidence and adaptability to dynamic scenarios.

Action Module and ACI

The action module encompasses both external and internal actions, enabling MindFlow to retrieve relevant information dynamically and execute structured decision-making processes. Inspired by ACI, MindFlow simplifies complex inputs using placeholders for token-heavy multimodal data such as images and URLs. This approach reduces token consumption and cognitive load, enhancing response time and reasoning accuracy.

Multimodal Integration Strategy

MindFlow employs the "MLLM-as-Tool" paradigm to address challenges associated with processing multimodal customer queries. By treating multimodal LLMs as specialized visual processors and maintaining high-level reasoning separate from perception, MindFlow avoids verbose, inaccurate responses typical of monolithic LLMs. This modular strategy yields more streamlined decision-making processes and facilitates easier debugging.

Figure 2: Module Ablation Performance Comparison.

Experimental Evaluation

Online A/B Testing

MindFlow was evaluated using online A/B testing in real-world e-commerce environments. Results showed substantial performance improvements over rule-based systems, particularly in resolving product consultations with a 93.53% relative improvement in AI contribution ratios. This validated the efficacy of dynamic tool invocation for obtaining accurate, up-to-date information and highlighted limitations in logistics order support improvements due to robust prior rule-based frameworks.

Simulation-Based Ablation

Additional evaluation using ECom-Bench simulated real-world complexities and verified MindFlow's robustness. Both the decision-making module and ACI highlighted significant performance improvements. The decision-making module's optimal action selection and ACI's input abstraction significantly reduced response delays, validated by pass^k metrics and task completion time reductions.

Figure 3: Multimodal Integration Strategy Comparison.

Multimodal Strategy Comparison

The "MLLM-as-Tool" paradigm consistently outperformed alternative strategies by isolating visual processing, reducing failure rates, and improving error traceability. This approach showed significant gains in complex task scenarios, reaffirming its effectiveness for enhancing robust multimodal reasoning.

Conclusion

MindFlow represents a significant advancement in e-commerce customer support systems through its integration of multimodal LLM agents. By harnessing a modular architecture and distinct MLLM-as-Tool strategy, MindFlow enhances real-time, contextually aware customer service capabilities, ensuring dynamic adaptability and scalability.

Future work includes refining long-term memory updates, enhancing decision modules' calibration for improved rejection handling, and expanding input abstractions to other complex data types. Broader environmental testing will solidify its versatility and reliability.

Despite its successes, MindFlow faces limitations such as static long-term memory updates, heuristic-based decision-making, and dependency on external tools. Addressing these areas will further strengthen its deployment robustness in varied and high-stakes e-commerce environments.