Social Opinions Prediction Utilizes Fusing Dynamics Equation with LLM-based Agents (2409.08717v4)

Abstract: In the context where social media emerges as a pivotal platform for social movements and shaping public opinion, accurately simulating and predicting the dynamics of user opinions is of significant importance. Such insights are vital for understanding social phenomena, informing policy decisions, and guiding public opinion. Unfortunately, traditional algorithms based on idealized models and disregarding social data often fail to capture the complexity and nuance of real-world social interactions. This study proposes the Fusing Dynamics Equation-LLM (FDE-LLM) algorithm. This innovative approach aligns the actions and evolution of opinions in LLMs with the real-world data on social networks. The FDE-LLM devides users into two roles: opinion leaders and followers. Opinion leaders use LLM for role-playing and employ Cellular Automata(CA) to constrain opinion changes. In contrast, opinion followers are integrated into a dynamic system that combines the CA model with the Susceptible-Infectious-Recovered (SIR) model. This innovative design significantly improves the accuracy of the simulation. Our experiments utilized four real-world datasets from Weibo. The result demonstrates that the FDE-LLM significantly outperforms traditional Agent-Based Modeling (ABM) algorithms and LLM-based algorithms. Additionally, our algorithm accurately simulates the decay and recovery of opinions over time, underscoring LLMs potential to revolutionize the understanding of social media dynamics.

• The paper presents a novel FDE-LLM approach combining dynamics equations and LLM-based models to simulate social opinion evolution.
• It distinguishes opinion leaders with CA-constrained LLMs and followers via a CA-SIR hybrid to enhance simulation fidelity.
• Experimental evaluations on Weibo data show improved DTW and correlation metrics over traditional opinion simulation models.

Social Opinions Prediction Utilizes Fusing Dynamics Equation with LLM-based Agents

Introduction

The paper introduces an innovative approach to predicting social opinions by integrating dynamics equations within LLM-based agents, termed FDE-LLM. This research targets the complexities of opinion evolution in social media environments by leveraging the strengths of both opinion dynamics models and epidemic models to enhance accuracy in simulating user behaviors on platforms like Weibo. Existing methodologies, such as Agent-Based Modeling (ABM) and standalone LLMs, face limitations in capturing the nuanced interactions of user opinions. This paper aims to address these limitations by categorizing users into opinion leaders and followers, with the former driven by LLMs constrained by Cellular Automata (CA), and the latter simulated through a hybrid system combining CA with the SIR epidemic model.

Methodology

FDE-LLM Structure

The FDE-LLM method distinguishes between opinion leaders and followers. Opinion leaders are modeled using a CA framework superimposed on LLM operations tailored to simulate influential user behavior patterns. Conversely, followers are modeled through a dynamic system utilizing CA and the SIR model, which accounts for the temporal shifts in opinion propagation and attenuation over time. This dual-approach results in a more nuanced model that bridges micro-level opinion changes with macro-level social phenomena.

Figure 1: Work Flow

Opinion Dynamics Fusion

The pivotal aspect of FDE-LLM lies in its synthesis of opinion dynamics mechanisms with an SIR model. Opinion dynamics aim to predict convergence, while the SIR model introduces a mechanism to simulate opinion susceptibility and recovery. Mathematically, the opinion updating process for leaders, influenced by observations from LLMs and CA, is described by the equation:

$$O_i^{t+1} = \mathrm{clip}\left(\alpha \cdot \left(r \cdot O_i^t + w \cdot \sum_{j \in N_i} T_{ij}^t\right) + (1 - \alpha) \cdot \text{LLM}, -1, 1\right)$$

This formula aims to balance intrinsic retention of opinions with external influences, leveraging the CA model to integrate local interaction effects with the overall influence inferred by LLM insights.

Figure 2: Actions and Attitudes. The left part represents the types of actions that can be chosen by LLM-Action, while the right part details the specific behaviors of the executed action type and the LLM-Attitude's scoring of the attitude towards that behavior.

Experimental Evaluation

Setup and Results

Experiments were conducted using Weibo datasets encompassing four high-profile incidents. The methodology involved utilizing classification-based Dynamic Time Warping (DTW) distance and Pearson correlation to benchmark FDE-LLM against CA models and hybrid LLM-ABM approaches. The results indicated superior performance of FDE-LLM in mirroring real-world opinion trends with high fidelity, as evidenced by improved DTW and correlation metrics. For instance, in the Pangmao event, FDE-LLM obtained a DTW of 0.3622 and a correlation of 0.9653, significantly outperforming standalone models.

Figure 3: Comparison of Real and Simulated Result on (a) Pangmao Incident (b) Jiangping Incident (c) Qingdao Incident (d) Dianduji Incident. Red lines represents the actual data.

Ablation Studies

A series of ablation studies highlighted the roles of CA constraints in ensuring stability and realism in opinion predictions. The absence of CA in leader dynamics resulted in overreactive projections, reinforcing the necessity of maintaining CA constraints for modeling accuracy.

Figure 4: FDE-LLM without CA

Implications and Future Work

The implications of the FDE-LLM model are substantial in its ability to provide more realistic simulations of opinion dynamics, facilitating better-informed policy making and strategic public opinion management. The introduction of decay via the SIR model into opinion dynamics presents a novel consideration for future explorations, particularly in scaling such simulations across diverse social platforms and global events.

Conclusion

By integrating large-scale LLM capabilities with dynamic interaction models, the FDE-LLM framework presents a nuanced and empirically validated approach to understanding and predicting opinion dynamics. The dual-components of this system—opinion leader modeling through CA-constrained LLMs and opinion follower moderation using CA and SIR—enable a more robust, predictable, and interpretable simulation of complex social interactions. The continued evolution of such methodologies holds promise for broader applications across varied domains of social network analysis and policy development.