An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies (2509.12577v1)

Abstract: In an era of increasing societal fragmentation, political polarization, and erosion of public trust in institutions, representative deliberative assemblies are emerging as a promising democratic forum for developing effective policy outcomes on complex global issues. Despite theoretical attention, there remains limited empirical work that systematically traces how specific ideas evolve, are prioritized, or are discarded during deliberation to form policy recommendations. Addressing these gaps, this work poses two central questions: (1) How might we trace the evolution and distillation of ideas into concrete recommendations within deliberative assemblies? (2) How does the deliberative process shape delegate perspectives and influence voting dynamics over the course of the assembly? To address these questions, we develop LLM-based methodologies for empirically analyzing transcripts from a tech-enhanced in-person deliberative assembly. The framework identifies and visualizes the space of expressed suggestions. We also empirically reconstruct each delegate's evolving perspective throughout the assembly. Our methods contribute novel empirical insights into deliberative processes and demonstrate how LLMs can surface high-resolution dynamics otherwise invisible in traditional assembly outputs.

• The paper introduces a novel LLM-based framework for high-resolution tracking of idea evolution in deliberative assemblies.
• The methodology integrates transcript analysis, refined prompt engineering for suggestion extraction, and semantic embedding with t-SNE visualization.
• The study highlights actionable findings, including overlooked policy proposals and dynamic delegate opinion shifts driven by expert input.

An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies

Introduction and Motivation

This paper presents a methodological and empirical advance in the paper of deliberative assemblies, focusing on the fine-grained tracing of idea evolution and delegate perspective dynamics using LLMs. The context is a three-day, in-person, tech-enhanced deliberative assembly at MIT, where undergraduate delegates generated and voted on sustainability policy recommendations. The central research questions are: (1) How can the evolution and distillation of ideas into concrete recommendations be empirically traced within deliberative assemblies? (2) How does the deliberative process shape delegate perspectives and influence voting dynamics?

The work is motivated by the lack of empirical tools for high-resolution analysis of deliberative processes, which are typically opaque beyond their final recommendations. The authors leverage LLMs for transcript analysis, suggestion extraction, semantic embedding, and stance profiling, aiming to surface latent deliberative dynamics and inform both theory and practice in deliberative democracy.

Figure 1: A schematic breakdown of deliberative assembly phases, illustrating the structured progression from learning to deliberation to recommendation.

Methodological Framework

Assembly Data Collection and Structure

The assembly comprised 19 undergraduate delegates over three days, with a six-day intermission between the second and third days. The process included plenary and breakout sessions, expert panels, iterative recommendation drafting, and two rounds of voting (preliminary and final). The assembly was fully audio-recorded, with over 40 hours of data transcribed and diarized by speaker, yielding 6,300 speaker turns across 34 transcripts.

LLM-Based Suggestion Extraction and Embedding

The core methodological innovation is the use of LLMs (OpenAI o3-mini) for high-recall extraction of explicit suggestions from transcripts. The prompt engineering process was iteratively refined to minimize false positives and negatives, with final outputs achieving near-zero false negatives and a 90% reduction in false positives (see Appendix for prompt details). Extracted suggestions were embedded using OpenAI's text-embedding-3-small model, and t-SNE was applied for 2D semantic visualization.

Figure 2: t-SNE visualization of the embedding space for all extracted suggestions (gray circles), failed recommendations (red X), and passed recommendations (green +).

This visualization reveals substantial regions of the suggestion space not covered by final recommendations, raising questions about deliberative filtering and potential omission of viable ideas.

Analysis of Suggestion Gaps

The 100 suggestions most semantically distant from any final recommendation were further analyzed. Manual qualitative coding distinguished between concrete, actionable policy ideas and approach-oriented or value-expressive suggestions.

Figure 3: Distribution of the 100 most distant suggestions by type, with a majority being concrete ideas in domains such as Food/Dining, Renewable Energy, and Information/Education.

Of these, 54 were concrete policy suggestions, and 13 were identified as distinct, actionable ideas not reflected in the final recommendations. The remainder were either redundant with existing recommendations or reflected values and priorities rather than actionable proposals.

Figure 4: t-SNE embedding with categorical overlays: approach-oriented (white), concrete (light pink), and the 13 actionable, missed suggestions (dark pink).

Spatial analysis shows that gaps between recommendations are populated by both concrete and approach-oriented suggestions, with some regions exhibiting higher concentrations of actionable ideas.

Delegate Stance Profiling and Dynamics

Modular Delegate Profile Construction

To address the second research question, the authors introduce a modular, evidence-grounded framework for reconstructing each delegate's evolving stance on key statements. For each delegate and statement, all transcript segments evidencing a stance are identified, enabling temporal tracking of opinion shifts.

Case Study: Vote Change on Divestment Recommendation

A detailed case paper traces a delegate's evolving stance on fossil fuel divestment. Initially supportive, the delegate's position shifted to neutral after expert input revealed the negligible financial impact and limited feasibility of divestment, leading to a vote change that contributed to the recommendation's failure to reach the supermajority threshold.

This analysis demonstrates the framework's capacity to anchor vote changes to specific deliberative moments and mechanisms, surfacing the nuanced interplay between values, feasibility assessments, and group dynamics.

Empirical Findings and Implications

Suggestion Space Coverage and Deliberative Filtering

The empirical results show that while the set of voiced suggestions is much larger than the set of final recommendations, the majority of "gaps" in the suggestion space are not indicative of deliberative failure. Most distant suggestions are either appropriately filtered as non-viable or reflect values and priorities that, while important, do not translate into actionable policy. However, the identification of 13 distinct, actionable, and overlooked suggestions highlights the potential for LLM-based tools to support more comprehensive deliberation and prevent omission of viable ideas.

Delegate Perspective Dynamics

The delegate profiling framework reveals that opinion change is often driven by new information (e.g., expert input), feasibility constraints, or strategic considerations. The ability to reconstruct and share these dynamics post-assembly enhances transparency and accountability, providing decision-makers with critical context that is otherwise lost in the aggregation to final recommendations.

Tooling and Transparency

The authors argue for the integration of such LLM-based frameworks into live deliberative assemblies, both to surface overlooked ideas in real time and to generate shareable outputs that capture the full spectrum of values, priorities, and trade-offs considered by delegates. This has direct implications for the legitimacy and trustworthiness of deliberative processes, addressing longstanding critiques regarding the opacity of assembly outputs.

Limitations and Future Directions

The analysis is limited by the focus on the 100 most distant suggestions rather than the full set of 488, and by the inability to recover the true reasons for omission of specific suggestions. The authors propose extending the framework to other assemblies and public forums, enabling comparative analysis of deliberative effectiveness, suggestion space coverage, and the influence of facilitation or group composition. The methodology is also adaptable for real-time intervention, potentially transforming the design and facilitation of future tech-enhanced assemblies.

Theoretical and Practical Implications

The work advances the empirical paper of deliberative democracy by operationalizing the tracing of idea evolution and delegate perspective change at high resolution. It demonstrates the utility of LLMs for both retrospective analysis and prospective tooling in democratic innovation. The findings challenge simplistic interpretations of suggestion-recommendation gaps as failures, instead highlighting the productive role of deliberative filtering and the importance of capturing value-expressive discourse.

Theoretically, the framework provides a basis for studying deliberative pluralism, the dynamics of consensus and disagreement, and the mechanisms of opinion change. Practically, it offers actionable pathways for increasing the transparency, accountability, and comprehensiveness of deliberative outputs, with direct relevance for policymakers, assembly designers, and democratic theorists.

Conclusion

This paper introduces a robust, LLM-powered framework for analyzing the evolution of ideas and delegate perspectives in deliberative assemblies. By combining high-recall suggestion extraction, semantic embedding, and modular stance profiling, the approach surfaces latent deliberative dynamics and actionable insights that are otherwise invisible in traditional assembly outputs. The empirical findings demonstrate both the strengths and limitations of current deliberative practices, and the framework lays the groundwork for future research and tooling aimed at enhancing the legitimacy, transparency, and effectiveness of democratic deliberation.