Automated Consensus Prompting with LLMs

• Automated Consensus Prompting is a framework that uses large language models to facilitate group decision-making by synthesizing diverse user inputs into coherent consensus proposals.
• It employs a multi-layered system architecture where iterative feedback, adaptive strategy selection, and real-time voting combine to refine consensus proposals through several rounds of evaluation.
• Quantitative metrics, such as cosine similarity and iteration count, are used to benchmark model performance and alignment, highlighting the strengths of different LLMs in achieving efficient consensus.

Automated Consensus Prompting with LLMs

Automated consensus prompting is the class of computational protocols and prompt engineering strategies that leverage LLMs to facilitate, measure, and optimize the formation of group consensus in decision-making or deliberative processes. Recent research advances systematically address the challenges of viewpoint reconciliation, alignment quantification, @@@@2@@@@, and adaptive strategy selection, establishing a rigorous, reproducible methodology for LLM-driven consensus facilitation (Triantafyllopoulos et al., 3 Feb 2025).

1. System Architecture for LLM-Mediated Consensus

A prototypical automated consensus prompt system comprises a multi-layered architecture integrating a browser-based multi-user chat frontend, a back-end for real-time message routing (Node.js), persistent storage (MySQL), session management (PHP), and a model orchestration layer interfacing to multiple LLM endpoints via API. Participants interact pseudonymously; their contributions and feedback are logged and made available to LLM facilitators tasked with synthesizing consensus proposals.

The consensus formation workflow proceeds in the following sequential steps:

1. System Initialization: Load system prompts (role definitions, norms), clear chat/proposal logs.
2. Opinion Collection: Post a moderator's question and collect free-text stances from participants.
3. Initial Proposal Generation: The orchestrator prompts the selected LLM (e.g., ChatGPT 4.0, AI21 Jamba-Instruct, Mistral Large 2) to ingest all user stances and generate a first-draft consensus statement.
4. Voting: Each participant votes to accept or reject the proposed consensus.
5. Feedback Gathering: For rejected proposals, prompt each rejecting participant for concise feedback specifying deficiencies or ambiguities.
6. Adaptive Strategy Selection: Analyze chat history and feedback to select a facilitation strategy (clarify, summarize, highlight common ground, propose compromise, reframe).
7. Proposal Revision: Re-prompt the LLM with a template encoding the selected strategy, the full transcript, and summarized feedback.
8. Loop or Terminate: Iterate until consensus is achieved (all accept) or a maximum number of iterations (typically 3–5) is reached.

This architecture enables structured, iterative consensus-building unconstrained by the scalability and subjectivity limitations of traditional human facilitation (Triantafyllopoulos et al., 3 Feb 2025).

2. Quantitative Alignment and Evaluation Metrics

Alignment between consensus proposals and individual participant viewpoints is rigorously quantified using cosine similarity in embedding space. Specifically, each participant's initial stance and each candidate consensus proposal are embedded (e.g., using the Universal Sentence Encoder); the cosine similarity between participant vectors $\mathbf{u}$ and proposal vector $\mathbf{c}$ is computed as

$$\cos(\theta) = \frac{\mathbf{u} \cdot \mathbf{c}}{\|\mathbf{u}\| \, \|\mathbf{c}\|}.$$

Session-level alignment is averaged over all participants. Intermediate proposals are similarly evaluated against evolving user stances to track convergence dynamics.

Additional key metrics include:

• Number of Iterations to Consensus: Tracks process efficiency.
• Statistical Summaries: Mean, standard deviation of alignment scores per model/topic.
• Similarity Trajectory Curves: Visualize alignment progression over refinement rounds.

Empirical evaluation across 75 sessions (N=30, 2 participants per session, four UN SDG domains) revealed model differentiation: ChatGPT 4.0 achieved a mean cosine alignment of 0.701 in 2.1 iterations, outperforming AI21 Jamba-Instruct (0.613/3.8) and Mistral Large 2 (0.581/4.2). Domain-dependence was observed, with the highest alignment in climate action and lowest in water/sanitation topics, attributed to semantic dispersion of input stances (Triantafyllopoulos et al., 3 Feb 2025).

3. Adaptive Facilitation Strategies

The orchestrator dynamically selects from five adaptive consensus strategies, each encoded as a distinct prompt template:

• ClarifyUnderstanding: “Please clarify any terms or concepts above that participants found confusing.”
• SummarizeDiscussion: “Provide a concise summary of the main agreements and disagreements so far.”
• HighlightCommonGround: “Identify statements on which all or most participants already agree.”
• ProposeCompromise: “Suggest a compromise that balances the conflicting priorities raised.”
• ReframeQuestion: “Rephrase the core question to emphasize shared goals rather than differences.”

Selection is informed by feedback and the chat transcript; chosen strategies are injected into subsequent LLM prompts. Pseudocode for this iterative refinement loop is:

initialize chat_history, proposals = [] while not consensus_reached(chat_history): if proposals is empty: prompt = build_initial_prompt(chat_history) else: feedback = collect_user_feedback(chat_history, last_proposal) strategy = select_strategy(feedback, chat_history) prompt = build_prompt_with_strategy(chat_history, feedback, strategy) new_proposal = LLM.generate(prompt) proposals.append(new_proposal) votes = collect_votes(new_proposal) if all(vote == "accept" for vote in votes): consensus = True else: consensus = False return final_proposal, proposals

Superiority of early use of “HighlightCommonGround” and “ProposeCompromise” was observed, especially in complex or polarized discussions (Triantafyllopoulos et al., 3 Feb 2025).

4. Empirical Insights and Model Comparison

ChatGPT 4.0 consistently demonstrated the highest alignment and lowest iteration count, attributed to its parameter scale, RLHF-based tuning, and effective strategy selection. It outperformed competitors both in initial alignment (≈0.75, first draft) and process efficiency.

Topical decomposition indicated pronounced difficulty in water/sanitation consensus, explained by broader semantic range (technical, infrastructural, geopolitical stances) among participants. In contrast, high alignment was achieved in climate action discussions due to ChatGPT 4.0’s ability to synthesize and balance specialized domain arguments.

Aggregate results:

Model	Avg. Cosine Similarity	Avg. # Iterations
ChatGPT 4.0	0.701	2.1
AI21 Jamba-Instruct	0.613	3.8
Mistral Large 2	0.581	4.2

Substantial performance margins (e.g., climate action: 0.849 for ChatGPT 4.0 vs. 0.557 [Mistral] and 0.497 [AI21]) were observed (Triantafyllopoulos et al., 3 Feb 2025).

5. Design Guidelines and Future Directions

Effective consensus prompt engineering is grounded in principles empirically validated by process metrics and qualitative user feedback. Core recommendations include:

• Role-Rigorous Initialization: Explicitly assign the LLM an impartial facilitation role.
• Dynamic Context Injection: Encode participant input and rejection feedback to inform LLM context.
• Explicit Strategy Declaration: Specify the selected facilitation strategy within system prompts.
• Length/Style Constraints: Impose concise, accessible statement guidelines (2–3 sentences, avoid jargon, foreground shared values).
• Iterative Check Structure: Each proposal ends with an explicit query for remaining concerns.

Limitations include reliance on cosine similarity as the sole quantitative alignment measure (omitting persuasiveness and satisfaction), small session sizes (N=2), and potential cross-language/fine-tuning biases.

Prospective research trajectories encompass:

• Composite Metrics: Integration of Jaccard overlap, human satisfaction scales, and perceived fairness.
• Cross-Cultural Generalization: Localized system prompts, polylingual participant pools.
• Scalable Multilateral Dialogues: Expansion to 5–15 users per session, group voting, parallel breakouts.
• Hybrid Orchestration: Alternation of LLM facilitation with human moderators or voting dashboards.
• Adaptive Learning: Leveraging rejection logs for continual model improvement.

These guidelines enable systematic deployment of consensus prompts that robustly and transparently drive agreement in diverse, scalable group settings (Triantafyllopoulos et al., 3 Feb 2025).

6. Broader Impact and Theoretical Significance

Automated consensus prompting with LLMs operationalizes a structured pathway for navigating stakeholder divergence, synthesizing user perspectives into measurable, high-alignment proposals, and iteratively refining agreement in multi-user environments. The methodology provides not only practical tools for digital deliberation and collective decision-making, but also a reproducible experimental testbed for benchmarking facilitation strategies, model architectures, and alignment metrics in computational social choice contexts.

By formalizing the interplay between adaptive prompt engineering, consensus-seeking strategies, and rigorous quantitative evaluation, this approach delineates both the capabilities and the limitations of current LLM-based consensus technologies, enabling the field to address persistent challenges in computer-mediated group dynamics and automated negotiation (Triantafyllopoulos et al., 3 Feb 2025).