Algorithmic Approaches to Opinion Selection for Online Deliberation: A Comparative Study

Abstract: During deliberation processes, mediators and facilitators typically need to select a small and representative set of opinions later used to produce digestible reports for stakeholders. In online deliberation platforms, algorithmic selection is increasingly used to automate this process. However, such automation is not without consequences. For instance, enforcing consensus-seeking algorithmic strategies can imply ignoring or flattening conflicting preferences, which may lead to erasing minority voices and reducing content diversity. More generally, across the variety of existing selection strategies (e.g., consensus, diversity), it remains unclear how each approach influences desired democratic criteria such as proportional representation. To address this gap, we benchmark several algorithmic approaches in this context. We also build on social choice theory to propose a novel algorithm that incorporates both diversity and a balanced notion of representation in the selection strategy. We find empirically that while no single strategy dominates across all democratic desiderata, our social-choice-inspired selection rule achieves the strongest trade-off between proportional representation and diversity.

• The paper benchmarks seven selection strategies across five democratic metrics and finds that DiverseBJR provides the strongest representation gains at small slate sizes while maintaining good coverage and near-Engagement consensus.
• DiverseBJR combines Balanced Justified Representation with approval-vector diversity constraints, reducing redundant selections without requiring sensitive demographic attributes or political labels.
• The results, replicated on Remesh and Habermas Machine data, show that fairness constraints need not reduce consensus, while LLM-based selection may fail basic reliability requirements such as returning the requested slate size.

Problem and Motivation

Online deliberation platforms such as Pol.is, Remesh, and make.org routinely confront a selection problem: participants produce hundreds of free-form opinions, and facilitators must distill them into a small, representative slate that can be digested by stakeholders or fed back into subsequent deliberation rounds. This task is increasingly automated, yet the choice of algorithmic strategy—consensus-seeking, diversity-maximizing, bridging-based—embeds normative commitments about what counts as "representative" without being transparent to participants. Consensus-oriented strategies risk flattening minority voices; diversity-oriented strategies lack any guarantee of proportionality. The paper addresses this gap by benchmarking existing selection algorithms against democratic desiderata and proposing a novel criterion, DiverseBJR, that combines balanced proportional representation with diversity guarantees derived from social choice theory (2602.15439).

Theoretical Framework

The authors adopt the multiwinner approval voting framework of Elkind et al., adapted to deliberation by prior work: given $n$ users each approving a subset of $m$ opinions, the task is to select a slate $S$ of size $k$ under an (additive) scoring rule $f$. Three axiom families are operationalized:

• Justified Representation (JR): every cohesive group of at least $n/k$ users has at least one member approving some selected opinion.
• Balanced Justified Representation (BJR): there exists an assignment $\omega$ of users to selected opinions such that each opinion is assigned either $\lfloor n/k \rfloor$ or $\lceil n/k \rceil$ users, and no large cohesive group unanimously approves an unassigned opinion. BJR corrects a known weakness of JR: a few highly popular opinions can satisfy JR while leaving most of the slate unconstrained—the paper cites evidence that for $k=8$, JR-sets are typically of size 1–2, leaving roughly 75% of the slate without representation guarantees.
• DiverseBJR (novel): BJR constraints plus local $m$0-diversity, defined via approximate clones: no selected opinion can be replaced by an unselected non-$m$1-neighbor while preserving BJR. Diversity relies solely on approval vectors (normalized Hamming distance), requiring no sensitive attributes.

The paper does not prove existence of DiverseBJR whenever BJR holds; instead it provides a GreedyCC-style approximation and uses a greedy feasibility simulator as a sufficient check before excluding $m$2-neighbors. Exact BJR verification via ILP is NP-hard and impractical at platform scale, motivating the approximation.

Evaluation Methodology

Five metrics operationalize democratic criteria: overall unrepresentation ($m$3), median group-level unrepresentation across political groups (a robust alternative to Rawlsian max-min), consensus (maximin cross-group approval), coverage gap (largest distance from any unselected opinion to its nearest selected neighbor), and redundancy (fraction of near-duplicate opinions within the slate). Baselines include Engagement (utilitarian rule), Bridging (cross-group agreement / CGA, as used by Community Notes and Pol.is), greedy maximin Diversity, greedy approximations of JR and BJR, random selection, and a zero-shot GPT-4.1-mini baseline.

Experiments use the publicly available Remesh "Polarized Issues Dataset" (six questions on protest rights, ~100–307 participants per question), split into consensual (random $m$4 leaves ≤5% unrepresented) and controversial (≥20%) questions, with missing votes inferred probabilistically following the platform's pipeline. Findings are validated on the Habermas Machine Virtual Citizen Assembly dataset (~200 UK participants, fully observed approvals), controlling for artifacts of vote inference.

Empirical Results

No single strategy dominates all desiderata, consistent with known NP-hardness results for jointly optimizing relevance and diversity, and for optimizing welfare or coverage under JR. The principal findings on controversial questions are:

Strategy family	Representation	Coverage	Consensus	Redundancy
Engagement	comparable to or worse than random	worst gaps	moderate	high
Bridging	strong early gains, behind constraint methods	good	best (by design)	moderate
Diversity	no guarantees	lowest CG	—	lowest
JR / BJR	strongest early gains	good	similar to Engagement	low at small $m$5
DiverseBJR	strongest gains at small $m$6	good	similar to Engagement	second only to Diversity

Three results deserve emphasis. First, DiverseBJR delivers the largest representation gains when the slate is small ($m$7–3), which is precisely the regime where digestibility constraints bind in practice. Second, enforcing proportional representation entails little consensus loss: JR/BJR/DiverseBJR select opinions whose cross-group approval resembles Engagement's, suggesting that fairness constraints do not force the selection of polarizing content when broadly acceptable opinions exist. Third, DiverseBJR spreads its selections across the opinion space (visualized via MDS embeddings), whereas Engagement selects clustered, redundant items. On consensual questions all methods improve, as expected when most opinions are widely approved.

A notable negative result concerns the LLM baseline: GPT-4.1-mini frequently returned fewer than $m$8 opinions despite explicit prompting, making cardinality-sensitive metrics incomparable; it was excluded from the main comparison and performed worse than most baselines where reported. This is a concrete, documented reliability failure of LLM-based slate selection.

Limitations and Open Questions

Several limitations are conceded explicitly. All results are offline/observational; whether the measured trade-offs persist under live deployment, where selection influences subsequent participation and opinion formation, is untested—the authors propose online deployment studies analogous to social-media intervention experiments. Existence of DiverseBJR whenever BJR is satisfiable is not established, and the feasibility checker returns "False" inconclusively when the greedy simulator fails. The diversity threshold $m$9 is data-dependent: sensitivity analyses show performance improves monotonically with $S$0 across all questions but that the optimum remains dataset-specific, though DiverseBJR improves over plain BJR for every tested value. Bridging requires political-leaning attributes and inherits their quality limitations, which also precluded its evaluation on the validation set. Finally, binary thresholding of inferred votes at 0.5 introduces discretization assumptions that, while mitigated by the dense-vote replication, remain a modeling choice.

Conclusion

This work contributes a systematic benchmark of algorithmic opinion-selection strategies against five quantified democratic desiderata, and a social-choice-grounded algorithm, DiverseBJR, that achieves the strongest empirical trade-off between balanced proportional representation and diversity at small slate sizes. The central finding—that representation-constrained selection sacrifices little consensus while avoiding the minority-erasure inherent to popularity- and consensus-driven rules—is replicated on two independent datasets with different vote-collection regimes. The released codebase and metric suite provide a reusable benchmark for future algorithmic and deployment-based studies in digital democracy.