Internal Pluralism and the Limits of Pairwise Comparisons

Abstract: Local pairwise comparisons are a standard tool for learning how people want decision rules to work, e.g., in participatory design or alignment. However, their use builds in two strong assumptions: that local comparisons are sufficient evidence about how a person wants an automated decision rule to behave, and that people can always answer those comparisons decisively. We investigate how these assumptions may be compromised under internal pluralism: the idea that an individual evaluates decision rules according to multiple authoritative priorities about how the rule should behave. We provide a formal model of such pluralistic preferences over decision rules, which then lets us identify two distinct failures of forced local pairwise comparison data. First, priorities such as proportionality, egalitarianism, and equal treatment are inherently global: what they imply in one case can depend on what happens elsewhere, so local comparisons may fail to capture them. Second, even when priorities are representable locally, tension between strongly-held priorities can generate internal conflict, producing potentially costly behavioral distortions when comparisons are forced. We then use our model to investigate the alternative -- allowing people to report indecision -- and our findings suggest that doing so can considerably reduce the number of queries needed to learn preferences accurately. We conclude by describing how our model points toward preference-learning methods that elicit these priorities directly, yielding more faithful and interpretable accounts of what people value.

• The paper introduces a pluralistic priority model that represents multiple, inseparable normative priorities in decision making.
• It demonstrates that forced pairwise comparisons can erase or distort indecisive and inseparable value signals, leading to suboptimal model learning.
• Empirical experiments show that allowing indecision reporting reduces sample complexity and improves active learning efficiency.

Essay: Internal Pluralism and the Limits of Pairwise Comparisons (2607.02672)

Introduction and Problem Setting

The paper "Internal Pluralism and the Limits of Pairwise Comparisons" (2607.02672) addresses a central methodological assumption in empirical preference learning: that forced local pairwise comparisons sufficiently capture an individual's preferences over complex decision rules. This approach, ubiquitous in preference learning for AI alignment, participatory design, and socially consequential algorithmic systems, presupposes two key conditions: (1) people's global objectives can be reduced to a series of local, decomposable comparisons, and (2) individuals are able to resolve every such comparison without indecision.

These premises neglect two important aspects of human normative reasoning: internal pluralism, where multiple values may be co-activated and come into conflict, and inseparability, where the acceptability of a choice at a given input depends on the rule's behavior elsewhere. The paper develops a mathematical framework for representing internal pluralism over decision rules and demonstrates theoretically and empirically that forced pairwise comparisons both erase inseparable priorities and obscure informative states of individual indecision.

Pluralistic Priority Model

The authors introduce a representation of individual normative reasoning as a pluralistic priority model. Instead of a single transitive preference ordering over decision rules, the model encodes $m$ distinct, fully-fledged (complete, transitive, weak) orderings—each called a "priority" and represented by a utility function $u_j: \mathcal{F} \to \mathbb{R}$—together with a weight vector $\omega \in \Delta_{m-1}$. This structure allows conflict or indifference across priorities at both the rule and query level, reflecting the empirical reality of value pluralism, moral conflict, and ambivalence.

For example, in the AI alignment context, a subject may simultaneously weight size (maximizing number saved), family (saving relatives), and proportionality (group fairness) in the trolley problem. The model admits both inseparable priorities (the rule’s behavior on one input depends on outputs elsewhere) and pluralistic conflict (different priorities providing irreconcilable evidence for a given local query).

Theoretical Results: Inseparability and Misspecification

A central formal contribution is the characterization of inseparability and its impact on downstream learning. A priority is inseparable if its evaluation of a local query cannot be divorced from the background rule's behavior at other inputs. When forced pairwise comparisons are used to elicit preferences, inseparable priorities cannot be detected or reconstructed by any algorithm restricted to such local queries.

The analysis shows that standard score-based random utility models (S-RUMs)—including Bradley-Terry and Thurstone-Mosteller models—are equivalent to the degenerate case where all priorities are perfectly separable and the individual is always decisive. In this regime, forced comparisons are valid. Departures from separability, however, induce two failure modes:

• Erasure of Perfectly Inseparable Priorities: If a true priority is perfectly inseparable, the empirical effect is that its influence is eliminated in the learned model—local queries generate no identifying signal, and it becomes impossible to determine its importance, no matter how much data is collected.
• Distortion of Generic Inseparable Priorities: Priorities that are not perfectly inseparable, but which still violate separability, produce behavioral signals that are misinterpreted by standard S-RUM estimation, often yielding aggregated optima that are highly suboptimal—sometimes approaching the worst possible regret under the true model.

These results generalize beyond specific models: the class of linear aggregators and even some non-linear, scale-preserving aggregators are affected, as local pairwise query responses cannot distinguish inseparable priorities or recover their weights.

Indecision and Latent States

Beyond inseparability, the paper formalizes the behavioral and statistical implications of latent indecision—internal states where no priority provides clear support, or where priorities provide equally strong but opposing evidence. The model allows explicit indecision (indifference or conflict), generalizing the random-utility model approach where all indecision collapses into noise.

Empirical simulations and re-analysis of qualitative data (e.g., interviews in moral dilemmas and resource allocation studies) substantiate the behavioral relevance of these states: many participants report conflict, discomfort, or explicit refusal to choose when presented with forced binary queries.

Numerical Experiments: Active Learning under Indecision

The paper presents active learning experiments in multi-feature allocation settings, with pluralistic priorities and query-level indecision regimes. Key findings include:

• Behavioral deviations in forced query response (lexicographic, random, self-similarity tie-breaking) produce substantial inferential errors in weight recovery (14-24% of maximum possible error) and, critically, can lead to poor worst-case utility loss (17-39% of maximum range).
• Average regret is less sensitive to forced tie-breaking, but remains nontrivial.
• Allowing individuals to report indecision (either differentiating between indifference/conflict, or as a generic state) reduces sample complexity, accelerates convergence under active learning, and avoids systematic distortions. In moderate to high indecision regimes, the difference in learning speed and regret is a full order of magnitude in query budgets—highly significant given practical elicitation constraints.
• The benefits of richer response alphabets persist when assuming unknown or misspecified noise models, suggesting robustness in practical deployment.

Implications and Future Directions

The findings have both practical and foundational implications:

• For empirical preference learning in AI alignment and other domains, reliance on forced local pairwise comparisons is theoretically unsound in the presence of internal pluralism and inseparable priorities. This directly challenges standard pipelines in e.g., reward modeling for LLM alignment, participatory algorithmic design, and resource allocation.
• Practitioners should enable reporting of indecision (at minimum), and ideally, seek to elicit richer structural information about the respondent's priorities and their interactions.
• Statistically, allowing pluralistic indecision yields improved identifiability and efficiency in active learning, and provides information about uncertainty at the rule level, not just the query level.
• Theoretically, the limits identified here show that no empirical refinement of local pairwise comparison protocols can overcome the barrier for perfectly inseparable priorities: rich, higher-order elicitation and representation are unavoidable for full fidelity preference modeling.

As a concrete proposal, the paper outlines priority-aware learning, where priorities are directly elicited (e.g., via textual or structured queries) and used to scaffold the interpretation of local comparisons. Learning proceeds over interpretable, user-specified dimensions of value, with direct benefit for transparency, downstream consensus-building, and explainability. However, this paradigm shift would require developing reliable protocols for priority elicitation and textual-to-formal mapping at scale.

Conclusion

This work exposes theoretical limitations of local pairwise comparison-based preference elicitation in representing internal pluralism and inseparable normative priorities. The results strongly suggest that alignment and participatory design in AI systems require richer models of individual and collective value, explicit support for indecision, and new learning protocols emphasizing priority elicitation and compositional representation. The framework provided here offers a mathematically rigorous foundation for such efforts, with clear guidance for the limitations of forced comparison pipelines and a roadmap for more faithful, interpretable, and efficient preference learning.