Emergent Alignment via Competition (2509.15090v1)

Abstract: Aligning AI systems with human values remains a fundamental challenge, but does our inability to create perfectly aligned models preclude obtaining the benefits of alignment? We study a strategic setting where a human user interacts with multiple differently misaligned AI agents, none of which are individually well-aligned. Our key insight is that when the users utility lies approximately within the convex hull of the agents utilities, a condition that becomes easier to satisfy as model diversity increases, strategic competition can yield outcomes comparable to interacting with a perfectly aligned model. We model this as a multi-leader Stackelberg game, extending Bayesian persuasion to multi-round conversations between differently informed parties, and prove three results: (1) when perfect alignment would allow the user to learn her Bayes-optimal action, she can also do so in all equilibria under the convex hull condition (2) under weaker assumptions requiring only approximate utility learning, a non-strategic user employing quantal response achieves near-optimal utility in all equilibria and (3) when the user selects the best single AI after an evaluation period, equilibrium guarantees remain near-optimal without further distributional assumptions. We complement the theory with two sets of experiments.

• The paper introduces a game-theoretic framework that leverages competition among misaligned AI agents to achieve near-optimal human utility under alignment constraints.
• It formalizes conditions such as approximate weighted alignment and an identical induced distribution condition, with experiments showing significant MSE reductions on ETHICS and MovieLens datasets.
• The analysis extends to bounded rationality and best-AI selection games, offering practical insights for market design, auditing, and AI alignment protocol development.

Emergent Alignment via Competition: A Game-Theoretic Analysis of AI Alignment

Overview and Motivation

The paper "Emergent Alignment via Competition" (2509.15090) presents a formal game-theoretic framework for analyzing how competition among multiple misaligned AI agents can yield outcomes for a human user that are comparable to those obtained from a perfectly aligned agent. The central insight is that if the user's utility function lies approximately within the convex hull of the utility functions of available AI agents, then strategic competition can induce equilibria in which the user achieves near-optimal utility, even when no individual agent is well-aligned.

This approach is motivated by practical concerns: in real-world AI markets, users interact with models produced by different providers, each with their own incentives and biases. The paper formalizes the conditions under which the diversity of these models, combined with competitive dynamics, can compensate for imperfect alignment at the individual agent level.

Formal Model and Alignment Assumption

The interaction is modeled as a multi-leader Stackelberg game, extending Bayesian persuasion to multi-round conversations. The human user (Alice) seeks to maximize her utility $u_A(a, y)$ by choosing an action $a$ given an unknown state $y$. Each AI agent (Bob$_i$) has its own utility $U_i(a, y)$, potentially misaligned with Alice's. The key modeling assumption is approximate weighted alignment: there exist non-negative weights $w_i$ and offset $c$ such that

$$\sup_{a, y} \left| \left( \sum_{i=1}^k w_i U_i(a, y) + c \right) - u_A(a, y) \right| \leq \varepsilon$$

This condition becomes easier to satisfy as the diversity and number of available agents increases, and is justified both by market incentives (e.g., competing drug companies) and by stochasticity in model training.

Game Structure and Equilibrium Analysis

The agents commit to conversation rules (fixed policies for message generation), and Alice best-responds by choosing her own conversation and decision rules. The analysis focuses on Nash equilibria of the induced game, with the main benchmark being the utility Alice could achieve with a perfectly aligned agent.

Identical Induced Distribution Condition

A central technical condition is that the outcome distribution induced by any agent deviating to the Alice-optimal strategy is independent of the agent's identity. This holds, for example, when the message space is rich enough for agents to fully reveal their private information, allowing Alice to compute her Bayes-optimal action.

Main Theoretical Results

• Optimality in Equilibrium: If the identical induced distribution condition holds and the weighted alignment assumption is satisfied, then in any Nash equilibrium, Alice's expected utility is at least $U_A(C_B^*) - 2\varepsilon$, where $C_B^*$ is the optimal conversation rule for a perfectly aligned agent.
• Bounded Rationality and Quantal Response: The analysis is extended to settings where Alice uses quantal response (softmax over expected utilities) rather than strict best response. Under the information substitutes condition (which ensures that information from different agents is substitutable for utility estimation), Alice can achieve near-optimal utility with explicit bounds depending on alignment error, estimation error, and the quantal response gap.
• Best-AI Selection Game: If Alice selects the single best agent after an evaluation period, the equilibrium guarantees near-optimal utility without additional distributional assumptions.

Empirical Validation

The paper provides two forms of empirical evidence:

Convex Hull Alignment Experiments

Experiments on the ETHICS and MovieLens datasets demonstrate that the best utility function in the convex hull of multiple LLM-generated utility functions is substantially better aligned to the "human" utility than any individual agent.

Figure 1: Alignment error (MSE) decreases as more agents are added to the convex hull on the ETHICS dataset.

Figure 2: Alignment error (MSE) decreases as more agents are added to the convex hull on the MovieLens dataset.

Key findings:

• At $K=100$ agents, non-negative weighted combinations reduce MSE by 50–75% compared to the best individual agent.
• Simple averaging performs poorly; optimal convex combinations are sparse and exploit diversity.

Strategic Equilibrium Simulations

Simulations of the Best-AI Selection Game using synthetic and MovieLens utility tables show that competition among misaligned agents reliably improves Alice's utility when the convex hull condition is satisfied.

Figure 3: Minimum Alice utility at equilibrium versus misalignment score ($\varepsilon$) for synthetic utility tables.

Figure 4: Alice's utility in equilibrium reached via best response dynamics compared to the minimum possible utility.

Figure 5: Alice's minimum utility as a function of increasing number of agents, showing monotonic improvement.

Observations:

• Alice's utility in equilibrium is tightly bounded below by the theoretical $OPT - 2\varepsilon$.
• Increasing the number of agents improves alignment and utility, with significant gains even in small markets.

Implications and Future Directions

Practical Implications

• Market Design for Alignment: The results suggest that fostering diversity and competition among AI providers can be a robust mechanism for achieving alignment, even when individual models are imperfect.
• Auditing and Regulation: Auditing collections of models for the convex hull condition could be a practical approach to alignment assurance.
• Protocol Design: The framework motivates the design of interaction protocols and market mechanisms that exploit competitive dynamics for alignment.

Theoretical Extensions

• Pluralistic Alignment: Extending the model to heterogeneous user populations, where each user has a distinct utility function, is a natural next step.
• Dynamic and Stateful Agents: As AI agents become more dynamic and stateful, models without commitment (e.g., cheap talk) may become relevant.
• Computational Tractability: The equilibrium computation is challenging; future work should address algorithmic aspects and robustness to bounded rationality.

Conclusion

This work provides a rigorous foundation for understanding how competitive dynamics among misaligned AI agents can induce emergent alignment for users. The convex hull condition is both theoretically justified and empirically validated, and the game-theoretic analysis yields explicit performance guarantees. The approach opens new avenues for mechanism design, auditing, and protocol development in AI alignment, with significant implications for both theory and practice.