CivAgent: Autonomous Social Choice & Game Agents

• CivAgent is an autonomous system blending agent-mediated social choice with LLM-driven game strategies to simulate human-like decision-making in complex environments.
• It integrates modules for preference learning, deliberation, and secure voting to represent citizen policies and maintain transparency.
• It leverages multi-step chain-of-thought reasoning and simulation-based rollouts to optimize strategies in competitive digital games.

CivAgent refers to two major lines of research: agent-mediated social choice systems, where autonomous software proxies deliberate and vote on behalf of human principals in democratic processes, and LLM-based autonomous agents for human-like interaction and decision making in strategy games. Both reflect the convergence of artificial intelligence, multiagent systems, and algorithmic social reasoning.

1. Definitions and Conceptual Scope

In agent-mediated social choice, a Civic Agent (or "voting avatar") is an autonomous agent that elicits, represents, and updates a citizen’s policy preferences, deliberates and negotiates with other agents, casts votes in collective decision processes, and provides human-readable explanations for its actions. Formally, a Civic Agent for citizen $i$ is specified as:

$$\text{CivAgent}_i = \langle U_i, R_i, D_i, M_i, V_i \rangle$$

where $U_i$ is the (compact) preference representation, $R_i$ is the reasoning engine (including strategic voting), $D_i$ is the deliberation/negotiation component, $M_i$ is the machine learning updater, and $V_i$ is the voting interface (Grandi, 2018).

In the context of LLM-based digital players, CivAgent refers to a pipeline coupling an LLM (such as GPT-3.5-turbo) to decision workflows in a complex, multi-agent strategy environment (e.g., the open-source "Unciv" game). Here, CivAgent parses game state, uses retrieval-augmented generation (RAG) memory, plans actions via chain-of-thought and simulations, engages in natural language negotiation, and stores reflection experience for iterative improvement (Wang et al., 28 Feb 2025).

2. System Architecture and Core Components

Agent-Mediated Social Choice

The architecture incorporates:

• User-Interface Module: Citizen authentication, agenda presentation, explanations, and override feedback.
• Preference Learning & Representation: Models and updates the citizen’s preferences; exposes API for queries.
• Deliberation & Negotiation: Argumentation-based protocols and message passing among agents.
• Voting & Aggregation: Implements configurable voting rules (e.g., Borda, Condorcet) and computes collective outcomes.
• Explanation & Transparency: Maintains a verifiable audit trail, generates human-readable rationales, and links to data sources.
• Security, Privacy, Audit: Encrypts user data, supports zero-knowledge proofs, enables third-party procedural verification (Grandi, 2018).

LLM-Based Digital Player

CivAgent’s architecture for the Unciv testbed follows a seven-step loop:

1. Extract structured observation $s_t$ from a serialized game state.
2. Retrieve relevant memory via RAG. 3-6. Multi-step chain-of-thought reasoning interleaved with tool-based rollouts.
3. Emit concrete actions, mapped to HTTP calls in the game engine.

After each phase, a reflection module analyzes the agent's trajectory and stores distilled experience ("experience chunks") in long-term memory for future retrieval (Wang et al., 28 Feb 2025).

3. Representation, Reasoning, and Learning

Preference Models and Deliberation (Social Choice)

To address combinatorial policy spaces, CivAgent employs:

• Logical Ballots: Ballots as truth assignments to propositions, with logical consistency constraints.
• CP-nets: Conditional Preference networks encoding partial orders over multi-issue domains.
• Weighted Utility Functions: Linear additive utility functions

$U_i(X) = \sum_{j=1}^{k} w_{i,j}\,u_{i,j}(X_j)$

Each module is computationally matched to tractable (e.g., tree-structured CP-nets) or expressive (e.g., judgment aggregation) domains (Grandi, 2018).

Learning mechanisms include Bayesian update of parameters $\theta_i$, reinforcement learning (Q-learning or policy gradients) for strategic adaptation, and supervised learning from citizen feedback (“Override”) with classifiers such as SVM or logistic regression. Social influence is modeled via trust scores $t_s$, updated through feedback or PageRank-style reputation dynamics.

Reasoning in Digital Games

CivAgent’s game workflow is modeled as an episodic MDP with state $s_t \in \mathcal S$ and action $a_t \in \mathcal A_t$. Actions encompass unit moves, city productions, tech choices, and diplomacy skills. The agent leverages dense reward functions:

$$R(s_t, a_t) = \text{Score}(s_{t+1}) - \text{Score}(s_t)$$

with detailed score decompositions reflecting multiple in-game metrics.

Decision making uses an LLM-driven chain-of-thought pipeline: propose multiple candidate skills, simulate $N$-turn rollouts for each, rerank based on prospective score deltas, and reflectively augment long-term memory. Dialogue and negotiation protocols support multi-turn bargaining and deception (with separate "skill proposal" and "skill response" workflows in prompt engineering). The reflection module synthesizes and archives "experience chunks" for bootstrapping future interactions (Wang et al., 28 Feb 2025).

4. Linguistic Interaction, Negotiation, and Social Reasoning

CivAgent’s dialogic capabilities span prompt engineering, memory management, and structured negotiation:

• Prompt Structure: Background, turn context, role profile, event log, and JSON-encoded skills form the LLM input.
• Memory: Short-term memory holds recent utterances; long-term memory uses RAG for retrieval.
• Negotiation: Multiround bargaining games with binary-search over simulator for bottom line computation; agents exchange offers with ±20% noisy hints of counterpart’s minima or maxima. Negotiation success is measured by normalized agreement within offered range.
• Deception: Agents may broadcast false claims; adversarial detection tasks measure false-negative (misinformation acceptance) rates.

A sample dialogue—presented in the source data—demonstrates sophisticated, context-aware negotiation reflecting both role-specific preferences and broader strategic context (Wang et al., 28 Feb 2025).

5. Voting, Aggregation, and Agent-Based Consensus

In social choice settings, CivAgent supports multiple voting rules:

• Plurality: Chooses the alternative with maximum votes.
• Borda Count: Aggregates ranked scores

$S(a) = \sum_{i=1}^n (m - \mathrm{rank}_i(a))$

• Condorcet: Identifies alternatives defeating all others in pairwise majority contests; computational cost $O(n m^2)$.
• K-Approval and Scoring Rules: Accept multiple approvals or weighted scores by position.

The aggregation function $f$ maps the profile of ballots to outcome, parameterized by social welfare objectives and robustness to strategic manipulation. Deliberation occurs through abstract argumentation frameworks (AFs), with protocols for proposal, attack, defense, acceptance/rejection, and commitment over bounded rounds (Grandi, 2018).

In the game AI context, CivAgent synthesizes voting-like consensus by combining LLM-proposed plans with simulated rollouts, but does not explicitly instantiate classical social choice protocols.

6. Evaluation, Performance, and Limitations

Game-Playing Agents

Empirical evaluation in Unciv involved 50 four-player games with variants of CivAgent. Table 1 from the source reports that the CivAgent-SR configuration (integrating both Simulator and Reflection modules) achieves a substantially higher average score (39.2) than ablated versions (17.6–24.9), also achieving the highest frequency of "SeekPeace" diplomatic skills (38.1%). Negotiation mini-games reveal GPT-4 is the strongest among evaluated LLMs but lags human expert negotiators; deception games show that LLM deceivers sometimes outperform humans, while LLM detectors are weaker (Wang et al., 28 Feb 2025).

Social Choice Agents

Complexity analyses indicate that dominance evaluation in CP-nets is PSPACE-complete (tractable for tree-nets). Voting rule aggregation varies from $O(n m \log m)$ (Borda) to NP-hard in Kemeny aggregation, but admits constant-factor approximation for small $m$. Deliberation protocol communication is quadratic in argument set size, but practical bounded-depth runs reduce overhead. Bayesian and RL updates are $O(k)$ or $O(|\text{Action}|)$ per feedback, scaling to thousands of issues (Grandi, 2018).

Limitations

CivAgent’s prompt-engineered LLMs lack adaptive policy networks, causing numeric drift over extended planning horizons and susceptibility to collusion-style adversarial manipulation. Reflection is currently offline, with no in-game fine-tuning. Absence of world-model learning restricts dynamic adaptability. In social choice, strategic support is limited to standard response heuristics; strategic manipulation and equilibrium are reachable only under idealized transparency.

Proposed enhancements include integrating learned components (RL micro-control), enabling online fine-tuning from the data flywheel of gameplay logs and human feedback, and generalizing to new domains by substituting simulators and skill schemas (Wang et al., 28 Feb 2025).

7. Trust, Transparency, and Security

In democratic applications, CivAgent secures user data with encryption and zero-knowledge proofs, logs all inputs and votes in a blockchain-like audit trail, and enables third-party verification of correct adherence to voting protocols and preference updates. Explainability is enforced via summarization of premises and reasoning steps for each vote, with natural language presentations cross-linked to the argument DAG. Votes can be cast under threshold-homomorphic encryption (e.g., Paillier) to prevent disclosure of individual ballots (Grandi, 2018).

This architecture grounds both trustworthiness and transparency, critical for deployment in high-stakes digital democracy and collaborative game environments.

---

References:

• Agent-mediated social choice systems and blueprint: (Grandi, 2018)
• LLM-based CivAgent for Unciv: (Wang et al., 28 Feb 2025)