Generative Emergent Communication

• Generative Emergent Communication is a paradigm in which artificial agents jointly develop internal world models and shared symbol systems using decentralized Bayesian inference and variational learning.
• The approach employs mechanisms like the Metropolis–Hastings naming game to achieve robust, iterative negotiation over latent generative models, enabling symbol alignment without explicit supervision.
• Emergent symbol systems support compositionality and cross-modal generalization, leading to zero-shot capabilities and competitive performance on clustering and mutual intelligibility metrics.

Generative Emergent Communication (EmCom) is a theoretical and empirical paradigm in which artificial agents—typically modeled as neural or probabilistic systems—jointly discover both an internal world model and a shared symbol system through decentralized, inference-based processes. Unlike discriminative emergent communication, where agents are optimized to map observations to pre-specified labels or actions via backpropagation or direct policy differentiation, generative EmCom casts communication as collective negotiation and sampling over latent generative models, drawing on principles from Bayesian inference, variational learning, and collective predictive coding. This approach seeks to explain not only the co-development of inter-agent language and conceptual structure but also the mechanisms underlying symbol grounding, compositionality, and the societal-level emergence of language-like protocols (Taniguchi et al., 2024, Taniguchi et al., 2022, Brandizzi, 2023, Hagiwara et al., 2021).

1. Theoretical Foundations: Decentralized Generative Modeling and Collective Predictive Coding

Central to generative EmCom is the concept of decentralized generative modeling. Agents each maintain a local generative model $p(x^i \mid z^i)p(z^i)$ over their private observations $x^i$ and latent states $z^i$. Communication emerges through inference over shared messages $m$, which serve as global latent causes linking the agents' individual world models. This process is formalized through a joint model: $$p\bigl(x^{1:N},z^{1:N},m\bigr) = p(m) \prod_{i=1}^N p(x^i \mid z^i) p(z^i \mid m).$$ The approximate posterior adopted for decentralized inference is: $$q\bigl(z^{1:N},m \mid x^{1:N}\bigr) = q(m \mid z^{1:N}) \prod_{i=1}^N q(z^i \mid x^i).$$ Each agent attempts to minimize variational free energy (negative evidence lower bound, ELBO), driving local posteriors toward agreement and facilitating the convergence of emergent symbol systems through mechanisms akin to a Metropolis-Hastings (MH) naming game (Taniguchi et al., 2024, Taniguchi et al., 2022).

Collective predictive coding (CPC) extends the single-agent predictive coding principle to the multi-agent domain. In CPC, language and symbol systems emerge to minimize society-wide free energy, integrating distributed sensory signals and latent inferences by means of communicative exchanges.

2. Algorithmic Frameworks: The Metropolis–Hastings Naming Game and Variational Protocols

A canonical implementation is the MH naming game, where agents alternate as "speaker" and "listener." Each proposes a candidate symbol $w$ based on local posteriors, and the listener probabilistically accepts or rejects the proposal by evaluating the likelihood ratio

$$r = \min \left(1, \frac{p(z^{Li} \mid \phi^{Li}, w^{Sp})}{p(z^{Li} \mid \phi^{Li}, w^{Li})} \right).$$

Over repeated exchanges, this decentralized stochastic negotiation provably converges to valid samplers of the joint posterior $p(w, z, \theta, \phi \mid o)$, allowing symbol systems and categories to emerge without explicit external rewards or supervision (Taniguchi et al., 2022, Hagiwara et al., 2021). Extensions generalize these mechanisms to multi-agent and multimodal settings, such as the interpersonal multimodal Dirichlet mixture (Inter-MDM), where shared symbols mediate latent category alignment across distinct perceptual modalities (Hagiwara et al., 2021).

In discriminative emergent communication—exemplified by referential games, REINFORCE policy gradients, and Gumbel-Softmax relaxations—messages $m$ are optimized directly for task reward. Generative EmCom, by contrast, replaces the reward mechanism with decentralized free energy minimization and Bayesian sampling procedures. Control-as-inference (CaI) formulations further bridge the probabilistic planning and communication processes by treating coordinated actions as inference conditioned on optimality variables $o$ (Taniguchi et al., 2024).

3. Representative Architectures and Empirical Protocols

Prominent architectures include VAEs with mixture priors (e.g., inter-GMM+VAE), codebook-based bottlenecks (as in Composition through Decomposition, CtD), and hybrid frameworks unifying discriminative self-supervised learning with generative latent variable inference (e.g., SimSiam+VAE) (Taniguchi et al., 2022, Carmeli et al., 15 Jan 2026, Hoang et al., 2024). Typical system components are:

• Generative encoders: Map raw observations to latent spaces, often via convolutional encoders or quantized embeddings.
• Message bottlenecks: Discrete or continuous message spaces, enforced via GMMs, codebooks, or variational quantization.
• Negotiation protocols: Alternating proposal-acceptance routines (MH), turn-taking, and referential exchanges.
• Mutual inference: Optional periodic sharing of prior statistics to refine cross-agent posteriors.
• Losses: Free energy, ELBO, KL-divergence on shared messages, task loss (for referential accuracy), and compositionality-inducing objectives (e.g., codebook loss).

The CtD paradigm utilizes a two-stage curriculum—first decomposing images into atomic codebook-concepts through multi-target games, then composing these to describe novel composite stimuli. This scaffolding results in zero-shot compositional generalization, a hallmark of productive generative communication (Carmeli et al., 15 Jan 2026).

The SimSiam Naming Game (SSNG) employs a VAE-integrated SimSiam predictor and alternates dynamic speaker/listener roles, leveraging both discriminative and generative pathways to achieve robust latent alignment and emergent language synchrony (Hoang et al., 2024).

The Inter-MDM model demonstrates the emergence of shared semiotic systems under multimodal categorization with only MH naming-game exchanges—no direct latent or gradient transfer—enabling robust cross-modal and interpersonal inference (Hagiwara et al., 2021).

4. Evaluation Metrics, Empirical Results, and Observed Phenomena

Generative EmCom evaluations span both emergent communication structure and downstream task efficacy. Key quantitative metrics include:

• Adjusted Rand Index (ARI): Clustering accuracy of induced categorical structure.
• Cohen’s κ: Alignment in shared symbol assignments.
• Topographic Similarity: Correlation between semantic and emergent message distances.
• Mutual Information: Signal-message coupling.
• Compositionality Measures: Context independence (CI), positional disentanglement (POS), bag-of-symbols disentanglement (BOS), concept best matching (CBM), and adjusted mutual information (AMI).
• Zero-shot Generalization: Accuracy on novel compositional tasks or object combinations (Carmeli et al., 15 Jan 2026).

Empirical findings across models demonstrate that generative protocols nearly match or surpass centralized supervised/topline baselines in ARI and mutual intelligibility (e.g., ARI $\approx 0.78$ for the MH naming game on MNIST, κ $\approx 0.92$ for Fruits360) (Taniguchi et al., 2022). Zero-shot composition is enabled by codebook-based architectures and is observed to achieve near-perfect context-independence in multi-concept tasks (Carmeli et al., 15 Jan 2026). SimSiam+VAE and SSNG approaches reach comparable semantic alignment and classification performance as referential game baselines and slightly outperform non-generative protocols in topographic similarity (Hoang et al., 2024).

Qualitative analyses include prototype reconstruction from emergent symbols, latent space alignment visualizations, and agent ability to reconstruct unobserved cross-modal data purely from shared tokens (Hagiwara et al., 2021, Taniguchi et al., 2022).

5. Compositionality, Symbol Grounding, and Cross-modal Generalization

A defining feature of generative EmCom is its ability to realize compositional symbol systems, where complex meanings are systematically synthesized from atomic components. This property is directly supported by architectures with discrete codebooks and multi-target curriculum learning (CtD), as well as by protocols that enforce information bottlenecks and promote mutual information between input attributes and message structure (Brandizzi, 2023, Carmeli et al., 15 Jan 2026).

Emergent symbol grounding is achieved via generative agreement over latent causes rather than by discriminative supervision. The MH-based naming game paradigm, Bayesian cross-modal inference, and mutual generative modeling provide the platform for agents to negotiate and align not only symbols but also perceptual and conceptual categories across modalities. Shared signs thus mediate both intra- and inter-personal generalizations, enabling agents to infer unobserved features or modalities from received symbols—mirroring aspects of human semiotic reasoning (Hagiwara et al., 2021, Taniguchi et al., 2022, Carmeli et al., 15 Jan 2026).

6. Connections to World Models, Language Evolution, and LLMs

Recent theoretical advances frame LLMs as collective world models, trained to predict text via parameter estimation that implicitly marginalizes over collective latent variables drawn from the sum of all societal experience (Taniguchi et al., 2024). Generative EmCom provides a unifying account in which language emergence is a society-level CPC process: the societal negotiation and continual refinement of externalized symbol systems minimizes group-level predictive error, linking micro-level agent interaction to macro-level language evolution. Under this view, emergent artificial languages—and LLMs—are manifestations of distributed active inference across populations, with linguistic tokens serving as compressed representations of shared contextual state.

7. Open Problems, Limitations, and Future Directions

Key challenges include:

• Scaling population size and message expressivity: Current experiments mostly involve two agents, single-symbol messages, and low-complexity observation spaces.
• Generalization and robustness: Addressing population dynamics (birth/death), transfer/generalization to new domains or new partners, and adaptation to embodied or multimodal environments.
• Human-alignment and interpretability: Designing loss functions and protocols to steer emergent systems towards human-interpretable forms, avoiding degenerate or adversarial protocol drift (Brandizzi, 2023).
• Theoretical tools: Sharper formal models for population-level emergence, compositionality bounds, and sample efficiency, as well as principled frameworks for ethical protocol development (Brandizzi, 2023, Boldt et al., 2024).
• Integration with advanced generative architectures: Extension to more flexible priors (e.g., DP-GMM, flow-based models), structured variable-length protocols, and integration with embodied perception and action (Taniguchi et al., 2022, Carmeli et al., 15 Jan 2026).

A plausible implication is that principled generative EmCom frameworks—grounded in probabilistic modeling, CPC, and decentralized negotiation—will underpin next-generation artificial agents capable of scalable, robust, human-compatible communication, both among themselves and with humans, across a wide variety of environments and domains. Continued progress pivotally depends on the development of scalable, interpretable, and population-robust generative protocols that are empirically validated in complex, real-world scenarios.