---
title: Referential Games Overview
url: https://www.emergentmind.com/topics/referential-games
type: topic
---

# Referential Games Overview

A referential game is a formal communicative interaction in which one agent (often called the "speaker" or "sender") observes a target referent and emits an utterance, while another agent (the "listener" or "receiver") observes the utterance and attempts to select the correct referent from a candidate set. Referential games provide a multipurpose, highly structured setting for investigating topics such as language emergence, compositionality, pragmatic inference, theory of mind, grounding, and alignment between agents with different perceptual or conceptual spaces. In recent years, extensive research in machine learning, linguistics, and cognitive science has utilized referential games as testbeds for emergent communication, systematic generalization, population dynamics, and human–machine interaction.

## 1. Formal Structure and Core Variants

The canonical referential game is defined as a tuple of agents, referent set, message space, and decision rules. Let $\mathcal{O}$ denote the set of possible objects (which can be symbolic attribute vectors, images, or real-world entities), and let $\mathcal{M}$ be the discrete message space. Each round proceeds as follows:

- A context $C = \{o_1, \dots, o_K\} \subseteq \mathcal{O}$ is sampled (e.g., one target, $K-1$ distractors).
- A target $t \in C$ is designated.
- The speaker observes $t$ (or, in some variants, the full context) and generates $m \sim \pi_S(\cdot \mid t)$.
- The listener observes $(m, C)$ and selects $\hat{t} \sim \pi_L(\cdot \mid m, C)$.
- Payoff is $1$ if $\hat{t} = t$, $0$ otherwise, or more generally a reward $u(\hat{t}, t)$.

Variants include:
- **Attribute-based:** The message space consists of attributes aligned with object factors [1910.04872].
- **Symbolic or visual:** Inputs are symbolic vectors [2012.02875, 1804.03984] or raw pixels [2304.14511].
- **Multi-modal:** Sender and listener may have access to different information modalities (e.g., image vs. text) [1705.10369].
- **Population games:** Pools of agents with varying perceptual, conceptual, or architectural profiles [2001.03361, 1910.04872].
- **Iterated/multi-step:** Communication unfolds over multiple back-and-forth utterances [1705.10369, 2511.03908, 2309.05162].
- **Hierarchical and set/concept games:** Targets are abstracted over multiple levels, or generalizations over sets are communicated [2203.13176, 2106.02668].

This formalization permits the specification of agent architectures, learning paradigms (policy-gradient, Gumbel-Softmax relaxation, meta-learning), and evaluation metrics.

## 2. Model Architectures and Learning Paradigms

Neural referential game agents are typically parameterized by encoders, decoders, and policy modules. Common features include:

- **Speaker (Sender):** Maps target input (image or vector) to a latent representation, then decodes to a message (e.g., via MLP, LSTM, or transformer) [2012.02875, 1705.10369, 1910.04872].
- **Listener (Receiver):** Encodes candidate objects and translates message to a hidden representation; decision made via similarity scoring (dot product) or MLP [2012.02875, 2304.14511].
- **Multi-agent and population settings:** Listeners are parameterized by conceptual understanding profiles or thresholds, allowing for modeling of inter-agent variation and adaptation [1910.04872, 2001.03361].
- **Structured Memory and Pragmatics:** Recurrent modules and explicit reference-resolution models support multi-turn and collaborative reasoning [2109.05042].

Training objectives differ based on communication constraints:
- **REINFORCE:** Policy gradient for discrete sampling [2012.02875, 1804.03984].
- **Straight-Through Gumbel-Softmax (STGS):** Enables end-to-end backpropagation through discrete message channels [2012.02875, 2012.10776, 2304.14511].
- **Obverter:** Speaker internally reasons via its own model of the listener to maximize mutual intelligibility [2304.14511].
- **Meta-learning:** Agents adapt quickly to new semantic spaces via meta-referential games [2207.08012].

The interaction between architecture, inductive bias (e.g., fixed message length, vocabulary size), and training regime strongly influences the emergent communication protocol [2012.02875, 2012.10776].

## 3. Compositionality, Expressivity, and Systematicity

A central objective in referential game research is analyzing the structure of emergent languages. Core metrics include:

- **Topographic Similarity (TopSim):** Measures alignment between distances in object (semantic) space and message space; Spearman correlation is typically reported [2012.02875, 2012.10776, 2304.14511].
- **Positional Disentanglement (PosDis):** Quantifies the degree to which individual message positions encode specific object attributes; original and extended forms are used [2304.14511, 2203.13176].
- **Normalized Mutual Information (NMI):** Captures one-to-one mapping between concepts and messages in hierarchy games [2203.13176].
- **Language expressivity:** Measured via transfer of a protocol to different tasks, e.g., reconstruction [2012.02875].
- **Disentanglement Metrics:** FactorVAE, MIG, and modularity of neural representations [2304.14511].

Empirical observations:
- Classical referential games yield highly compositional emergent protocols if input structure is disentangled [2012.02875, 1804.03984, 2207.08012].
- Increased channel capacity (longer messages) improves compositionality, but merely increasing vocabulary size can encourage idiosyncratic codes [2012.10776].
- Zero-shot systematicity (generalization to novel combinations) is not guaranteed even for protocols with high compositionality scores, especially in visual or entangled domains [2304.14511, 2012.10776].

## 4. Grounded, Hierarchical, and Generalization Extensions

Extensions of referential games address abstraction, hierarchical reference, and concept generalization:

- **Hierarchical Reference Games:** Agents communicate about abstractions over sets of primitive attributes, producing protocols that support both implicit (symbol omission) and explicit (dedicated tokens) abstraction [2203.13176].
- **SetRef and Concept Games:** Teachers must succinctly communicate categories or Boolean concepts over object sets, leading to increased systematicity and interpretable emergent codes [2106.02668].
- **Approximate Compositional Reconstruction:** Extraction of logical operation analogs (AND, OR, NOT) from emergent codes via data-driven sequence-to-sequence models [2106.02668].
- **Meta-Referential Games and CLBs:** Agents are evaluated on their ability to rapidly solve the binding problem and generalize compositionally in new semantic spaces, requiring both receptive and constructive compositional learning behaviors [2207.08012].

This work highlights abstraction as a pragmatic driver of compositionality, grounding as essential for agent coordination, and meta-learning as a route to robust symbolic communication.

## 5. Population Dynamics, Adaptation, and Learnability

Recent studies emphasize the importance of variation and adaptation:

- **Population of Listeners:** Agents must model and adapt to listeners with different perceptual/conceptual semantics, using rapid probing and stateful memory (e.g., LSTM embeddings) [1910.04872].
- **Cultural Transmission and Co-evolution:** Language learnability is shaped by both cultural selection (replacement of low-fitness agents) and architectural evolution (mutation of neural structures); optimal languages and protocols are those that are both expressive and easily acquired by new agents [2001.03361].
- **Interaction with Novel Partners and Perceptual Mismatches:** Protocols built for heterogeneous populations must be robust to divergent perceptual encoders (e.g., ResNet-Speaker, PNASNet-Listener) [1910.04872].

Learnability experiments verify that co-evolutionary regimes yield highly structured, low-entropy languages that are maximally compact and generalizable.

## 6. Extensions to Dialogue, Pragmatics, and Human Interaction

Referential games support modeling pragmatic and dialogic interaction:

- **Collaborative Agreement and Mixed-Initiative Dialogue:** Multi-round, role-symmetric games such as "A Game Of Sorts" prioritize collaborative ranking and negotiation, with empirical measurements of dialog balance, initiative, and grounding [2309.05162].
- **Grounded Collaborative Dialogue:** Structured reference resolution, referent memory, and pragmatic generation modules enable agents to track, disambiguate, and refer to spatial entities in dynamic contexts, achieving substantial improvements in task completion [2109.05042].
- **Uncertainty and Clarification Requests:** Recent VLM experiments leverage referential games to measure models' ability to signal and act upon internal uncertainty through clarification, revealing systematic differences both between models and with respect to human performance [2601.07820].
- **Iterated Reference and Context-sensitive Pragmatics:** Few-shot, multi-round reference games show that modern VLMs benefit from in-context exemplars but still fail to match human calibration or ad hoc adaptation to abstract referents [2511.03908].

These results underscore the value of referential games as testbeds for pragmatic reasoning, interactive competence, and human-aligned emergent communication in artificial systems.

## 7. Practical Frameworks and Implementation Toolchains

Standardized toolkits and benchmarks drive reproducibility:

- **ReferentialGym:** Taxonomizes and implements a wide variety of referential game paradigms (full/partial observability, variable length, multi-modal, discriminative/generative), provides agent class APIs, integrates major training algorithms (REINFORCE, Gumbel-Softmax, Obverter), and ships with evaluation modules for compositionality, ambiguity, and coordination [2012.09486].
- **Baseline Experiments:** Prototypical experiments demonstrate the effect of architecture and optimization (fixed vocab vs. variable length, pretrained feature extractors vs. end-to-end training, inclusion of auxiliary objectives) on emergent protocols and their semantic properties [2012.09486, 1911.05546].
- **Downstream Applications:** Protocols and encodings learned from referential games have been used for vision transformer acceleration, pretraining, and interpretable mid-level patch selection [2111.01785].

These frameworks facilitate rapid exploration of design choices, evaluation of emergent language properties, and cross-domain comparison of communication protocols.

---

In sum, referential games represent a foundational methodology for investigating communication, coordination, compositionality, and pragmatic adaptation from both a computational and cognitive perspective. The framework's flexibility—in message space, learning paradigm, population composition, and evaluation—has made it central to state-of-the-art research on emergent language and meaningful alignment in both artificial agents and human-machine systems [1910.04872, 2012.02875, 1705.10369, 2106.02668, 2203.13176, 2309.05162, 2109.05042, 2012.10776, 2001.03361, 2304.14511, 2111.01785, 2511.03908, 2601.07820, 2207.08012].

Source: https://www.emergentmind.com/topics/referential-games