Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions

Abstract: Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific conventions grounded in shared interaction history. Prior work shows that multimodal LLMs fail to become more efficient across rounds, although they align on the labels they use. How can we determine whether this alignment reflects partner-specific grounding rather than a shared task vocabulary? We address this question by comparing capable multimodal agent dyads with human dyads from the KTH Tangrams corpus. Our novel methodological contribution is a constrained pseudo-dyad baseline that matches the original referential task structure, but breaks partner history. This baseline enables us to test whether the observed label alignment depends on interaction with a specific partner. Across three analytic layers (task competence, description strategy, alignment dynamics), we find clear differences. Humans reduce effort through entrainment, compressing descriptions and increasing label alignment with partners. Agents instead maintain fixed effort levels, producing verbose descriptions from round one, with near-ceiling label overlap that is statistically indistinguishable between real and pseudo dyads. MLLMs thus achieve coordination without convention, succeeding by verbose description rather than by forming the compact, history-dependent referring expressions characteristic of human dialogue.

• The paper demonstrates that multimodal LLMs achieve high task success using exhaustive, non-adaptive description strategies rather than partner-specific compression.
• It employs a novel pseudo-dyad methodology to differentiate genuine interaction history from globally-shared pretrained priors in referential games.
• Experimental results reveal that humans compress language over repeated rounds while agents maintain verbose, static descriptions despite high accuracy.

Distinguishing Partner-Specific Grounding in Multimodal LLM Agents: Evidence from Referential Games

Introduction

This essay provides a technical analysis of "Aligned but Not Partner-Specific: Distinguishing How Multimodal LLM Agents Succeed in Reference Games Without Human-Like Conventions" (2606.08081). The paper interrogates whether the apparent lexical alignment achieved by advanced multimodal LLMs (MLLMs) in repeated reference games reflects genuine partner-specific convention formation, or merely the deployment of globally-shared pretrained priors. Through a rigorous experimental methodology that adapts the canonical KTH Tangrams paradigm, combines human and agent dyadic corpora, and introduces a novel pseudo-dyad baseline, the study provides clear evidence that current MLLMs accomplish coordination through exhaustive, non-adaptive description strategies rather than dynamic, partner-grounded convention formation.

Experimental Design and Pseudo-Dyad Methodology

The authors leverage the KTH Tangrams reference game, which requires dyads to establish referential conventions for ambiguous abstract shapes over repeated interaction. Human dyads are expected to incrementally compress and partner-adapt their descriptions, producing conventionalized, compact labels. The MLLM setting is particularly demanding: it requires not only multimodal grounding and memory of visual-dialogic context, but also the avoidance of trivial referent leakage via system-level tags.

A central methodological innovation is the construction of constrained pseudo-dyads, which control for task structure, referent, and turn position, but remove shared partner interaction history. This baseline enables a direct comparison: genuine partner-specific convergence should drive a divergence between real and pseudo pairings. The creation of pseudo-dyads through combinatorial pairing (Figure 1) is critical for disentangling the locus of alignment, distinguishing priors from negotiated interactive conventions.

Figure 1: Pseudo-dyad generation aligns task and structural features while severing partner history to assess the interaction-dependence of alignment.

The MLLM agent pool is constructed via an exhaustive parameter sweep over recent GPT-5 and GPT-4o variants, using a robust pipeline to ensure fair task protocol translation and system constraint enforcement. Only the GPT-5 configuration passes all visual, communicative, and non-leakage criteria, achieving 94% exact-match accuracy.

Figure 2: MLLM adaptation of the KTH Tangrams protocol maintains multimodal context, turn structure, and rigorous avoidance of label leakage.

Agent and Human Performance: Task Competence and Interactional Effort

Both humans and GPT-5 dyads achieve high overall task success (humans: 99.8%, agents: 95.0%). However, the dynamic trajectories of effort allocation diverge. Humans display significant efficiency gains across rounds: mean turns-to-success decrease steeply, consistent with literature on conceptual pact formation and referential compression. Agent dyads, by contrast, operate with persistently flat effort levels, showing no evidence of round-wise adaptation or compression (Figure 3).

Figure 3: Human dyads rapidly compress interactional effort whereas agents maintain static, minimal exchange counts.

This near-constant sparseness in agent turns emerges from high initial informativeness—the agent starts each exchange with maximal relevant detail, thus obviating the need for negotiation or history-dependent adaptation.

Description Strategy and Lexical Compression

A key signature of convention formation is the proportional compression of referential descriptions, driven by entrainment. Humans robustly halve and eventually reduce content-word counts to near a third of their initial values over repeated mentions, reflecting efficient, partner-specific labelling (Figure 4).

Figure 4: Content-word compression over interactional deciles illustrates human but not agent conventionalization.

Agents, despite their high success, show near-constant verbosity, with content-word counts remaining at ~90% of initial levels throughout. This supports the claim that success is achieved not via adaptive compression, but by redundancy and descriptional exhaustiveness.

Lexical Alignment and Conventionalization

The essay’s central diagnostic is whether shared lexical cores—clusters of near-equivalent referential descriptions—grow more prominent through real interaction history compared to pseudo-dyad controls.

At the turn level, human real dyads show significantly higher probabilities (6.8x odds) of producing a turn with partner-aligned lexical material than pseudo-dyads (Figure 5). For MLLMs, the real-pseudo distinction collapses: both conditions yield near-ceiling overlap, reflecting task-vocabulary saturation rather than interaction-adaptive pacts.

Figure 5: Turn-level shared lexical core ratios show strong real versus pseudo separation for humans, but indistinguishable ceiling effects for agents.

Peak-normalized trajectories of lexical core reuse (Figure 6) reinforce these findings. Human real dyads display a rising and sustained reuse signature, while pseudo trajectories decay quickly. Agents, by contrast, show parallel curves for real and pseudo pairings, with only marginal divergence.

Figure 6: Lexical core reuse intensity across rounds highlights sustained, interaction-dependent alignment only in human real dyads.

Structural Properties of Lexical Cores

Analysis of core morphosyntactic properties further distinguishes human and agent alignment. Human lexical cores cluster in compact, high-content domains (1–4 tokens, high content-word ratio), mapping precisely onto the syntactically-reduced, semantically-rich units identified in classic referential game work. Agent lexical cores are more variable and lengthier (up to 30+ tokens), often reflecting recycled, exhaustive descriptions (Figure 7).

Figure 7: Lexical core length and composition underscore human preference for minimal, content-dense labels; agent cores reflect descriptional saturation.

Practical and Theoretical Implications

The findings have important ramifications. On the practical front, raw performance metrics such as task success rate and even lexical overlap fail to diagnose the mechanisms of coordination. MLLMs can achieve high accuracy via descriptional exhaustiveness, without adaptive reduction, dialogic accommodation, or partner-tracking. From a theoretical perspective, this signals that current instruction-tuned models—optimized for aggregate correct output—do not reinstantiate the microgenesis of conventions that is fundamental to human communication.

Furthermore, the study’s analytic protocol (including the pseudo-dyad baseline and lexical-core alignment framework) is released as a reusable, model-agnostic diagnostic. Application to other architectures, training regimes, and interactional formats will be essential for measuring progress toward genuine partner-adaptive language technology.

Future Directions

Directions for future research include: (i) replication on open-weight and alternative architectural models, (ii) expanded factorial designs involving human-agent mixed dyads to index accommodation and adaptation in more diverse social settings, and (iii) exploration of explicit cost-sensitive or convention-inducing training objectives. There is also scope for closer interrogation of the intersection of multimodal grounding, interaction memory, and the inductive biases that underpin partner specificity.

Conclusion

This paper demonstrates that multimodal LLM agents, despite achieving high levels of apparent alignment and task success, do not form human-like, partner-specific conventions in repeated reference games. Instead, they operate through static, verbose, descriptional strategies shaped by pretrained priors and task requirements. Only humans display robust, incremental compression and partner-grounded entrainment. The study’s methodological and analytic contributions provide a crucial foundation for advancing rigorous, interaction-sensitive evaluation of cooperative AI LLMs.