Interaction as Intelligence
- Interaction as Intelligence is a paradigm defining intelligence as emerging from dynamic relationships among agents, interfaces, and environments.
- Research in this area emphasizes measurable outcomes from closed-loop perception, adaptive communication, and coordinated multi-agent behavior.
- The approach redefines evaluation by focusing on interaction trajectories and real-time feedback rather than isolated, static performance metrics.
“Interaction as intelligence” denotes a family of positions in which intelligence is not treated as an isolated property of a bounded model, but as something constituted, disclosed, amplified, or constrained through ongoing interaction among agents, interfaces, bodies, environments, and socio-technical systems. Across recent work, the term covers several distinct but convergent claims: humans infer intelligence from communicative success and predictability rather than from internal model size alone; embodied agents become intelligent through closed perception–decision–action–feedback loops; generalization can be organized around reusable patterns of object interaction; collective intelligence is governed by social influence structures; and human–AI systems should often be evaluated through interaction trajectories rather than outputs alone (Adkins, 2024, Jiang et al., 11 May 2025, Davis, 30 May 2026).
1. Conceptual scope and major formulations
A common theme across the literature is a shift in the unit of analysis from isolated computation to relational process. In Human–AI Interaction, interaction is defined not merely as message exchange but as a cognitively loaded process in which people adapt their linguistic behavior, attempt to build a theory of mind of the system, and judge whether the exchange feels natural relative to human–human communication (Adkins, 2024). In embodied AI, intelligence is defined as arising “through the interaction between its body and its environment,” with real-time interaction and sensorimotor coupling treated as foundational rather than auxiliary (Jiang et al., 11 May 2025). In co-creative AI, “Interaction-Centered Intelligence” makes interaction trajectories, coordination patterns, participatory engagement, adaptive regulation, and interactional drift the primary explanatory variables (Davis, 30 May 2026). In social psychology, “socially-minded intelligence” is defined as the extent to which agents can move between acting as individuals and acting as group members in a context-sensitive way directed toward specific goals (Bingley et al., 2024). In swarm systems, intelligent search is analyzed as the structure and dynamics of who influences whom over time, formalized as an interaction network (Oliveira et al., 2018). Social NeuroAI sharpens the point further by treating social interaction as the “dark matter” of AI: largely neglected, yet essential to advanced cognitive ability (Bolotta et al., 2021).
| Domain | Interaction as | Representative papers |
|---|---|---|
| Human–AI communication | adaptation, theory of mind, naturalness baseline | (Adkins, 2024) |
| Embodied agents | closed-loop perception–decision–action–feedback | (Jiang et al., 11 May 2025, Freedman et al., 2019) |
| Generalization | reusable interaction goals and object relations | (Kumar et al., 2024) |
| Collective systems | dynamic social influence structure | (Oliveira et al., 2018, Bingley et al., 2024) |
| Co-creative and educational systems | primary unit of analysis, participatory engagement | (Davis, 30 May 2026, Moldoveanu et al., 1 Jan 2025) |
| Data and evaluation | human knowledge injection and perceptual judgment | (Chen, 2018, Krishna et al., 2021) |
This diversity of formulations does not imply a single theory. It instead indicates a shared reorientation: interaction is treated variously as the medium in which intelligence develops, the evidence by which it is attributed, the mechanism by which it generalizes, or the object by which it should be measured. A plausible implication is that “interaction as intelligence” is best understood as a research program spanning HAI, embodied AI, multi-agent systems, co-creative systems, education, and evaluation, rather than as a narrow philosophical slogan.
2. Communicative interaction, theory of mind, and perceived intelligence
In natural-language HAI, intelligence is strongly mediated by users’ ability to form a workable theory of mind of the AI. A 101-person survey found systematic linguistic adaptation: participants reported that they “change how [they] phrase things to AIs to try to help it understand [them] better” with mean Likert score approximately $1.02$, “have to think more systematically to communicate with an AI” with mean approximately $0.76$, and do not phrase things to AI as they would with a friend, with mean approximately . They also reported paying more attention to sentence structure with AI than with a friend, mean approximately $0.83$, and speaking to a voice assistant differently from speaking to a friend, mean approximately (Adkins, 2024). Interaction is therefore framed as adaptation relative to a close-friend baseline, not as frictionless dialogue.
The same study treats communicative success as a proxy for perceived intelligence. Participants somewhat agreed that AI has “limitations when it comes to understanding the way I speak or write,” mean approximately $0.97$. Naturalness was low across modalities: 75% answered “No” to whether speaking to a voice assistant feels natural, 69% answered “No” for AI-powered chat, and 79% answered “No” to whether conversation with a voice assistant or AI-powered chat feels comparable to conversation with another person (Adkins, 2024). The argument is not that AI lacks internal capability, but that opaque behavior, limited modality, and unstable predictability prevent robust theory-of-mind formation. Intelligence, from the user’s standpoint, is therefore relational: it appears when interaction becomes stable, predictable, and cognitively low-effort.
Work on explainable AI makes the same point in more formal dialogue terms. A grounded interaction protocol for explanation was derived from 398 explanation dialogues and formalized in the Agent Dialogue Framework as
with explanation and argumentation as coupled dialogue types. The explanation locutions are
and the argumentation locutions are
In a human-agent study with 101 dialogues from 14 participants, the model held for 96 out of 101 dialogue games (Madumal et al., 2019). The implication is that explanatory intelligence is not reducible to feature saliency or post-hoc rationalization; it includes the ability to sustain question–explanation–challenge–repair sequences recognizable as valid social explanation.
A more explicit interactional claim appears in a dual-channel architecture that treats multi-turn interaction as a necessary condition for intelligence emergence. There, probabilistic generation is coupled to a white-box procedural chain-of-thought graph, and dialogue depth is claimed to correlate positively with human-alignment degree (Zhou et al., 12 Apr 2025). This suggests that communicative intelligence depends not only on producing plausible outputs, but on remaining corrigible under extended interaction.
3. Embodied, closed-loop, and real-time forms of interaction
Embodied AI makes interaction architecturally central. A recent review defines embodied intelligence through four modules—perception, intelligent decision-making, action, and feedback—organized as a closed loop (Jiang et al., 11 May 2025). Perception is multimodal and action-dependent; decision-making is typically formalized as RL over states , actions $0.76$0, and rewards $0.76$1, with objective
$0.76$2
Feedback closes the loop by monitoring perceptual, decision, and execution errors and updating the system online. Within this framing, intelligence is evidenced by what an agent can do in ecologically valid, interactive environments rather than by static benchmark competence alone.
Closed-loop interaction is also the core of responsive assistive systems. The PReTCIL framework—“Planning and Recognition Together Close the Interaction Loop”—integrates activity recognition, plan recognition, intent recognition, planning, execution, and monitoring. The agent infers what the user is trying to accomplish, autonomously selects actions, predicts how the user should respond, and re-runs recognition and planning when observed behavior diverges from expectation (Freedman et al., 2019). Intelligence here lies in continual hypothesis testing inside interaction, not in fixed input parsing or preset responses.
Recent real-time vision-language systems radicalize this idea by giving the model initiative over whether to speak at all. JoyAI-VL-Interaction operates at 1 Hz over streaming video with action space
$0.76$3
implemented through control tokens and optimized with weighted cross-entropy plus RL over stream-level rewards (Yao et al., 10 Jun 2026). It continuously watches video, decides whether a moment warrants response, and can delegate hard subtasks to a background model while remaining present in the stream. Across six real-world scenarios, human raters preferred it over the in-app video-call assistants of Doubao and Gemini, with overall win rates of 77.6% and 87.9% respectively (Yao et al., 10 Jun 2026). The significant point is not only performance, but that timing, silence, delegation, and persistence are themselves treated as dimensions of intelligence.
Digital-human systems extend the same principle to multimodal embodiment. Mio, a “Multimodal Interactive Omni-Avatar,” couples a Thinker, Talker, Face Animator, Body Animator, and Renderer, with the Thinker producing a unified action plan
$0.76$4
where $0.76$5 is multimodal input and $0.76$6 is internal state (Cai et al., 15 Dec 2025). Intelligence is evaluated not just on dialogue quality but on coordinated speech, listening behavior, body motion, and visual consistency. Its benchmark aggregates cognitive, acoustic, facial, somatic, and visual performance into an Interactive Intelligence Score, with Mio reported at approximately 76 and outperforming a composite best-prior baseline by 8.4 points (Cai et al., 15 Dec 2025).
4. Interaction as a mechanism for learning, generalization, and collective organization
In reinforcement learning, interaction can be elevated from low-level action to a structured representational layer. KIX—Knowledge-Interaction-eXecution—organizes generalization around a type-space knowledge graph and an intermediate layer of reusable interaction types $0.76$7. Its basic control unit is the interaction goal
$0.76$8
pairing an object instance with an interaction type such as “open,” “reveal,” or “pickup” (Kumar et al., 2024). A meta-policy reasons over type graphs to recommend such goals, while lower-level policies realize them as primitive actions. In MiniGrid ObstructedMaze-Full, both KIX variants achieved higher returns than a base PPO agent on all tasks, with especially strong gains on the dynamic Task 3; across 12,000 evaluation episodes per task, KIX2 also showed smaller Wasserstein distance shifts in traversal patterns on the harder transfer settings (Kumar et al., 2024). The central claim is that intelligence is not just mapping states to actions, but selecting and composing interactions in a transferable way.
Collective systems make the same claim at the swarm level. In swarm intelligence, the interaction network $0.76$9 records the extent to which one agent influences another, abstracting away metaphor-specific algorithm details. For PSO, interaction diversity 0 is derived from how the interaction network fragments as weak ties are removed, and it captures exploration–exploitation structure directly from interaction topology (Oliveira et al., 2018). In large-scale benchmarks with swarm size 100 and dimensionality 1000, changing topology from global to connected 1-regular graphs produced non-monotonic performance changes; on one function, final fitness improved from 2 to 3 and then deteriorated to 4 as 5 was reduced further (Oliveira et al., 2018). The result reframes swarm “intelligence” as the self-organization of influence patterns rather than the complexity of individual update rules.
Human interaction can also be the engine of a model’s data and evaluation regime. In computer vision, rapid crowdsourcing reduced image verification cost from 5.1 s per item to 0.5 s per item at precision 0.97, a 10.2× speedup, while a multiclass cascade case study reached a 50× total speedup (Krishna et al., 2021). A social-strategy agent on Instagram increased informative response rates from 15.8% for unaugmented questions to 30.54% with rule-based strategies and 58.1% with learning-based strategies (Krishna et al., 2021). HYPE then grounded generative model evaluation in psychophysics rather than proxy feature distances, with 30 evaluators sufficient for stable bootstrap confidence intervals (Krishna et al., 2021). In each case, interaction is not ancillary; it determines what the system can learn and how its competence is judged.
Grounded language learning via interaction provides a closely related result. An agent that learns to speak by jointly exploiting sentence feedback and reward feedback substantially outperformed pure imitation and pure reinforcement. In held-out compositional generalization, the joint approach reached 98.3% accuracy versus 75.1% for imitation and 0.0% for pure reinforcement; in mixed compositional settings it reached 98.9% versus 83.7% and 0.0% (Zhang et al., 2017). The claim is direct: language intelligence improves when speaking is treated as action whose consequences are informative.
5. Interfaces, explanation, education, and prototyping as sites of intelligence
Intelligent User Interfaces define intelligence explicitly in interactive terms. IUIs aim to incorporate intelligent automated capabilities into HCI so that human performance or usability improves in critical ways, and they involve AI components that leverage human skills and capabilities rather than merely replacing them (Sonntag, 2017). This includes intelligence in the interface itself—speech, gesture, adaptive user interfaces, multimodal fusion—and intelligence behind the interface—recommenders, tutoring systems, interface agents. In both cases, the relevant question is how intelligence is expressed through interaction management, user modeling, discourse modeling, and context-sensitive action.
Extended-reality testbeds make this interactional conception experimentally manipulable. The XR–AI continuum distinguishes AI for XR from XR for AI; the latter, termed “eXtended Artificial Intelligence,” uses XR as a rapid-prototyping and research environment for future AI embodiments (Wienrich et al., 2021). In one XR factory study, adding conversational behavior to a robot increased humanness, attractiveness, social presence, and positive valence, while also increasing mental demand; the study additionally revealed a gender effect, with women’s evaluations shifting more strongly in the conversational condition (Wienrich et al., 2021). In another XR study, a visually more complex embodied recommender produced an Eliza effect: users inferred greater trustworthiness and lower risk from appearance alone (Wienrich et al., 2021). These results show that perceived intelligence is interactionally staged through embodiment and context, often before full algorithmic systems exist.
Educational work on “Interactionalism” extends the same logic to human development. There, interactional intelligence is defined as a new skill set for the LLM era: interactive, interpretive, and dialogical rather than individualistic and monological (Moldoveanu et al., 1 Jan 2025). Reading becomes interactive processing of text with a dialogical agent; writing becomes interactive production of text with an AI partner; analysis becomes interactive production of plausible inferences; coding becomes interactive production of code with co-pilots and agent workflows (Moldoveanu et al., 1 Jan 2025). The framework decomposes these practices into meta-cognitive tasks such as self-explicitation, task specification, task decomposition, task switching, partial credit assignment, objective refinement, and task specification refinement, plus meta-emotional and meta-relational competencies. A plausible implication is that interaction with AI is not merely a delivery mechanism for instruction but part of the skill being cultivated.
Co-creative AI pushes the argument further by making interaction itself the primary analytical object. “Interaction-Centered Intelligence” in co-creative systems emphasizes interaction trajectories, coordination quality, participation balance, adaptive regulation, and interactional drift rather than isolated output quality (Davis, 30 May 2026). This does not deny the importance of generated artifacts, but it relocates creativity and intelligence to the temporal organization of joint activity.
6. Measurement, misconceptions, and open problems
Several formalisms attempt to quantify interactional intelligence directly. Socially-minded intelligence is defined at both individual and group levels. Individual socially-minded intelligence is
6
where 7 is socially-minded ability and 8 is the weighted sum of social resources in context 9 (Bingley et al., 2024). Group socially-minded intelligence is
$0.83$0
capturing how flexibly members act as individuals, subgroup members, or group members relative to superordinate goals (Bingley et al., 2024). These equations explicitly place interaction variables—shared identity, goal alignment, role flexibility—inside the definition of intelligence rather than treating them as external modifiers.
Information-theoretic work makes a different measurement move by treating HCI as entropy reduction supplied by users. In that framework, the action capacity of an interaction is the entropy of the input action alphabet, and the value of an HCI step is its alphabet compression minus potential distortion, normalized by cost (Chen, 2018). The claim is that interaction injects knowledge the computer does not have, thereby breaking the assumptions behind the Data Processing Inequality. Interaction is therefore measurable as a contribution of human intelligence to data-intelligence workflows, from simple selection tasks to interactive machine learning (Chen, 2018).
Several recurrent misconceptions are challenged across the literature. One is that passing a Turing-like text test is sufficient evidence of intelligence. The HAI work argues that the Turing Test resembles interaction with a stranger and ignores long-term relationship formation, mutual theory of mind, and cognitive ease (Adkins, 2024). Another is that larger probabilistic models or more realistic embodiments automatically produce better intelligence. The recommender-system Eliza effect shows that visual complexity alone can inflate trust and perceived competence (Wienrich et al., 2021), while dual-channel human-aligned work argues that open-loop probabilistic generation remains vulnerable to hallucination and unpredictability unless coupled to explicit procedural verification and multi-turn correction (Zhou et al., 12 Apr 2025). A third misconception is that natural language interfaces are inherently more natural; survey evidence instead indicates that they often increase cognitive load because users must continually adapt and refine shallow mental models of the AI (Adkins, 2024).
Open questions remain extensive. Human–AI communication work identifies direct measurement of cognitive load, links between social anxiety and AI interaction, pathways to mutual theory of mind, and principled criteria for when natural language interfaces justify their cognitive cost as unresolved problems (Adkins, 2024). Embodied AI identifies coordination between perception and action, multi-task adaptability, long-term memory, transfer learning, and integration of learning with reasoning as unresolved AGI-scale challenges (Jiang et al., 11 May 2025). Co-creative work identifies operationalization of interaction units, temporal scales of analysis, scalable logging infrastructure, and ethics of participatory agency as open problems (Davis, 30 May 2026). Socially-minded intelligence raises further questions about how humans come to treat AI as ingroup members, how to prevent pathological over-identification or over-deference, and how to engineer flexible social influence in human–AI teams without reproducing harmful collective dynamics (Bingley et al., 2024).
Taken together, these literatures support a strong but differentiated conclusion. Intelligence can still be studied as internal capability, benchmark performance, or representational power. Yet in many settings—conversation, embodied control, co-creation, collective coordination, explanation, education, and data intelligence—the more revealing question is what kinds of interaction a system can sustain, induce, or exploit. On that view, interaction is not merely where intelligence is displayed; it is often where intelligence is formed, attributed, and measured.