Interactive Intelligence in AI Systems

• Interactive Intelligence is a paradigm where systems continuously refine outputs through cycles of observation, feedback, and adaptation, spanning domains like robotics, explainable AI, and digital humans.
• Recurring architectural patterns include modular closed-loop designs that integrate multimodal sensing, cognitive reasoning, and adaptive feedback mechanisms.
• Robust evaluations reveal current limitations in interactive refinement, highlighting challenges in safety, interpretability, and reliable human–AI interaction.

Searching arXiv for papers on “interactive intelligence” and closely related formulations to ground the article in published work. Interactive intelligence designates a class of AI paradigms in which competence is expressed through iterative observation, action, feedback, and adaptation inside an ongoing interaction loop, rather than through one-shot inference alone. Across the literature, the term is used heterogeneously: as a research environment for parameter-driven exploration of swarm intelligence algorithms, as a cyclic explanation–feedback partnership in explainable AI, as explicit human–robot co-adaptation in semi-autonomous robotics, as feedback-conditioned refinement in large multimodal models, and as personality-aligned, adaptive, self-evolving embodiment in digital humans (Kose, 2017, Wang et al., 2023, Dey, 9 Feb 2025, Zhao et al., 20 Feb 2025, Cai et al., 15 Dec 2025). Despite this variation, the shared emphasis is on systems that must decide what to perceive, what to ask, when to act, how to update internal state, and how to incorporate human or environmental feedback over time.

1. Conceptual scope and domain-specific meanings

The literature does not present a single universal definition of interactive intelligence. Instead, distinct fields operationalize it through their own problem settings. In networking, “interactive AI” is described as a stage beyond traditional data interaction paradigms, one that “engage[s] human users in dynamic and reciprocal information exchanges” and also adapts to dynamic system and network conditions through an environment–perception–brain–action loop (Zhang et al., 2024). In explainable AI, interactive intelligence is framed as the strength of a cyclic, symbiotic relationship between a human and an AI system, especially when explanations are not merely displayed but are themselves objects of user feedback and revision (Kim et al., 2019). In social-agent research, interactive intelligence appears as social intelligence under multi-turn, goal-directed interaction with other agents, where success depends on negotiation, persuasion, cooperation, competition, and theory-of-mind-like inference (Zhou et al., 2023). In game-based intention studies, it is narrowed further to the capacity to choose, execute, summarize, and infer intentions in a live multi-agent setting (Liu et al., 2024). In interactive XAI interfaces such as IXAII, the term is embodied in the user’s ability to configure explanation content, format, and audience profile while the system exposes multiple post-hoc explanation methods (Speckmann et al., 26 Jun 2025).

Taken together, these works suggest a useful synthesis: interactive intelligence is not a single algorithmic family but a systems-level property of AI agents or interfaces that are coupled to external entities through recurrent feedback, with internal updates or external reconfiguration changing future behavior.

Domain	Operationalization	Representative works
Swarm intelligence research	Human-in-the-loop parameter tuning and visual monitoring of SI algorithms	(Kose, 2017)
Explainable AI	Cyclic explanation–feedback loop and configurable explanation interfaces	(Kim et al., 2019, Speckmann et al., 26 Jun 2025)
Semi-autonomous robotics	Multi-agent co-adaptation with internal models of human state, intent, and fatigue	(Dey, 9 Feb 2025)
Social and language agents	Multi-turn goal pursuit, intention management, and social reasoning	(Zhou et al., 2023, Liu et al., 2024)
Streaming multimodal assistants	Feedback-based refinement and proactive triggering under continuous streams	(Zhao et al., 20 Feb 2025, Li et al., 26 May 2026)

2. Recurrent architectural patterns

A striking regularity across the literature is the reappearance of modular closed-loop architectures. In semi-autonomous robotics, a neuroscience-inspired four-module stack is proposed: multimodal sensing, an ad-hoc teamwork model, a predictive world belief model, and a memory/feedback mechanism (Dey, 9 Feb 2025). The sensing module fuses language, vision, EMG, IMU, and joint-angle signals; the teamwork model treats the human and robot as interactive agents with hidden states; the predictive model anticipates user and environment dynamics; and memory accumulates long-term preference information and reinforcement-like updates. In networking, a homologous environment–perception–brain–action decomposition is used, with a pluggable LLM module and a retrieval-augmented generation module supplying knowledge and contextual memory to the brain unit (Zhang et al., 2024).

Digital-human systems generalize this pattern to multimodal embodiment. Mio is defined as an end-to-end framework with five modules—Thinker, Talker, Face Animator, Body Animator, and Renderer—so that cognitive reasoning, voice, facial motion, body motion, and visual synthesis are all coordinated within a real-time interaction loop (Cai et al., 15 Dec 2025). The Thinker maintains hierarchical memory and a diegetic knowledge graph; the Talker maps text and conditioning signals to discrete audio tokens and waveform generation; the Face Animator unifies speaking and listening facial motion; the Body Animator performs streaming text-guided motion generation; and the Renderer synthesizes identity-consistent video frames.

Earlier interactive systems instantiate the same principle in simpler software forms. The swarm-intelligence toolbox introduced in 2017 organizes interaction through a central GUI linked to separate PSO, ACO, ABC, IWDs, and FA modules, each with its own parameters, visual controls, and XML-based problem files (Kose, 2017). The virtual doctor similarly uses a workflow in which sensor acquisition, speech-based anamnesis, DNN inference, probability adjustment, database storage, and PDF report generation are coordinated across Java, R, MySQL, and Arduino-based sensing (Spänig et al., 2019). In surgical robotics, SurRoL is extended into a three-part platform consisting of a human interaction interface, intelligent policy learning, and a surgical embodied AI virtual environment, thereby aligning human teleoperation and RL control within a common simulation substrate (Long et al., 2023).

Across these systems, architecture is not auxiliary. It is the mechanism by which interaction becomes persistent, stateful, and consequential.

3. Human roles, interfaces, and feedback channels

Interactive intelligence is also distinguished by the diversity of human roles it admits. The iNLP survey explicitly treats LLMs as agents that interact with humans, knowledge bases, models and tools, and environments, with observation, action, and feedback as the basic loop variables (Wang et al., 2023). That general schema appears in more specialized forms across domains.

In explainable AI, humans act as interpreters, critics, and co-designers of model reasoning. In the explanatory-efficacy framework, participants inspect recommendations, judge explanatory features as correct or incorrect with confidence ratings, adjust feature weights, and then trigger model retraining by pressing “Get new recommendation,” producing a fully cyclic, symbiotic interaction (Kim et al., 2019). IXAII broadens the same idea into a web interface where users select reference methods such as Why, Why Not, What If, When, Inputs, What Output, and Certainty; switch among text, formal rules, tables, and charts; and choose whether they are developers, users, business entities, regulatory entities, or affected parties (Speckmann et al., 26 Jun 2025).

In medical and surgical systems, humans occupy different but equally central roles. The virtual doctor conducts autonomous anamnesis by asking age, gender, polyuria, polydipsia, and alcohol and tobacco questions through speech recognition and speech synthesis, while weight and height are acquired by non-invasive sensors in a self-contained patient cube (Spänig et al., 2019). In surgical robot learning, humans become teleoperators using Geomagic Touch devices; their incremental motions and button presses are mapped into the same action representation used by RL agents, and the resulting trajectories populate an expert buffer that regularizes policy learning (Long et al., 2023).

In educational settings, human participation is even more literalized. Tangible interactive games for AI literacy ask students to become neurons, classifiers, trainers, sensors, deciders, or users, with ropes, cards, boards, and role-play enacting threshold logic, supervised learning, pattern recognition, and recommendation feedback loops (Sampanis, 31 May 2025). This suggests an important boundary case: interactive intelligence can denote not only machine capabilities but also hybrid systems in which intelligent behavior is distributed across people, artifacts, and rules.

4. Evaluation paradigms and benchmark infrastructures

A major development in recent work is the migration of interactive intelligence from a descriptive term to an explicitly benchmarked target. InterFeedback formalizes interactive problem solving for large multimodal models as a POMDP, with state, observation, action, transition, and a binary exact-match reward. It then distinguishes initial accuracy from correction rate, and shows in InterFeedback-Human that even state-of-the-art models such as OpenAI-o1 achieve an average score of less than 50% when asked to refine responses based on human feedback across 120 manually collected cases (Zhao et al., 20 Feb 2025). The central empirical claim is that static problem-solving ability and interactive intelligence are separable.

In social-agent evaluation, SOTOPIA defines open-ended multi-agent role-play episodes and scores them along seven dimensions—believability, relationship, knowledge, secrets, social rules, financial/material benefit, and goal completion—thereby turning social intelligence into a vector-valued interactive evaluation problem (Zhou et al., 2023). SOTOPIA-$\pi$ then uses filtered interactive trajectories and LLM-based ratings to train a 7B model via behavior cloning and self-reinforcement; the resulting agent approaches GPT-4-level goal completion under GPT-4 evaluation while improving safety and preserving MMLU performance, but the study also finds that LLM evaluators overestimate agents trained specifically for social interaction (Wang et al., 2024).

Other benchmark families target different aspects of interaction. Multi-Bench evaluates spoken dialogue models on about 3.2K samples across five tasks, organized into a basic track for emotion understanding and reasoning and an advanced track for emotion support and application, with a dedicated multi-turn interactive dialogue component (Deng et al., 2 Nov 2025). ChildAgentEval uses 10 web-based subtests inspired by WISC to assess cognitive age alignment in MLLM-based interactive agents, emphasizing whether a system can intentionally constrain reasoning depth, language, and working memory to mimic age-specific developmental stages (Shen et al., 18 May 2026). “Interactive Benchmarks” proposes a unified paradigm spanning Interactive Proofs and Interactive Games, where models must acquire information actively under budget constraints in logic, mathematics, poker, and trust-game settings (Yue et al., 5 Mar 2026). IPIBench extends the evaluation frontier to continuous streams, measuring proactive monitoring, proactive task management, and interleaved reactive–proactive behavior under streaming video (Li et al., 26 May 2026).

These benchmarks collectively shift emphasis from answer quality alone to interaction policy: what information is requested, how feedback is used, whether tasks persist across turns, and how performance changes under constrained sequential exchange.

5. Formal and algorithmic interpretations

The mathematical formulations associated with interactive intelligence are diverse, but several recurring motifs can be isolated. In explainable AI, explanatory efficacy is defined as a coupled measure of system performance and user understanding:

$\xi_t = a_t f(x_t),$

where

$$a_t = \sum_{m=1}^{M} S(P_t(m)), \qquad x_t = \sum_{m=1}^{M}\sum_{n=1}^{N} U_t(E_t(m,n)).$$

This formalism encodes the claim that accurate prediction without user-understood reasoning does not constitute a strong interactive explanatory partnership (Kim et al., 2019).

In interactive robotics, the multi-agent interpretation is explicit. The robot policy is conditioned not only on physical state but on estimates of human state and beliefs,

$\pi_r(a_r \mid s, \hat{s}_h, b_h),$

with belief updates of the form

$b_{t+1}^h = \mathcal{B}(b_t^h, o_{t+1}, a_t^r).$

This formulation is aligned conceptually with MARL and joint-action frameworks, even though the paper presents it as an architectural mapping rather than a complete algorithm (Dey, 9 Feb 2025).

In feedback-based multimodal evaluation, InterFeedback models interaction as a POMDP whose reward is exact-match correctness after each refinement round, making feedback interpretation and belief revision central latent skills rather than auxiliary interface features (Zhao et al., 20 Feb 2025). In the interactive-benchmark framework, the same idea is cast as budgeted sequential optimization. For truth-seeking tasks such as Interactive Proofs, the optimal policy is defined by

$$\pi^\star_{\mathrm{IP}} \in \arg\max_{\pi}\; \mathbb{E}\big[\mathbf{1}\{\hat{y}=y^\star(x)\}\big] \quad \text{s.t.}\quad \sum_{t=1}^{T} c(a_t)\le B,$$

where correctness must be achieved under explicit interaction costs. For Interactive Games, the objective becomes long-horizon utility maximization,

$$\pi^\star_{\mathrm{Game}} \in \arg\max_{\pi}\; \mathbb{E}\Big[\sum_{t=1}^{T}\gamma^{t-1} r_t\Big].$$

These equations formalize interactive intelligence as active information acquisition and sequential decision-making under resource constraints (Yue et al., 5 Mar 2026).

In digital humans, the formulation extends to multimodal embodiment. Mio’s Thinker learns a policy

$A_t = \pi(I_t, M),$

mapping multimodal input and memory to action plans, while the benchmark-level Interactive Intelligence Score aggregates cognitive, acoustic, facial, somatic, and visual dimensions:

$$\mathcal{S}_{\text{IIS}} = \frac{1}{5}(S_{\text{cog}} + S_{\text{aco}} + S_{\text{fac}} + S_{\text{som}} + S_{\text{vis}}).$$

Here interaction is evaluated not only in linguistic terms but as synchronized reasoning, speech, facial behavior, body motion, and rendering fidelity (Cai et al., 15 Dec 2025).

Even in earlier swarm-intelligence software, mathematical interaction is made visible through direct manipulation of PSO, ACO, ABC, IWDs, and FA parameters, with per-iteration logs and graphics exposing how parameter changes alter convergence behavior in real time (Kose, 2017). That formulation is less agentic, but it already treats interactive intelligence as the coupling of adaptive algorithms with user-driven experimentation.

6. Limitations, controversies, and open problems

Several limitations recur across the literature. In robotics and cyborg systems, the shift from single-agent autonomy to interactive multi-agent design introduces unresolved problems of safety, robustness, privacy, bias, interpretability, and regulatory validation, especially when physiological data and assistive control are involved (Dey, 9 Feb 2025). Networking-oriented interactive AI raises analogous concerns around poisoned knowledge bases, unreliable telemetry, adversarial prompts, and the need for standardized evaluation criteria for models generated through interactive reasoning rather than one-shot prediction (Zhang et al., 2024).

Benchmark studies show that current frontier systems remain weak in core interactive behaviors. InterFeedback reports that even strong large multimodal models struggle to refine their responses from human feedback, with average performance remaining below 50% in the manually curated setting (Zhao et al., 20 Feb 2025). Multi-Bench finds that spoken dialogue models perform substantially better on basic emotion-understanding tasks than on advanced multi-turn interactive dialogue and reasoning-related tasks, especially emotion awareness and application (Deng et al., 2 Nov 2025). ChildAgentEval shows that age prompting alone does not create developmentally ordered trajectories; explicit cognitive filters are required, and even then models remain disproportionately strong in language and working memory while weak in visual reasoning and processing speed relative to human developmental norms (Shen et al., 18 May 2026). IPIBench, finally, identifies unstable proactive triggering and weak coordination between reactive and proactive behaviors as major limitations of current MLLMs under continuous streaming conditions (Li et al., 26 May 2026).

There are also methodological controversies around evaluation. SOTOPIA-$\pi$ shows that LLM-based evaluators overestimate the social competence of agents trained specifically for social interaction, a concrete instance of evaluator over-optimization in interactive settings (Wang et al., 2024). Interactive XAI research likewise indicates that the mere presence of explanations is insufficient; only interfaces that allow users to alter the model’s reasoning or its presentation format appear to strengthen the loop that explanatory efficacy is intended to measure (Kim et al., 2019). Earlier systems expose a different, more infrastructural limitation: the swarm-intelligence research toolbox was positively received by 50 scientist/researcher users over two weeks, but participants also stated that “There must be more SI based algorithms within the software system,” underscoring the incompleteness of any static interactive environment (Kose, 2017).

A plausible implication is that interactive intelligence will remain a plural rather than singular research target. The field spans human-centered explanation loops, semi-autonomous motor co-adaptation, long-horizon social strategy, streaming proactive assistance, developmental calibration, and multimodal embodiment. What unifies these lines is not a single architecture or metric, but the insistence that intelligence must be assessed in the dynamics of interaction itself: under feedback, under uncertainty, under budget, and across time.