Interaction Initiation System Overview

• Interaction Initiation System (IIS) is a framework that defines when and how interactions begin by converting cues into actionable system responses.
• IIS designs span adaptive learning, human-robot interaction, and natural-language interfaces, employing methods from physiological triggers to multimodal sensor fusion.
• Empirical studies show that nuanced initiation strategies, such as human-mediated triggers, enhance performance and user engagement more effectively than simplistic automated methods.

Interaction Initiation System (IIS) denotes a class of mechanisms that govern how an interaction begins, but the term is used differently across research areas. In adaptive learning, it refers to the design problem of who or what is allowed to initiate the interaction and at what moment, especially for the delivery of scaffolds such as Eye Movement Modeling Examples (EMME) during debugging (Golrang et al., 6 May 2026). In human-robot interaction (HRI), IIS denotes systems that detect or decide the opening of an exchange from embodied cues, including body language, speech direction, face orientation, and sustained visual attention (Schiffmann et al., 3 Dec 2025, Yun et al., 11 May 2026). In software interfaces, a related interaction-initiation role appears in iDian, which lets users initiate software operations by expressing intent in natural language (Rong, 2010). The acronym is also used in a distinct statistical sense—innovated interaction screening—for high-dimensional nonlinear classification, where it does not denote interaction initiation at all (Fan et al., 2015). This polysemy makes IIS a cross-domain term whose meaning depends on the surrounding methodological and application context.

1. Conceptual scope and domain variants

Across the interaction-oriented papers, IIS is fundamentally concerned with the transition from a latent intention or need to an explicit system-level response. In the adaptive-learning study, the central claim is that the effectiveness of EMME depends on the initiation system that governs when support appears, not just on the content of the support itself (Golrang et al., 6 May 2026). In HRI, the same general problem appears as detection of the moment at which a human intends to begin interaction, either by conversational opening behavior in public space or by nonverbal cues in domestic environments (Schiffmann et al., 3 Dec 2025, Yun et al., 11 May 2026). In iDian, interaction initiation is framed as the user’s ability to begin a software action by typing a natural-language instruction rather than navigating menus or memorizing command syntax (Rong, 2010).

Domain	IIS meaning	Core initiation question
Adaptive learning	Scaffold initiation mode	Who or what launches support, and when?
HRI conversation opening	Timing and opener detection	Should the robot wait, speak, or listen?
HRI IoI detection	Initiation of interaction detection	Has the user begun interacting with the robot?
Natural-language software control	Intent-to-action initiation layer	How is a user’s intention converted into software operations?
High-dimensional statistics	Innovated interaction screening	Which variables participate in interactions?

A plausible implication is that IIS is best understood as a control layer over the onset of interaction, rather than as a property of content alone. The interaction may be pedagogical, conversational, robotic, or operational, but the decisive issue is the rule or mechanism that converts a potentially meaningful cue into an actual intervention, greeting, service transition, or software command.

2. IIS as scaffold initiation in adaptive learning

In the debugging study, scaffold initiation mode is explicitly treated as an IIS design problem. The key question is not only what scaffold is shown, but who or what is allowed to initiate the interaction and at what moment (Golrang et al., 6 May 2026). The study compares four conditions—Teacher-initiated EMME (T-EMME), Learner-initiated EMME (L-EMME), Automated EMME (Auto-EMME), and No-scaffold control—which instantiate different assumptions about regulatory ownership: guided regulation, self-regulation, system regulation, and no initiation system, respectively.

The experiment used a between-subjects design with 120 undergraduate computer science students, 30 participants per condition, at a European university. Participants had all completed at least one introductory programming course, and the sample included 31 female students with Mean age = 20.78 years, SD = 2.73. They completed 10 Python debugging tasks involving logical bugs rather than syntax errors. The learning environment included a main debugging panel, an EMME player, video controls, and a Disable Help option. The EMME materials showed an expert’s screen recording together with overlaid expert gaze as a moving dot and trail; the authors report that they ultimately used teaching assistants rather than experts because the experts’ debugging behavior was too efficient and their gaze patterns too compressed to model effectively (Golrang et al., 6 May 2026).

The primary performance measure was the number of correctly debugged programs out of 10. A one-way ANOVA showed a significant effect of condition, $F(3,116)=132.44,\ p<.05$. Mean performance was 2.50 / 10, SD = 1.13 for Control, 6.93 / 10, SD = 0.83 for L-EMME, 6.56 / 10, SD = 1.16 for T-EMME, and 4.06 / 10, SD = 0.83 for Auto-EMME. L-EMME and T-EMME were the best performers, with no significant difference between them, $F(1,58)=1.97,\ p>.05$. Auto-EMME was significantly worse than both human-mediated conditions, though all EMME conditions outperformed the control (Golrang et al., 6 May 2026).

From an IIS perspective, the central result is that the quality of support depended on initiation mode. The study reports that human mediated initiation whether teacher or learner consistently produced higher performance than automated triggering and more integrative engagement with the EMME material, while automated triggering based on sustained low pupillary activity was associated with disruptive behavioral patterns suggesting mistimed delivery (Golrang et al., 6 May 2026). This directly supports the claim that a scaffold can become effective only if the IIS launches it at a cognitively usable time and in a usable way.

The study also reports that EMME eliminated the performance advantage of prior programming knowledge. In the control condition, prior expertise correlated with performance at $r = 0.58,\ p<.05$. In the EMME conditions, the correlations were $r=-0.04$ for T-EMME, $r=0.31$ for L-EMME (non-significant), and $r=-0.23$ for Auto-EMME (non-significant). The ANCOVA interaction terms were T-EMME × expertise: $\beta=-0.40,\ p=.006$, Auto-EMME × expertise: $\beta=-0.48,\ p=.007$, and L-EMME × expertise: $\beta=-0.22,\ p=.10$ (Golrang et al., 6 May 2026). This suggests that IIS design affects not only mean performance but also how access to support is distributed across expertise levels.

3. Physiological triggering, behavioral signatures, and the timing problem

The automated condition in the debugging study operationalizes IIS as a physiological trigger policy. Eye tracking served both as a source of the EMME model and as a live sensing input. The automated trigger used the Index of Pupillary Activity (IPA) and was defined as a participant’s IPA falling more than two standard deviations below their individual resting-state baseline for longer than 60 consecutive seconds:

$$\text{trigger if } IPA(t) < \mu_{\text{baseline}} - 2\sigma_{\text{baseline}} \quad \text{for } t \ge 60 \text{ s}$$

This state was interpreted as sustained low mental effort or disengagement (Golrang et al., 6 May 2026). The paper’s core argument, however, is that this is an ambiguous IIS signal: low effort may indicate disengagement, but it may also indicate productive fluency.

Behavioral log data make this ambiguity empirically visible. Seven interaction measures were logged: number of pauses, average pause duration, proportion of video played, number of backward jumps, number of forward jumps, average scanpath length, and transitions between task and EMME. Only three were meaningfully related to performance: more pauses → worse performance with $F(1,58)=1.97,\ p>.05$0; more backward jumps → worse performance with $F(1,58)=1.97,\ p>.05$1; and more task-video transitions → better performance with $F(1,58)=1.97,\ p>.05$2 (Golrang et al., 6 May 2026).

Condition differences were sharply patterned. Auto-EMME produced more pauses, more backward jumps, and fewer task-video transitions, whereas L-EMME and T-EMME produced more transitions. The study interprets the transition pattern as integrative engagement, meaning that learners accessed the support when they were ready to use it and then returned to the task to integrate what they had seen. A significant ANOVA was reported for behavioral differences across EMME conditions in task-video transitions, $F(1,58)=1.97,\ p>.05$3, with pairwise comparisons showing L-EMME > Auto-EMME and T-EMME > Auto-EMME. The paper also notes that backward jumps were especially harmful in T-EMME, with interaction estimate $F(1,58)=1.97,\ p>.05$4 (Golrang et al., 6 May 2026).

These findings have broader significance for IIS design. The paper concludes that a single low effort physiological threshold is insufficient as a trigger criterion for complex problem solving support and characterizes the automated system as lacking adequate context sensitivity (Golrang et al., 6 May 2026). This suggests that in adaptive systems, initiation logic based on a single physiological metric may be structurally underdetermined when the same signal can arise from both disengagement and efficient task execution.

4. IIS in human-robot interaction: conversation openings and nonverbal IoI detection

In HRI, IIS addresses the socially consequential problem of how a robot determines that interaction is beginning. One paper defines IIS as a machine-learning-driven module for deciding when a socially interactive agent should open a conversation and, ultimately, who opens it first (Schiffmann et al., 3 Dec 2025). A second paper presents an Initiation of Interaction (IoI) Detection Framework that does not rely on hotwords/keywords, but instead infers intention when the user either speaks while facing the robot or looks at the robot for a sustained period without speaking (Yun et al., 11 May 2026).

The museum study used a Furhat robot from Furhat Robotics, named Mira, deployed near the entrance of the Deutsches Museum Bonn (DMB) from July 23, 2024 to August 15, 2024 as a public service and information point. Visitors were not given instructions beyond a poster explaining the study and data-protection procedure. Entry into the interaction area required consent by pressing a red buzzer. After preprocessing, the dataset contained $F(1,58)=1.97,\ p>.05$5 suitable single-user interactions, with 88 female and 113 male participants. These yielded 26,675 labeled data instances, distributed as 12,898 wait, 6,542 listen, and 7,235 speak. The split was approximately 89.1% training and 10.9% test, corresponding to 23,775 training instances and 2,900 test instances, with the test set covering 22 interactions (Schiffmann et al., 3 Dec 2025).

The domestic IoI study used the authors’ mobile monitoring robot equipped with an Azure Kinect and implemented all components in ROS. Its perception pipeline combined HARK for sound source localization, YOLOv7 for person detection, DeepSORT for tracking, and MediaPipe head pose estimation to infer whether a user’s face was oriented toward the robot (Yun et al., 11 May 2026). The framework defines two initiation routes. In the audio-and-vision-based IoI route, the robot is in the monitoring state, a user begins speaking, the system estimates speech direction, associates that direction with a tracked person, and infers interaction if the localized speaker’s frontal face is oriented toward the robot. The person association rule is

$F(1,58)=1.97,\ p>.05$6

with an angular rejection threshold of typically $F(1,58)=1.97,\ p>.05$7. The corresponding indicator is

$F(1,58)=1.97,\ p>.05$8

where $F(1,58)=1.97,\ p>.05$9 means sound source detected, $r = 0.58,\ p<.05$0 means frontal face detected, and $r = 0.58,\ p<.05$1 means the person is tracked (Yun et al., 11 May 2026).

The vision-based IoI route handles silent initiation. If the robot detects a frontal face of a tracked person, it measures the elapsed time $r = 0.58,\ p<.05$2 and enters the visual attention state when

$r = 0.58,\ p<.05$3

with typical values $r = 0.58,\ p<.05$4. After that transition, it measures $r = 0.58,\ p<.05$5 and enters IoI when

$r = 0.58,\ p<.05$6

with $r = 0.58,\ p<.05$7. The visual-only IoI indicator is

$r = 0.58,\ p<.05$8

and the overall decision rule is

$r = 0.58,\ p<.05$9

(Yun et al., 11 May 2026).

The two HRI lines of work differ in formulation but converge on a shared IIS problem: interaction onset is treated as a temporally extended, socially ambiguous state rather than a single keyword event. The museum IIS predicts whether the robot should wait, speak, or listen during the greeting period (Schiffmann et al., 3 Dec 2025). The domestic IoI framework detects whether the user is initiating interaction through speech-plus-facing or prolonged facing alone (Yun et al., 11 May 2026). In both cases, timing and initiation are inferred from embodied cues rather than from explicit trigger words.

5. Architectures and computational realizations

The museum IIS uses frame-level body-language features extracted with Google MediaPipe. The framework provides 543 key landmarks—33 body landmarks, 468 facial landmarks, and 21 landmarks per hand—with $r=-0.04$0, $r=-0.04$1, $r=-0.04$2 coordinates and a visibility score in $r=-0.04$3. The paper states the visibility rule as $r=-0.04$4. In addition, the system computes 52 facial expression shape descriptors, producing feature vectors of 1,682 values per instance (Schiffmann et al., 3 Dec 2025).

Its timing pipeline has two stages: Human Pose Forecasting and Action Classification. The forecasting model is a BlockRNN time series model with input from the last 10 instances and prediction target of the next 5 instances. At 10 frames per second, this corresponds to about 0.5 seconds of lead time. The forecasting model achieved $r=-0.04$5. The second stage is a support vector classifier (SVC) that predicts wait, speak, or listen. For the first 10 data instances, the action classifier is applied directly; from the 11th instance onward, the full pipeline is used. The paper writes the decision logic as

$r=-0.04$6

The best SVM configuration used $r=-0.04$7, $r=-0.04$8, kernel = RBF, and class_weight = balanced; Random Forest was also tested, but the SVM performed better and was selected as the final action classifier (Schiffmann et al., 3 Dec 2025).

The domestic IoI framework uses a ROS-based sensor-fusion architecture rather than a forecasting classifier. Its state transition model includes Monitoring state, Vocal attention state, Visual attention state, and IoI state. From monitoring, the robot can follow the path Monitoring → Vocal attention → Visual attention → IoI for the audio-plus-vision route, or Monitoring → Visual attention → IoI for the silent visual route. Backward transitions occur when evidence is inconsistent, such as speech without a matching face toward the robot, speech directed elsewhere, face detection failure, or background audio from TV or radio (Yun et al., 11 May 2026).

A related but distinct initiation architecture appears in iDian, presented as an entire interaction solution, rather than a front user interface (Rong, 2010). Its multi-layer structure consists of Syntactic Layer, Semantic Layer, Explainer, Executor, and Learner. The workflow begins when the user inputs a natural-language sentence. The Syntactic Layer performs segmentation, dictionary lookup, conversion into integer indices, synonym handling, quotation extraction, number recognition, and special expression handling. The Semantic Layer then applies NUMFORMAT transformation together with general rule-based transformation and specific rule-based transformation, using wildcard characters *N, ?N, #N, and !N. The Explainer tags the command structure as Action (VP), Primary Object (NP), Secondary Object (NP, Optional), and Conditions (PP, Optional), then sends an “End” message to trigger the Executor, which performs the operation in the application. The Learner handles unfamiliar words, suggests nearest known words using a thesaurus dataset, refreshes the dictionary if a suggestion is accepted, and supports online sharing of application-specific special suits (Rong, 2010).

These architectures differ substantially in substrate—forecasting over pose sequences, multimodal state transitions, and rule-based natural-language pipelines—but all instantiate IIS as a mediating layer between a cue of intent and executable system behavior.

6. Evaluation, limitations, controversies, and acronym ambiguity

The HRI systems report measurable but uneven performance. In the museum study, the SVC action classifier achieved 75.3% accuracy on the test data, and the full timing classifier combining forecasting and classification achieved 73.6% accuracy, 74% weighted precision, 74% weighted recall, 74% weighted F1, 69% macro precision, 69% macro recall, and 69% macro F1. The paper notes that wait performed best and listen performed worst, with the confusion behavior indicating that the listen class is the weakest part of the system (Schiffmann et al., 3 Dec 2025). In the domestic IoI framework, AV-IoI achieved Precision: 82.35%, Recall: 70%, and F-measure: 75.68%, while Full-IoI achieved Precision: 86.36%, Recall: 95%, and F-measure: 90.48%. The addition of the vision-only route improved performance substantially, especially recall, because it captured interaction attempts in which the user did not speak (Yun et al., 11 May 2026).

The major limitations are also explicit. The museum study emphasizes that listen is difficult because the distinction between listen and the other classes can be very subtle, involving minimal mouth movement, short and uncertain greetings, and noisy pose readings. It also notes that MediaPipe outputs can be noisy, that recording angle and movement type affect detection accuracy, that class imbalance is structurally hard to avoid, and that the Type Classifier for greeting-style selection was developed only theoretically and not yet evaluated (Schiffmann et al., 3 Dec 2025). The domestic IoI framework identifies small voice / distant speech and face detection failure as key limitations (Yun et al., 11 May 2026). The debugging study identifies a different controversy: whether physiological adaptation can replace human judgment. Its answer is negative for the tested trigger policy, concluding that low pupillary activity is ambiguous and that IPA < baseline − 2 SD for >60 seconds was too simplistic for complex debugging (Golrang et al., 6 May 2026).

A recurrent misconception across these literatures is that interaction initiation can be reduced to a simple trigger. The available evidence argues against that view. In adaptive learning, support timing based on a single physiological criterion can become disruptive (Golrang et al., 6 May 2026). In HRI, keyword-free initiation requires multimodal and temporally persistent evidence rather than a single verbal event (Yun et al., 11 May 2026). In public-space social robotics, deciding whether the robot should greet or listen depends on a short but behaviorally rich greeting period rather than on a fixed script (Schiffmann et al., 3 Dec 2025). In software control, iDian is described not as a menu shortcut or speech recognizer but as a layered mechanism for mapping natural-language intention into executable operations (Rong, 2010).

Finally, the acronym itself is ambiguous. In high-dimensional statistics, IIS refers to innovated interaction screening, the first stage of IIS-SQDA, a two-step procedure for nonlinear classification (Fan et al., 2015). There, IIS screens important interactions by transforming the original $r=-0.04$9-dimensional feature vector with class-specific precision matrices and examining variance differences in only $r=0.31$0 transformed coordinates rather than all two-way interactions of order $r=0.31$1. The method comes with a sure screening property under stated conditions and is followed by sparse quadratic discriminant analysis (SQDA) for further selection and classification (Fan et al., 2015). This usage is unrelated to interaction initiation in learning systems, HRI, or user interfaces. Any technical discussion of IIS therefore requires immediate domain disambiguation.

Taken together, the interaction-oriented IIS literature treats initiation as a design variable with substantive consequences for performance, coordination, and usability. Whether the system is deciding when to display EMME, when a robot should say “hi,” whether a silent user is initiating contact, or how a natural-language command should launch a software action, the decisive issue is the same: initiation is not merely the start of interaction, but a structured inference and control problem whose quality can determine whether the resulting interaction is supportive, natural, or disruptive.