ET-BCI Fusion Algorithm Overview

• ET-BCI fusion algorithm is a multimodal strategy that combines eye-tracking and EEG data to enhance intent detection in clinical settings.
• The method involves real-time, parallel acquisition and preprocessing with modality-specific confidence score calculations and Bayesian decision rules.
• Empirical results demonstrate robust classification accuracies (>90%) and dynamic modality switching to support communication in progressive neurodegenerative conditions.

An ET-BCI fusion algorithm refers broadly to any computational strategy that integrates eye-tracking (ET) data with brain–computer interface (@@@@1@@@@) signals, most commonly EEG, to enhance the accuracy, robustness, or adaptability of user intention detection. This approach is particularly salient for clinical and assistive communication contexts, such as maintaining reliable communication in patients with progressive motor and oculomotor impairment, or supporting seamless operation in hybrid human–machine interfaces. The following sections comprehensively delineate the technical principles, methodological specifics, and empirical impact of ET-BCI fusion, with focus on real-time, classifier-level, and confidence-driven integration within clinical BCI paradigms (Pinto et al., 27 Sep 2025).

1. Motivation and Context for ET-BCI Fusion

The impetus for integrating ET with BCI is grounded in compensating for the progressive loss of communicative motor abilities, such as in Amyotrophic Lateral Sclerosis (ALS) patients transitioning from Locked-In Syndrome (LIS) to Complete Locked-In Syndrome (CLIS). While eye-tracking enables intuitive, muscle-based interface control during early LIS, its utility degrades as oculomotor control diminishes. Conversely, EEG-based BCIs provide a muscle-independent channel but typically exhibit lower accuracy in the presence of fatigue, reduced motivation, or loss of goal-directed cognitive states. Fusion algorithms act to optimize communication continuity by adaptively leveraging the most informative modality at any given stage, thus ensuring both responsiveness and robustness during progressive neurodegeneration (Pinto et al., 27 Sep 2025).

2. Data Acquisition and Preprocessing Pipeline

ET-BCI fusion architectures necessitate the simultaneous, synchronized acquisition of both gaze and EEG data streams:

• Eye-Tracking (ET): High-resolution gaze coordinates and pupil diameter are captured via devices such as the Tobii Pro Spark, typically sampled at ≥60 Hz using frameworks such as Titta. These data provide real-time measures of gaze focus and intensity.
• EEG: Multi-channel EEG is acquired (e.g., sampling rate 256 Hz, using g.USBamp), filtered with 0.1–30 Hz bandpass and 50 Hz notch filters. Real-time artifact removal and segmentation are implemented within environments such as Simulink.

Preprocessing operations aim to extract features that are informative for intention detection, balancing temporal precision (critical for P300/oddball paradigms) and noise reduction.

3. Confidence Score Calculation for Each Modality

Central to the fusion approach is the calculation of a modality-specific confidence score at each decision epoch:

• Eye-Tracking Confidence: For each Area of Interest (AOI), the confidence is computed as the ratio

$C_{\mathrm{ET}}(k) = \frac{n_k}{N_{\text{total}}}$

where $n_k$ is the number of gaze points within AOI $k$, and $N_{\text{total}}$ is the total number of gaze points for the trial (Pinto et al., 27 Sep 2025). This metric captures the relative focus of gaze on each candidate target.

• EEG (BCI) Confidence: Feature vectors from EEG epochs are classified using a Gaussian Naïve Bayes approach. The Bayesian score for class $i$ at event $k$ is computed as

$$\text{Score}_i^k = \text{Prior}_i^\top \cdot \mathrm{pdf}_{\text{Gauss}}(x | \mu_i, \Sigma_i)$$

with $\mu_i$, $\Sigma_i$ the mean and covariance for class $i$ and $\text{Prior}_i$ the class prior. The normalized confidence for target detection is then

$$C_{\mathrm{EEG}}^k = 1 - \frac{\max(\text{Score}_1^k, \text{Score}_2^k)}{\text{Score}_1^k + \text{Score}_2^k}$$

This provides a probabilistic measure of evidence for a particular class, normalized within the trial.

4. Decision-Level Fusion and Thresholding Strategy

Fusion of ET and BCI confidence scores is realized through a conditional selection mechanism that exploits the strengths of both modalities:

• For each candidate class $k$ (AOI), classes are ranked in descending order of $C_{\mathrm{EEG}}^k$.
• The selected class $\hat{y}(t)$ for trial $t$ is determined as the highest-ranked class for which $C_{\mathrm{ET}}^k$ exceeds an empirically set threshold (e.g., 0.85):

$$\hat{y}(t) = \arg\max_{k} \bigl( C_{\mathrm{EEG}}^k(t) \;\big|\; C_{\mathrm{ET}}^k(t) \geq \text{threshold} \bigr)$$

• If no class exceeds the ET threshold, the algorithm proceeds to the next-highest EEG candidate (Pinto et al., 27 Sep 2025).

This rule ensures that only options with corroborative evidence from both gaze and neural response are selected, reducing false positives and improving reliability when modalities disagree.

5. Real-Time Operation and Transition Handling

All computations are performed with low latency, permitting real-time interaction. Both gaze and EEG features are processed and fused in parallel within the user interface loop, with continuous monitoring of user attention via gaze centroids.

A key feature of the ET-BCI algorithm is its ability to support dynamic transitions:

• In early stages of LIS, ET data dominates, enabling fast and accurate selections as long as reliable eye movements persist.
• As oculomotor ability declines (progression to CLIS), the system relies increasingly on BCI (EEG) confidence, providing a gradual adaptation pathway.
• The redundancy between ET and EEG streams mitigates communication breakdown, and the system can be tuned to increase EEG weight as required.

6. Empirical Evaluation and Clinical Implications

In pilot testing with healthy participants ($N = 5$), the fusion strategy achieved high classification accuracies ($>90\%$), with low variance across trials (Pinto et al., 27 Sep 2025). The threshold was set at 0.85 to balance specificity and sensitivity, and only one trial with deliberate gaze diversion led to misclassification. This result demonstrates the algorithm’s robustness and its capacity to accommodate transient lapses in either signal stream.

For clinical contexts, a major implication is the potential to avert communication gaps during the transition from LIS to CLIS, maintaining engagement and goal-directed behavior critical for BCI performance. Early adoption of the hybrid ET-BCI system may also forestall extinction of volitional cognitive control, an observed risk in late-stage disease.

7. Algorithmic Summary and Theoretical Considerations

The ET-BCI fusion approach in (Pinto et al., 27 Sep 2025) is characterized by the following key points:

• Parallel, real-time acquisition and preprocessing of ET and EEG signals.
• Modality-specific computation of confidence scores: $C_{\mathrm{ET}}(k)$ per AOI from gaze density; $C_{\mathrm{EEG}}^k$ from probabilistic EEG classification.
• A thresholded decision rule that selects an action only when both modalities agree above threshold, prioritizing EEG confidence in ambiguous cases.
• Real-time operation, with low-latency update of selection predictions and adaptive handling of modality reliability.

A plausible implication is that this modular, threshold-based fusion architecture is extensible to other multimodal integration tasks where signal reliability shifts dynamically over time or disease course. Threshold selection, modality weighting, and real-time calibration remain active research areas for optimizing hybrid BCI performance in diverse end-user populations.

---

Table: Summary of Key ET-BCI Fusion Algorithm Steps (Pinto et al., 27 Sep 2025)

Step	EEG (BCI) Processing	Eye-Tracking (ET) Processing
Acquisition	Multi-electrode EEG, 256 Hz, real-time preprocessing	60 Hz gaze coordinates, AOI, gaze centroid
Feature/Score	Bayesian classifier, $C_{\mathrm{EEG}}^k$ per AOI	Gaze ratio, $C_{\mathrm{ET}}(k)$ per AOI
Fusion Rule	Ranked by $C_{\mathrm{EEG}}^k$, select with $C_{\mathrm{ET}}(k)\geq$ threshold	Threshold gating ($0.85$)
Output	Predicted class/word, real-time update	Validates selection, monitors focus

This methodology supports robust, high-accuracy intent detection, especially valuable in clinical assistive communication for progressive neurodegenerative conditions.