NeuroGaze-Distill: EEG-Informed FER Distillation

• NeuroGaze-Distill is a cross-modal distillation framework that transfers brain-informed priors into an image-only facial emotion recognition model using static V/A prototypes and a depression-inspired geometric constraint.
• The framework decouples EEG-based prior formation from visual inference by training a standard ResNet-based student with cosine similarity and KL divergence to align features with neuro-informed prototypes.
• The approach improves cross-dataset robustness and class separability in FER by importing affective structure from EEG signals while maintaining low inference complexity.

Searching arXiv for the main paper and closely related naming context. arXiv search: "(Li et al., 15 Sep 2025) NeuroGaze-Distill" NeuroGaze-Distill is a cross-modal distillation framework for facial emotion recognition (FER) that transfers brain-informed priors into an image-only student by combining static Valence/Arousal (V/A) prototypes with a depression-inspired geometric prior (D-Geo) (Li et al., 15 Sep 2025). It is motivated by the observation that FER models trained only on pixels often generalize poorly across datasets because facial appearance is an indirect and biased proxy for underlying affect, whereas physiological signals such as EEG encode affective dynamics more directly and are less tied to observable appearance. The framework therefore uses EEG only to form priors, then trains a standard image-based FER model that requires no EEG-face pairing and no non-visual signals at deployment.

1. Problem formulation and conceptual basis

NeuroGaze-Distill addresses a recurrent failure mode in FER: models that learn only from images often generalize poorly across datasets due to biases in facial appearance and variations in data labeling, demographics, and imaging conditions (Li et al., 15 Sep 2025). The method takes the position that robustness can be improved by shaping the facial embedding space with priors derived from a different modality, rather than by increasing architectural complexity in the image model itself.

Its central design choice is to treat EEG-derived affect structure as a reusable prior rather than as a co-equal deployment modality. A teacher trained on EEG topographic maps from DREAMER, with MAHNOB-HCI as unlabeled support, produces a consolidated $5 \times 5$ V/A prototype grid that is frozen and reused. The student is then trained only on face images. This decouples prior formation from visual inference and eliminates the need for paired EEG-face data at any point.

The framework combines two forms of regularization. The first is Proto-KD, which aligns student features to static EEG-informed prototypes through cosine-based similarity and a KL objective over prototype bins. The second is D-Geo, a weak geometric regularizer motivated by affective findings often reported in depression research, including anhedonia-like contraction in high-valence regions. This combination places NeuroGaze-Distill at the intersection of cross-modal knowledge distillation, affect-space representation learning, and geometry-aware regularization.

2. Architecture and learning pipeline

The teacher network takes EEG topographic maps as input and is implemented as a CNN- or ViT-based regressor trained to predict valence/arousal coordinates from EEG topomaps (Li et al., 15 Sep 2025). Its penultimate-layer embedding vectors are used to form static prototypes in V/A space. DREAMER supplies EEG topomaps with V/A labels, and MAHNOB-HCI is used as unlabeled support.

The student network is a standard ResNet-18 or ResNet-50 backbone with a 256-dimensional feature projection followed by an 8-class FER classifier. Its outputs are 256-D L2-normalized features and 8-way classification logits. The main face dataset is FERPlus, with evaluation extending to AffectNet-mini and optionally CK+.

The training objective combines standard supervised learning and distillation with two lightweight regularizers. In the paper’s notation,

$$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$

Here, CE uses label smoothing and class weights; $\mathcal{L}_{\mathrm{KD}}$ is logit distillation with temperature; the prototype term regularizes student assignments toward the EEG-derived prototype prior; and $\mathcal{L}_{\mathrm{D\text{-}Geo}}$ imposes the depression-inspired geometric prior. The weights are chosen small so that the main classification loss dominates and the regularization remains light.

This design is intentionally conservative in deployment terms. The student remains a standard ResNet-18/50, with no increase in inference cost versus typical FER models. A plausible implication is that the contribution of the framework lies primarily in representation shaping rather than in changes to the inference graph.

3. Static V/A prototypes and Proto-KD

The prototype mechanism is the core cross-modal bridge in NeuroGaze-Distill (Li et al., 15 Sep 2025). Instead of distilling from paired EEG-face data, the method aggregates the teacher’s validation embeddings after discretizing continuous V/A coordinates into a $5 \times 5$ grid. V/A coordinates are linearly mapped to $[-1, 1]$, and the grid centers range from $-0.8$ to $0.8$. Each bin covers a local region in V/A space.

Teacher validation embeddings are binned by V/A coordinate, and the mean embedding vector is computed for each bin, yielding 25 frozen EEG-informed static prototypes. If a bin is empty, its prototype is filled using the nearest non-empty bin’s mean. The process also yields a prior distribution $q^\text{pro}$ over bins, proportional to sample counts in each bin. The resulting prototype bank is formed once, frozen, and reused across all experiments and datasets.

During student training, each face image is projected to a 256-D feature vector and L2-normalized. Cosine similarities to the 25 static prototypes are then computed:

$s_k = \cos(f(x), p_k).$

These similarities are transformed into a soft assignment over V/A bins using softmax with temperature $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$0:

$$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$1

The student distribution $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$2 is regularized via KL divergence toward $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$3.

This formulation makes the distillation cross-modal in a specific sense: the image-only student is not asked to match EEG signals directly, but to align its embedding geometry with a static, neurophysiologically derived affect structure. Because the prototypes are frozen and reused, no EEG data or non-visual features are used for the student or for inference. This suggests a separation between prior construction and operational inference that is unusual in multimodal FER systems.

4. Depression-inspired geometric prior

D-Geo is the framework’s second regularization mechanism and is described as a depression-inspired geometric prior (Li et al., 15 Sep 2025). It is motivated by affective findings often reported in depression research: anhedonia is characterized as a blunting or contraction of neural and affective responses to positive, high-valence stimuli. In the V/A circumplex, this is reflected as contracted representational geometry for high-valence emotions, while negative or neutral regions remain more varied.

The method operationalizes this idea as a weak, non-diagnostic, dataset-wide geometric regularizer. High-valence emotion classes are defined as

$$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$4

On each batch, the model computes class means $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$5 and variances $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$6 for the 256-D features. For $$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$7, it penalizes excessive intra-class spread through a variance cap:

$$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$8

It also enforces global separability through a margin term over all class pairs:

$$\mathcal{L} = \text{CE (with LS+CW)} + \lambda_{\mathrm{kd}} \mathcal{L}_{\mathrm{KD}} + \lambda_{\mathrm{proto}} D_{\mathrm{KL}}(q^\mathrm{pro} \,\|\, q^\mathrm{stu}) + \lambda_{\mathrm{geo}} \mathcal{L}_{\mathrm{D\text{-}Geo}}.$$9

The combined D-Geo term is activated with a late-start cosine ramp schedule from epochs 20 to 60.

The paper emphasizes that this prior is non-diagnostic and acts only as a weak geometric constraint. It compacts high-valence categories while preserving margins elsewhere. This is important because a likely misconception would be to interpret D-Geo as a clinical or subject-level depression detector; the formulation explicitly does not make such a claim.

Component	Description	Formula / policy
CE (LS+CW)	Supervised learning with label smoothing and class weights	Main classification term
Logit KD	Logit distillation from vision teacher	MSE/KL on logits; temperature $\mathcal{L}_{\mathrm{KD}}$0
Prototype KD (Proto-KD)	Cosine similarity to static neuro-informed prototypes	$\mathcal{L}_{\mathrm{KD}}$1 via softmax of cosine similarities
D-Geo	Depression-inspired geometric regularizer	Variance cap for high-valence classes and inter-class margin globally

5. Experimental protocols, ablations, and reported performance

The experimental setup separates EEG-side prior formation from face-side FER evaluation (Li et al., 15 Sep 2025). The EEG teacher uses DREAMER with MAHNOB-HCI as unlabeled support. The student is trained on FERPlus and evaluated both within-domain and under cross-dataset transfer. Within-domain evaluation uses FERPlus validation with 8-way emotion recognition. Cross-dataset evaluation applies the FERPlus-trained model to AffectNet-mini and optionally CK+.

The evaluation protocol reports standard 8-way scores and also present-only metrics to handle label-set mismatch fairly. The reported metrics are Accuracy, Macro-F1, Balanced Accuracy (mean per-class recall), and Macro-AUROC. Present-only Macro-F1 and balanced accuracy are highlighted because only classes present in the target are considered.

The ablation structure uses four variants: B0 (CE only), B1 (CE + logit KD), B2 (B1 + Proto-KD), and B3 (B2 + D-Geo, the full method). Additional ablations vary loss weights, grid size, D-Geo scheduling, and backbone. These studies attribute consistent gains to prototypes and D-Geo and favor a $\mathcal{L}_{\mathrm{KD}}$2 grid over denser alternatives such as $\mathcal{L}_{\mathrm{KD}}$3 for stability.

Within FERPlus, Macro-F1 rises from 51.29 for CE only to 63.56 with KD, 64.21 with Proto-KD, and 64.74 for the full method with D-Geo. Balanced Accuracy and overall accuracy also improve, especially with Proto-KD and D-Geo. On AffectNet-mini, accuracy and Macro-F1 on present-only classes reach approximately 76% with the full method trained on FERPlus. CK+ performance is lower due to dataset size and class imbalance, but present-only metrics are reported as a fairer transfer view.

These results support two narrow conclusions stated in the paper: first, the static prototype prior contributes to cross-dataset robustness; second, D-Geo further improves or preserves class separability while shaping high-valence geometry. A plausible implication is that robustness gains arise from regularizing affect structure rather than from stronger dataset-specific fitting.

6. Embedding behavior, deployment properties, and nomenclature

Qualitative analysis using t-SNE and UMAP indicates that Proto-KD improves class cluster separation and aligns features to neuro-informed structure, while D-Geo further compacts high-valence clusters, especially happiness and surprise, while maintaining margin between classes (Li et al., 15 Sep 2025). The paper interprets this as making the learned embedding better mirror affective geometry and less tied to spurious dataset artifacts. Because this claim is qualitative, it is best read as an observed embedding-space tendency rather than a formal proof of causal invariance.

The implementation emphasizes deployability. Training uses AdamW, a cosine learning-rate schedule, mixed precision (AMP), channels-last format, gradient clipping, label smoothing, and class reweighting. Models and prototypes are versioned via SHA-256, and training/evaluation logs, metrics, and ablation fingerprints are provided for reproducibility. Crucially, the prototype bank is formed once, frozen, and reused; EEG or other non-visual data are not needed during student training or inference, and no paired EEG-face data are required at any point.

A common source of confusion is nomenclature. NeuroGaze-Distill is distinct from “NeuroGaze,” which is a hybrid EEG and eye-tracking brain-computer interface for hands-free interaction in virtual reality rather than a facial emotion recognition framework (Coutray et al., 9 Sep 2025). NeuroGaze-Distill uses EEG topographic maps only to construct static affective priors and ultimately produces a deployable image-only FER model. The overlap in naming reflects the use of neurophysiological signals, but the tasks, inputs, and deployment assumptions are different.

Taken together, NeuroGaze-Distill is best understood as a brain-informed regularization strategy for FER: EEG is used to define a reusable prior over affect space, D-Geo shapes the student embedding with a weak geometric constraint, and the final inference system remains a standard ResNet-based image classifier. Its contribution lies in showing that robustness can be improved by importing cross-modal structure into the training objective without carrying multimodal complexity into deployment.