Affective Grow-When-Required Networks

• Affective GWR Networks are dynamic neural systems that incrementally adapt to encode user-specific affective patterns through personalized prototype formation.
• They integrate a Conditional Adversarial Autoencoder with unsupervised, prototype-based growth mechanisms to create responsive, real-time emotion recognition models.
• Dynamic topology management, including neuron growth and edge pruning based on activation thresholds, enhances classification by normalizing intra-subject variability.

Affective Grow-When-Required (GWR) Networks are a class of neural architectures designed to serve as dynamic, personalized memory modules for encoding individuated affective (emotion-related) patterns within an online learning setup. This approach integrates the Grow-When-Required paradigm, developed for dynamic network structure adaptation, within a broader multimodal emotion recognition system based on adversarial autoencoder representations. Affective GWRs enable the continuous, unsupervised adaptation of neural prototypes: as new, user-specific expression data arrives, the topology and content of the memory evolve, incrementally capturing idiosyncratic emotional expressions unaddressed by generic supervised models (Barros et al., 2019).

1. Integration within Personalized Emotion Recognition Models

The foundational model combines a Conditional Adversarial Autoencoder (CAAE) to learn distributed representations of general emotion expressions, with a personalized GWR network to encode user-specific deviation patterns. The CAAE provides an embedding space in which each $\mathbf{x} \in \mathbb{R}^D$ denotes the latent representation of a facial expression. The personalized GWR then operates on this embedding, interpreting it as a sequence of affective events, and incrementally forms specialized prototypes or neurons in response to novel or underrepresented features in the individual's expressive style.

During inference, embeddings routed through both the CAAE and the GWR enable real-time adaptation: the GWR's closest neuron, representative of the subject's typical expressions, enables “editing” of the embedding (e.g., shifting it toward the prototype associated with observed user affect), thereby improving the specificity and reliability of downstream emotion classification tasks (Barros et al., 2019).

2. The Standard GWR Algorithm and Adaptation Mechanisms

The standard GWR algorithm is an unsupervised, prototype-based neural network with explicit topological adaptation. For each input $\mathbf{x}$, the distance to each neuron's weight vector $\mathbf{w}_i$ is calculated via squared Euclidean distance:

$$d(\mathbf{x}, \mathbf{w}_i) = \left\lVert \mathbf{x} - \mathbf{w}_i \right\rVert^2 = \sum_{d=1}^D (x_d - w_{i,d})^2.$$

The best-matching unit (BMU) $b$ and second-best $s$ are identified:

• $b = \arg\min_i d(\mathbf{x}, \mathbf{w}_i)$
• $$s = \arg\min_{i \neq b} d(\mathbf{x}, \mathbf{w}_i)$$

Each neuron maintains a habituation counter $h_i(t)$, quantifying its adaptation and recency of activation. Upon winning (being the BMU), $h_i$ decays:

$h_i(t+1) = h_i(t) - \frac{h_i(t)}{\tau_i}$

where $\tau_i$ is a neuron- or context-specific time constant.

A new neuron is allocated (growth event) if:

• $d(\mathbf{x}, \mathbf{w}_b) > \rho_b$ (activation insufficient)
• $h_b < h_{\min}$ (BMU over-habituated)

Upon insertion, $$w_{\text{new}} = \frac{1}{2}(\mathbf{x} + \mathbf{w}_b)$$ and $h_{\text{new}} = h_b$.

If growth is not triggered, weights are updated for $b$ and its topological neighbors $i \in \mathcal{N}_b$:

$$\Delta \mathbf{w}_i = \varepsilon_i h_i(t) (\mathbf{x} - \mathbf{w}_i), \quad \varepsilon_i = \begin{cases} \varepsilon_b, & i = b \ \varepsilon_n, & i \in \mathcal{N}_b,~i \neq b \end{cases}$$

3. Dynamic Topology and Memory Formation

The GWR network topology is governed by edge management and prototype growth:

• New edges are created between $b$ and $s$ for the current input.
• Edges age monotonically; those surpassing the maximum allowable age are pruned.
• Node deletion is optional, typically applied to units with persistent low habituation or infrequent activation.

Growth hyperparameters—$\rho_b$ (distance threshold), $h_{\min}$ (habituation threshold), $\tau$ (time constants), $\varepsilon$ (learning rates), and MaxEdgeAge—control the structural plasticity and stability of the memory. Seed initialization begins with two prototype neurons; continued online interaction grows and sculpts user-specific topologies that store polymorphic expression prototypes (Barros et al., 2019).

4. Online Training and Personalization Workflow

The affective GWR operates in an online, data-driven personalization loop:

1. The network is initialized with a minimal neuron set.
2. For each incoming embedding $\mathbf{x}$, BMU and second-BMU are found.
3. Edges are incrementally updated between $b$ and $s$, and edge aging is performed.
4. Insertion criteria are evaluated: if $d(\mathbf{x}, \mathbf{w}_b) > \rho_b$ or $h_b < h_{\min}$, a new neuron is inserted; otherwise, BMU and neighbors undergo weight and habituation updates.
5. Edge and node pruning is periodically performed if topology maintenance criteria are met.

A representative update cycle can be formalized as follows:

Require: input embedding x, network G=(V,E) Find b,s ← argmin_i d(x, w_i), and second best Increment age of all edges from b Reset age of edge (b,s) to 0 (or create if absent) if d(x,w_b) > ρ_b or h_b < h_min then Insert new node n w_n ← 0.5*(x + w_b); h_n ← h_b Connect n to b, set age(b,n) ← 0 else For all i in {b} ∪ N_b: Δw_i ← ε_i*h_i*(x - w_i) w_i ← w_i + Δw_i h_i ← h_i - h_i/τ_i Remove edges older than MaxEdgeAge Remove isolated nodes if deletion policy enabled

This pseudocode appears in the reconstructed summary, as the “Methods” details are not quoted verbatim in the paper's abstract (Barros et al., 2019).

5. Influence on Affective Feature Representation and Classification

Trained affective GWRs encode individualized prototypes corresponding to a subject's characteristic modes of affective expression for distinct classes such as “happiness” or “anger.” At test time, a new embedding $\mathbf{x}$ is matched to its BMU $\mathbf{w}_b$; downstream classifiers utilize either:

• Prototype substitution (replace $\mathbf{x}$ with $\mathbf{w}_b$)
• Residual augmentation ($\Delta \mathbf{x} = \mathbf{x} - \mathbf{w}_b$)

This facilitates normalization of intra-subject expressive variability, permitting the final classifier to emphasize affective content over idiosyncratic facial or behavioral features. This suggests an increase in subject-adaptive emotion recognition accuracy, as the classifier operates on features disentangled from identity-dependent confounds (Barros et al., 2019).

6. Empirical Results and Observations

Reported evaluations indicate that integration of a personalized affective GWR memory with a CAAE achieves state-of-the-art emotion recognition performance on in-the-wild datasets. Ablation studies and neural visualizations are conducted to interpret its behavior. A plausible implication is that continual adaptation via GWR allows the memory to retain only salient, user-specific expression patterns, focusing classifier attention on genuine affect rather than superficial or transient features (Barros et al., 2019).

7. Implications, Limitations, and Future Prospects

Affective GWR networks serve as a modular solution for personalized, lifelong adaptation in emotion recognition systems, operating in tandem with deep generative models to bridge the gap between generic and idiosyncratic affective display. Limitations stem from the lack of direct information in the study on specific hyperparameter schedules and deletion mechanisms; further empirical results may clarify optimal growth parameters for various affective representation domains. Extensions may involve coupling GWRs with additional data modalities, such as vocal or physiological signals, or deploying them in continual learning pipelines for real-world emotion-aware human–machine interfaces (Barros et al., 2019).