Weighted Instance Aggregation Network (WIAN)

• WIAN is a graph-enhanced, context-aware aggregation module that selectively weights video frames to boost salient facial cues.
• It integrates dynamic weight gates within an LSTM, combined with multi-head self-attention, to produce robust video representations.
• Demonstrated in MICACL, WIAN mitigates head-class bias and improves minority class recall in long-tailed dynamic facial expression datasets.

Searching arXiv for the cited WIAN-related papers to ground the article in current sources. Weighted Instance Aggregation Network (WIAN) is the aggregation backbone in the MICACL framework for long-tailed dynamic facial expression recognition (DFER), where it converts a bag of temporally ordered instance features into a single, class-discriminative video representation while explicitly accounting for the uneven importance of instances (Cui et al., 4 Sep 2025). In MICACL, WIAN operates after the Graph-Enhanced Instance Interaction Module (GEIIM), uses graph-enhanced instance features to derive context-aware importance weights, injects those weights into a gated LSTM update through a Dynamic Weight Gate (DWG), and then refines the resulting hidden-state sequence with multi-head self-attention before producing the final video-level feature used for classification and contrastive learning (Cui et al., 4 Sep 2025). Related literature contains conceptually similar weighted instance aggregation mechanisms, notably the instance-weighted label aggregation of "Deep Weighted Averaging Classifiers" (Card et al., 2018) and the query-conditioned support-instance aggregation in "Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification" (Ye et al., 2019), although the explicit term WIAN is used in MICACL.

1. Problem formulation and motivation

WIAN is formulated in a multi-instance learning (MIL) setting in which a video is a bag containing $T$ instances, such as frames or short clips, and only a subset of those instances carry emotion cues (Cui et al., 4 Sep 2025). In long-tailed DFER, this setting is especially difficult because head classes dominate the training distribution, minority categories offer far fewer clips and often present subtle, short-lived expressions, and in-the-wild videos contain many neutral or noisy frames (Cui et al., 4 Sep 2025). Aggregation must therefore be selective rather than uniform.

The MICACL formulation identifies two failure modes of naive temporal pooling. Mean pooling treats all instances equally, which dilutes salient frames, whereas max pooling can overemphasize spurious peaks; together these behaviors induce head-class bias and low recall for weak expressions (Cui et al., 4 Sep 2025). The aggregated feature space is then dominated by head-style dynamics, and tail classes with few clips and weak expressions fail to surface their critical frames (Cui et al., 4 Sep 2025).

WIAN addresses this by learning instance importance and gating the contribution of each instance into the video-level representation, with the importance guided by graph-enhanced context from GEIIM rather than computed from an isolated instance descriptor (Cui et al., 4 Sep 2025). This design makes the weighting context-aware and noise-robust, because instances connected to salient neighbors can be emphasized while isolated or noisy instances can be downweighted (Cui et al., 4 Sep 2025).

2. Placement within MICACL and data flow

MICACL comprises four stages: backbone feature extraction, GEIIM, WIAN, and classifier-plus-losses (Cui et al., 4 Sep 2025). The backbone is R3D, and in implementation each video is represented by $T=16$ sampled frames with resized $112\times112$ faces, producing instance features with batch-form tensor

$X \in \mathbb{R}^{B\times T\times C},$

where $B$ is batch size, $T$ is the number of instances per video, and $C$ is the backbone channel dimension (Cui et al., 4 Sep 2025).

GEIIM first constructs an adaptive adjacency matrix and propagates features to obtain context-enriched per-instance features $H$ (Cui et al., 4 Sep 2025). WIAN then consumes both the original instance sequence $\{x_t\}_{t=1}^T$ and the GEIIM output $H$, computes dynamic gates $T=16$0 and weights $T=16$1 for each instance, aggregates the sequence through a gated LSTM, refines the hidden states with multi-head self-attention, and outputs a single aggregated representation

$T=16$2

for classification and MCCL (Cui et al., 4 Sep 2025).

The input-output interface is therefore explicit. The input instances satisfy $T=16$3 with $T=16$4 in practice, the GEIIM context input is $T=16$5, and the output is a single video representation $T=16$6 (Cui et al., 4 Sep 2025). Internally, WIAN produces hidden states $T=16$7, then applies multi-head self-attention and pooling before projection back to classifier dimension $T=16$8 (Cui et al., 4 Sep 2025).

3. Mathematical structure of weighted aggregation

The upstream GEIIM context used by WIAN is defined through adaptive graph construction. GEIIM learns node embeddings $T=16$9 and $112\times112$0, constructs

$112\times112$1

and propagates features by

$112\times112$2

with learnable $112\times112$3 (Cui et al., 4 Sep 2025). In this role, $112\times112$4 supplies graph-enhanced, noise-suppressed instance features to WIAN.

WIAN derives instance importance from the GEIIM output rather than directly from raw $112\times112$5. For each time step $112\times112$6,

$112\times112$7

yielding a weight vector aligned with the feature dimension (Cui et al., 4 Sep 2025). In practice, this vector is mapped through learned linear projection or broadcasting to match the dimensionality of the LSTM gate, retaining per-channel saliency while enabling per-step gating (Cui et al., 4 Sep 2025).

The central mechanism is the Dynamic Weight Gate. WIAN augments a standard LSTM with a gate that scales the candidate cell update by instance importance:

$112\times112$8

where $112\times112$9 is Sigmoid, $X \in \mathbb{R}^{B\times T\times C},$0, $X \in \mathbb{R}^{B\times T\times C},$1, and $X \in \mathbb{R}^{B\times T\times C},$2 (Cui et al., 4 Sep 2025). The standard LSTM gates and candidate state are

$X \in \mathbb{R}^{B\times T\times C},$3

WIAN inserts $X \in \mathbb{R}^{B\times T\times C},$4 into the cell update:

$X \in \mathbb{R}^{B\times T\times C},$5

$X \in \mathbb{R}^{B\times T\times C},$6

This makes an instance’s contribution dependent on both temporal context, through $X \in \mathbb{R}^{B\times T\times C},$7, and graph-enhanced saliency, through $X \in \mathbb{R}^{B\times T\times C},$8 (Cui et al., 4 Sep 2025).

The recurrent sequence is subsequently refined by multi-head self-attention:

$X \in \mathbb{R}^{B\times T\times C},$9

and then aggregated by pooling and projected to classifier dimension:

$B$0

The weighted aggregation core is therefore not a simple weighted sum at sequence output; it is embedded inside the recurrent state transition itself (Cui et al., 4 Sep 2025).

4. Training objective, regularization, and efficiency

WIAN is trained jointly with the rest of MICACL through cross-entropy classification and Multiscale Category-aware Contrastive Learning (MCCL) (Cui et al., 4 Sep 2025). The cross-entropy term is applied to logits derived from $B$1:

$B$2

while MCCL receives $B$3 and constructs multiscale projections, category-aware weights, and scale-specific similarities (Cui et al., 4 Sep 2025). The full training objective is

$B$4

with $B$5 controlling the balance (Cui et al., 4 Sep 2025). The role division is explicit in the source: MCCL emphasizes tail categories via $B$6 weighting and hard samples via $B$7, complementing WIAN’s instance-selective aggregation (Cui et al., 4 Sep 2025).

No explicit entropy or sparsity penalty on $B$8 is reported (Cui et al., 4 Sep 2025). Robustness is instead attributed to GEIIM normalization, specifically the softmax normalization of the adaptive adjacency matrix, and to the DWG’s sigmoid gating (Cui et al., 4 Sep 2025). This is significant because it distinguishes WIAN from weighting schemes that enforce sparsity directly through additional regularizers.

The implementation details pertinent to WIAN are also specified. The backbone is R3D with Torchvision pretrained weights; the model uses $B$9 frames per video; faces are resized to $T$0; optimization uses AdamW with cosine decay, initial learning rate $T$1, minimum learning rate $T$2, weight decay $T$3, and $T$4 epochs on an RTX A6000 GPU (Cui et al., 4 Sep 2025). Batch size, gate hidden size $T$5, and $T$6 are tuned empirically and are not explicitly reported (Cui et al., 4 Sep 2025).

From a computational perspective, WIAN scales linearly in the sequence length $T$7 (Cui et al., 4 Sep 2025). Its per-step cost is $T$8 for computing $T$9 through softmax over feature channels and $C$0 for the DWG mapping and LSTM gates, giving per-video complexity

$C$1

(Cui et al., 4 Sep 2025). GEIIM adds $C$2 for adjacency construction and propagation, and multi-head self-attention adds a heads$C$3 parameter overhead (Cui et al., 4 Sep 2025). The MICACL system reports $C$4 FLOPs on DFEW, which the source presents as efficient relative to transformer-heavy baselines (Cui et al., 4 Sep 2025).

5. Empirical behavior and ablation evidence

On DFEW under 5-fold evaluation, MICACL achieves WAR $C$5 and UAR $C$6, with minority-class results including disgust $C$7 and fear $C$8, described as far above prior $C$9–$H$0 ranges (Cui et al., 4 Sep 2025). On FERV39k, MICACL reaches WAR $H$1 and UAR $H$2, surpassing prior state of the art in UAR (Cui et al., 4 Sep 2025).

Ablation evidence on DFEW Fold 5 isolates the contribution of WIAN relative to GEIIM and MCCL. The source reports the following configurations (Cui et al., 4 Sep 2025):

Configuration	WAR	UAR
Baseline (no GEIIM/WIAN/MCCL)	64.55	54.17
+GEIIM	68.11	57.39
+GEIIM+WIAN	68.62	56.26
+GEIIM+WIAN+MCCL	69.91	64.34

These results support two distinct observations. First, WIAN improves overall accuracy relative to GEIIM alone, as seen in the WAR increase from $H$3 to $H$4 (Cui et al., 4 Sep 2025). Second, WIAN alone does not maximize balanced recall under long-tailed conditions; UAR decreases from $H$5 to $H$6 when MCCL is absent, and then rises to $H$7 when MCCL is added (Cui et al., 4 Sep 2025). A common misconception is therefore that selective instance weighting by itself solves category imbalance. The ablation instead indicates that WIAN’s selective aggregation and MCCL’s category-aware reweighting are complementary rather than interchangeable (Cui et al., 4 Sep 2025).

The reported failure cases are consistent with this interpretation. In extremely noisy or near-neutral sequences, if GEIIM fails to construct informative adjacency, then $H$8 may underdiscriminate and the DWG may suppress genuine subtle cues (Cui et al., 4 Sep 2025). For very short sequences, limited $H$9 reduces the benefit of graph-enhanced context and gating, and mean pooling may be competitive when dynamics are minimal (Cui et al., 4 Sep 2025). Performance is also sensitive to $\{x_t\}_{t=1}^T$0, gate hidden size $\{x_t\}_{t=1}^T$1, and the projection aligning $\{x_t\}_{t=1}^T$2 to gate dimensionality (Cui et al., 4 Sep 2025).

6. Relation to other weighted-instance aggregation paradigms

Within the supplied literature, WIAN has both a narrow and a broader comparative meaning. Narrowly, it denotes the specific MICACL module that derives $\{x_t\}_{t=1}^T$3 from GEIIM-enhanced features, injects those weights into an LSTM cell through DWG, and refines the resulting sequence with multi-head self-attention (Cui et al., 4 Sep 2025). More broadly, related work contains architectures that aggregate instances by learned weights, even when they do not use the name WIAN. This suggests a family resemblance across exemplar-weighted classification, query-conditioned support aggregation, and context-aware temporal MIL.

"Deep Weighted Averaging Classifiers" replaces a final softmax with instance-weighted aggregation over training examples in a learned embedding space (Card et al., 2018). Its embedding is

$\{x_t\}_{t=1}^T$4

its kernel weight is

$\{x_t\}_{t=1}^T$5

with fixed $\{x_t\}_{t=1}^T$6, and its class prediction is a normalized weighted sum of one-hot training labels (Card et al., 2018). In the comparative description provided for DWAC, this model is described as a prototypical WIAN in the sense that it embeds inputs with a deep network, computes kernel-based weights to training instances, and aggregates labels with normalized weights (Card et al., 2018). The difference from MICACL’s WIAN is structural: DWAC aggregates over stored training instances at inference, whereas MICACL’s WIAN aggregates temporally ordered instances within a video and places the weighting mechanism inside recurrent state updates (Card et al., 2018).

"Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification" provides a second comparison point (Ye et al., 2019). There, support instances are aggregated by query-conditioned attention:

$\{x_t\}_{t=1}^T$7

The supplied comparative discussion explicitly characterizes this attentive aggregation as a WIAN because it aggregates support instances through learned, query-conditioned weights (Ye et al., 2019). In contrast to MICACL, the weighting signal is conditioned on a query representation rather than on graph-enhanced temporal context, and the aggregate is a class prototype rather than a video descriptor (Ye et al., 2019).

These comparisons clarify what distinguishes MICACL’s WIAN from simpler alternatives. Versus mean or max pooling, WIAN learns per-instance weights conditioned on temporal context; versus standard attention-based MIL such as the Ilse et al. style described in the source, WIAN derives $\{x_t\}_{t=1}^T$8 from GEIIM-enhanced features $\{x_t\}_{t=1}^T$9 and injects them inside an LSTM cell via DWG rather than forming only a single weighted sum (Cui et al., 4 Sep 2025). A plausible implication is that the term WIAN is most precise when reserved for architectures in which instance weighting is part of the aggregation dynamics rather than only an output-stage reweighting.

7. Conceptual significance and scope of applicability

In MICACL, WIAN is presented as the aggregation engine that turns instance features into a robust video-level representation via context-aware weighted gating (Cui et al., 4 Sep 2025). Its defining computations are the importance scores

$H$0

the dynamic gate

$H$1

and the gated cell update

$H$2

(Cui et al., 4 Sep 2025). In this sense, the essential contribution is not merely weighted aggregation, but weighted recurrent aggregation with graph-aware context.

Its significance in long-tailed DFER follows directly from the design constraints stated in the source. Minority categories are weakly represented and often subtle; in-the-wild sequences contain many neutral, occluded, or off-angle frames; and naive pooling induces decision boundaries biased toward head classes (Cui et al., 4 Sep 2025). WIAN is therefore aimed at suppressing noisy instances and boosting salient ones before the final classification and contrastive objectives operate (Cui et al., 4 Sep 2025).

At the same time, the ablation and limitation analysis bounds the scope of the claim. WIAN alone does not rebalance categories, does not guarantee improved UAR in isolation, and can underperform when graph context is uninformative or sequence length is too short (Cui et al., 4 Sep 2025). Its strongest reported behavior emerges when it is coupled with GEIIM’s adjacency-aware context and MCCL’s category-aware contrastive optimization, producing improved overall accuracy and balanced recall on DFEW and FERV39k (Cui et al., 4 Sep 2025).