---
title: Weighted Instance Aggregation Network (WIAN)
url: https://www.emergentmind.com/topics/weighted-instance-aggregation-network-wian
type: topic
---

# Weighted Instance Aggregation Network (WIAN)

Searching arXiv for the cited WIAN-related papers to ground the article in current sources.
Weighted Instance Aggregation Network (WIAN) is the aggregation backbone in the MICACL framework for long-tailed dynamic facial expression recognition (DFER), where it converts a bag of temporally ordered instance features into a single, class-discriminative video representation while explicitly accounting for the uneven importance of instances [2509.04344]. In MICACL, WIAN operates after the Graph-Enhanced Instance Interaction Module (GEIIM), uses graph-enhanced instance features to derive context-aware importance weights, injects those weights into a gated LSTM update through a Dynamic Weight Gate (DWG), and then refines the resulting hidden-state sequence with multi-head self-attention before producing the final video-level feature used for classification and contrastive learning [2509.04344]. Related literature contains conceptually similar weighted instance aggregation mechanisms, notably the instance-weighted label aggregation of "Deep Weighted Averaging Classifiers" [1811.02579] and the query-conditioned support-instance aggregation in "Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification" [1906.06678], although the explicit term WIAN is used in MICACL.

## 1. Problem formulation and motivation

WIAN is formulated in a multi-instance learning (MIL) setting in which a video is a bag containing $T$ instances, such as frames or short clips, and only a subset of those instances carry emotion cues [2509.04344]. In long-tailed DFER, this setting is especially difficult because head classes dominate the training distribution, minority categories offer far fewer clips and often present subtle, short-lived expressions, and in-the-wild videos contain many neutral or noisy frames [2509.04344]. Aggregation must therefore be selective rather than uniform.

The MICACL formulation identifies two failure modes of naive temporal pooling. Mean pooling treats all instances equally, which dilutes salient frames, whereas max pooling can overemphasize spurious peaks; together these behaviors induce head-class bias and low recall for weak expressions [2509.04344]. The aggregated feature space is then dominated by head-style dynamics, and tail classes with few clips and weak expressions fail to surface their critical frames [2509.04344].

WIAN addresses this by learning instance importance and gating the contribution of each instance into the video-level representation, with the importance guided by graph-enhanced context from GEIIM rather than computed from an isolated instance descriptor [2509.04344]. This design makes the weighting context-aware and noise-robust, because instances connected to salient neighbors can be emphasized while isolated or noisy instances can be downweighted [2509.04344].

## 2. Placement within MICACL and data flow

MICACL comprises four stages: backbone feature extraction, GEIIM, WIAN, and classifier-plus-losses [2509.04344]. The backbone is R3D, and in implementation each video is represented by $T=16$ sampled frames with resized $112\times112$ faces, producing instance features with batch-form tensor
$$
X \in \mathbb{R}^{B\times T\times C},
$$
where $B$ is batch size, $T$ is the number of instances per video, and $C$ is the backbone channel dimension [2509.04344].

GEIIM first constructs an adaptive adjacency matrix and propagates features to obtain context-enriched per-instance features $H$ [2509.04344]. WIAN then consumes both the original instance sequence $\{x_t\}_{t=1}^T$ and the GEIIM output $H$, computes dynamic gates $d_t$ and weights $w_t$ for each instance, aggregates the sequence through a gated LSTM, refines the hidden states with multi-head self-attention, and outputs a single aggregated representation
$$
X' \in \mathbb{R}^{B\times C}
$$
for classification and MCCL [2509.04344].

The input-output interface is therefore explicit. The input instances satisfy $x_t\in\mathbb{R}^d$ with $d=C$ in practice, the GEIIM context input is $H\in\mathbb{R}^{B\times T\times C}$, and the output is a single video representation $X'\in\mathbb{R}^{B\times C}$ [2509.04344]. Internally, WIAN produces hidden states $\{h_t\}_{t=1}^T$, then applies multi-head self-attention and pooling before projection back to classifier dimension $C$ [2509.04344].

## 3. Mathematical structure of weighted aggregation

The upstream GEIIM context used by WIAN is defined through adaptive graph construction. GEIIM learns node embeddings $N_1\in\mathbb{R}^{T\times d_a}$ and $N_2\in\mathbb{R}^{d_a\times T}$, constructs
$$
\mathbf{A} = \mathrm{Softmax}\big(\mathrm{ReLU}(\mathbf{N}_1 \cdot \mathbf{N}_2)\big),
$$
and propagates features by
$$
\mathbf{H} = \alpha \mathbf{X} + (1-\alpha)\,\mathbf{A}\cdot\mathbf{X},
$$
with learnable $\alpha\in[0,1]$ [2509.04344]. In this role, $H$ supplies graph-enhanced, noise-suppressed instance features to WIAN.

WIAN derives instance importance from the GEIIM output rather than directly from raw $x_t$. For each time step $t$,
$$
w_t = \mathrm{Softmax}\big(\mathbf{H}_t,\ \mathrm{dim}=-1\big),
$$
yielding a weight vector aligned with the feature dimension [2509.04344]. In practice, this vector is mapped through learned linear projection or broadcasting to match the dimensionality of the LSTM gate, retaining per-channel saliency while enabling per-step gating [2509.04344].

The central mechanism is the Dynamic Weight Gate. WIAN augments a standard LSTM with a gate that scales the candidate cell update by instance importance:
$$
\mathbf{d}_t = \sigma\big(\mathbf{W}_s \mathbf{x}_t + \mathbf{U}_s \mathbf{h}_{t-1}\big)\ \cdot\ w_t,
$$
where $\sigma$ is Sigmoid, $\mathbf{W}_s \in \mathbb{R}^{m\times d}$, $\mathbf{U}_s \in \mathbb{R}^{m\times m}$, and $\mathbf{h}_{t-1}\in\mathbb{R}^m$ [2509.04344]. The standard LSTM gates and candidate state are
$$
\begin{aligned}
\mathbf{f}_t &= \sigma(\mathbf{W}_f \mathbf{x}_t + \mathbf{U}_f \mathbf{h}_{t-1} + \mathbf{b}_f),\\
\mathbf{i}_t &= \sigma(\mathbf{W}_i \mathbf{x}_t + \mathbf{U}_i \mathbf{h}_{t-1} + \mathbf{b}_i),\\
\mathbf{o}_t &= \sigma(\mathbf{W}_o \mathbf{x}_t + \mathbf{U}_o \mathbf{h}_{t-1} + \mathbf{b}_o),\\
\tilde{\mathbf{c}}_t &= \tanh(\mathbf{W}_c \mathbf{x}_t + \mathbf{U}_c \mathbf{h}_{t-1} + \mathbf{b}_c).
\end{aligned}
$$

WIAN inserts $\mathbf{d}_t$ into the cell update:
$$
\mathbf{c}_t = \mathbf{f}_t \odot \mathbf{c}_{t-1} + \mathbf{i}_t \odot \big(\tilde{\mathbf{c}}_t \odot \mathbf{d}_t\big),
$$
$$
\mathbf{h}_t = \mathbf{o}_t \odot \tanh(\mathbf{c}_t).
$$
This makes an instance’s contribution dependent on both temporal context, through $\mathbf{h}_{t-1}$, and graph-enhanced saliency, through $w_t$ [2509.04344].

The recurrent sequence is subsequently refined by multi-head self-attention:
$$
\mathbf{Z} = \mathrm{MHA}\big([\mathbf{h}_1,\dots,\mathbf{h}_T]\big)\in \mathbb{R}^{T\times m},
$$
and then aggregated by pooling and projected to classifier dimension:
$$
\mathbf{X}' = \mathrm{Pool}(\mathbf{Z})\in \mathbb{R}^{m}, \qquad
\mathbf{X}' \leftarrow \mathrm{Proj}(\mathbf{X}')\in\mathbb{R}^{C}.
$$
The weighted aggregation core is therefore not a simple weighted sum at sequence output; it is embedded inside the recurrent state transition itself [2509.04344].

## 4. Training objective, regularization, and efficiency

WIAN is trained jointly with the rest of MICACL through cross-entropy classification and Multiscale Category-aware Contrastive Learning (MCCL) [2509.04344]. The cross-entropy term is applied to logits derived from $X'$:
$$
\mathcal{L}_{\mathrm{CE}} = -\frac{1}{B}\sum_{b=1}^{B}\sum_{c=1}^{C} y_{b,c}\,\log p_{b,c}(\mathbf{X}'_b),
$$
while MCCL receives $X'$ and constructs multiscale projections, category-aware weights, and scale-specific similarities [2509.04344]. The full training objective is
$$
\mathcal{L} = \mathcal{L}_{\mathrm{CE}} + \lambda\,\mathcal{L}_{\mathrm{all}},
$$
with $\lambda$ controlling the balance [2509.04344]. The role division is explicit in the source: MCCL emphasizes tail categories via $1/n_c$ weighting and hard samples via $(1-\mathrm{softmax}(y_p))$, complementing WIAN’s instance-selective aggregation [2509.04344].

No explicit entropy or sparsity penalty on $w_t$ is reported [2509.04344]. Robustness is instead attributed to GEIIM normalization, specifically the softmax normalization of the adaptive adjacency matrix, and to the DWG’s sigmoid gating [2509.04344]. This is significant because it distinguishes WIAN from weighting schemes that enforce sparsity directly through additional regularizers.

The implementation details pertinent to WIAN are also specified. The backbone is R3D with Torchvision pretrained weights; the model uses $T=16$ frames per video; faces are resized to $112\times112$; optimization uses AdamW with cosine decay, initial learning rate $4\mathrm{e}{-4}$, minimum learning rate $3\mathrm{e}{-6}$, weight decay $0.05$, and $300$ epochs on an RTX A6000 GPU [2509.04344]. Batch size, gate hidden size $m$, and $\lambda$ are tuned empirically and are not explicitly reported [2509.04344].

From a computational perspective, WIAN scales linearly in the sequence length $T$ [2509.04344]. Its per-step cost is $O(C)$ for computing $w_t$ through softmax over feature channels and $O(md + m^2)$ for the DWG mapping and LSTM gates, giving per-video complexity
$$
O\big(T(md + m^2 + C)\big)
$$
[2509.04344]. GEIIM adds $O(Td_aT + TC)$ for adjacency construction and propagation, and multi-head self-attention adds a heads$\times m^2$ parameter overhead [2509.04344]. The MICACL system reports $1.69\mathrm{G}$ FLOPs on DFEW, which the source presents as efficient relative to transformer-heavy baselines [2509.04344].

## 5. Empirical behavior and ablation evidence

On DFEW under 5-fold evaluation, MICACL achieves WAR $69.56\%$ and UAR $63.21\%$, with minority-class results including disgust $29.65\%$ and fear $42.57\%$, described as far above prior $0$–$34\%$ ranges [2509.04344]. On FERV39k, MICACL reaches WAR $48.57\%$ and UAR $40.25\%$, surpassing prior state of the art in UAR [2509.04344].

Ablation evidence on DFEW Fold 5 isolates the contribution of WIAN relative to GEIIM and MCCL. The source reports the following configurations [2509.04344]:

| Configuration | WAR | UAR |
|---|---:|---:|
| Baseline (no GEIIM/WIAN/MCCL) | 64.55 | 54.17 |
| +GEIIM | 68.11 | 57.39 |
| +GEIIM+WIAN | 68.62 | 56.26 |
| +GEIIM+WIAN+MCCL | 69.91 | 64.34 |

These results support two distinct observations. First, WIAN improves overall accuracy relative to GEIIM alone, as seen in the WAR increase from $68.11$ to $68.62$ [2509.04344]. Second, WIAN alone does not maximize balanced recall under long-tailed conditions; UAR decreases from $57.39$ to $56.26$ when MCCL is absent, and then rises to $64.34$ when MCCL is added [2509.04344]. A common misconception is therefore that selective instance weighting by itself solves category imbalance. The ablation instead indicates that WIAN’s selective aggregation and MCCL’s category-aware reweighting are complementary rather than interchangeable [2509.04344].

The reported failure cases are consistent with this interpretation. In extremely noisy or near-neutral sequences, if GEIIM fails to construct informative adjacency, then $w_t$ may underdiscriminate and the DWG may suppress genuine subtle cues [2509.04344]. For very short sequences, limited $T$ reduces the benefit of graph-enhanced context and gating, and mean pooling may be competitive when dynamics are minimal [2509.04344]. Performance is also sensitive to $\alpha$, gate hidden size $m$, and the projection aligning $w_t$ to gate dimensionality [2509.04344].

## 6. Relation to other weighted-instance aggregation paradigms

Within the supplied literature, WIAN has both a narrow and a broader comparative meaning. Narrowly, it denotes the specific MICACL module that derives $w_t$ from GEIIM-enhanced features, injects those weights into an LSTM cell through DWG, and refines the resulting sequence with multi-head self-attention [2509.04344]. More broadly, related work contains architectures that aggregate instances by learned weights, even when they do not use the name WIAN. This suggests a family resemblance across exemplar-weighted classification, query-conditioned support aggregation, and context-aware temporal MIL.

"Deep Weighted Averaging Classifiers" replaces a final softmax with instance-weighted aggregation over training examples in a learned embedding space [1811.02579]. Its embedding is
$$
h = W f_\theta(x) + b,
$$
its kernel weight is
$$
w(h, h_i) = \exp\!\left(- \|h-h_i\|_2^2 / (2\sigma)\right),
$$
with fixed $\sigma = 1/2$, and its class prediction is a normalized weighted sum of one-hot training labels [1811.02579]. In the comparative description provided for DWAC, this model is described as a prototypical WIAN in the sense that it embeds inputs with a deep network, computes kernel-based weights to training instances, and aggregates labels with normalized weights [1811.02579]. The difference from MICACL’s WIAN is structural: DWAC aggregates over stored training instances at inference, whereas MICACL’s WIAN aggregates temporally ordered instances within a video and places the weighting mechanism inside recurrent state updates [1811.02579].

"Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification" provides a second comparison point [1906.06678]. There, support instances are aggregated by query-conditioned attention:
$$
\beta_k = \mathbf{v}^{\top}\operatorname{ReLU}\left(\mathbf{W}_2[\widehat{\mathbf{s}}_k;\widehat{\mathbf{q}}]\right),\qquad
\alpha_k = \frac{\exp(\beta_k)}{\sum_{k'=1}^{K}\exp(\beta_{k'})},\qquad
\widehat{\mathbf{s}} = \sum_{k=1}^{K}\alpha_k \widehat{\mathbf{s}}_k.
$$
The supplied comparative discussion explicitly characterizes this attentive aggregation as a WIAN because it aggregates support instances through learned, query-conditioned weights [1906.06678]. In contrast to MICACL, the weighting signal is conditioned on a query representation rather than on graph-enhanced temporal context, and the aggregate is a class prototype rather than a video descriptor [1906.06678].

These comparisons clarify what distinguishes MICACL’s WIAN from simpler alternatives. Versus mean or max pooling, WIAN learns per-instance weights conditioned on temporal context; versus standard attention-based MIL such as the Ilse et al. style described in the source, WIAN derives $w_t$ from GEIIM-enhanced features $H_t$ and injects them inside an LSTM cell via DWG rather than forming only a single weighted sum [2509.04344]. A plausible implication is that the term WIAN is most precise when reserved for architectures in which instance weighting is part of the aggregation dynamics rather than only an output-stage reweighting.

## 7. Conceptual significance and scope of applicability

In MICACL, WIAN is presented as the aggregation engine that turns instance features into a robust video-level representation via context-aware weighted gating [2509.04344]. Its defining computations are the importance scores
$$
w_t = \mathrm{Softmax}(H_t,\mathrm{dim}=-1),
$$
the dynamic gate
$$
d_t = \sigma(W_s x_t + U_s h_{t-1}) \cdot w_t,
$$
and the gated cell update
$$
c_t = f_t \cdot c_{t-1} + i_t \cdot (\tilde c_t \cdot d_t), \qquad
h_t = o_t \cdot \tanh(c_t)
$$
[2509.04344]. In this sense, the essential contribution is not merely weighted aggregation, but weighted recurrent aggregation with graph-aware context.

Its significance in long-tailed DFER follows directly from the design constraints stated in the source. Minority categories are weakly represented and often subtle; in-the-wild sequences contain many neutral, occluded, or off-angle frames; and naive pooling induces decision boundaries biased toward head classes [2509.04344]. WIAN is therefore aimed at suppressing noisy instances and boosting salient ones before the final classification and contrastive objectives operate [2509.04344].

At the same time, the ablation and limitation analysis bounds the scope of the claim. WIAN alone does not rebalance categories, does not guarantee improved UAR in isolation, and can underperform when graph context is uninformative or sequence length is too short [2509.04344]. Its strongest reported behavior emerges when it is coupled with GEIIM’s adjacency-aware context and MCCL’s category-aware contrastive optimization, producing improved overall accuracy and balanced recall on DFEW and FERV39k [2509.04344].

Source: https://www.emergentmind.com/topics/weighted-instance-aggregation-network-wian