---
title: 'Exp-Graph: Facial Expression Recognition'
url: https://www.emergentmind.com/topics/exp-graph
type: topic
---

# Exp-Graph: Facial Expression Recognition

Searching arXiv for the topic and related papers.

Exp-Graph is a graph-based modeling framework for facial expression recognition in which facial landmarks serve as graph vertices, edge formation combines landmark proximity with local appearance similarity encoded by a vision transformer, and graph convolutional networks integrate these structural dependencies into the facial-attribute representation. The framework is designed to exploit both local and global dependencies among facial attributes that are essential for the recognition of facial expressions, and it was evaluated on Oulu-CASIA, eNTERFACE05, and AFEW, where it achieved recognition accuracies of \(98.09\%\), \(79.01\%\), and \(56.39\%\), respectively [2507.14608].

## 1. Facial-attribute graph construction

The core object in Exp-Graph is a facial-attribute graph \(G=(V,E)\) derived from a detected and aligned face image \(I\). A fixed set of facial landmarks \(P=\{p_1,\dots,p_n\}\), with \(p_i\in\mathbb{R}^2\), becomes the vertex set. Around each landmark \(p_i\), a local patch \(I_i\) of size \(H\times W\) is cropped, resized, and encoded by a pre-trained Vision Transformer into a \(d\)-dimensional feature vector \(x_i\in\mathbb{R}^d\). The feature vectors are \(L_2\)-normalized, and pairwise appearance affinity is computed by
\[
K(x_i,x_j)=\frac{x_i^\top x_j}{\|x_i\|\,\|x_j\|}.
\]
Geometry is incorporated through the Euclidean landmark distance
\[
d_{ij}=\|p_i-p_j\|_2.
\]

Exp-Graph combines appearance and geometry in an un-thresholded adjacency score
\[
\bar A_{ij}=K(x_i,x_j)\cdot e^{-d_{ij}}.
\]
To retain only the most significant connections, the global mean \(\mu_K\) and standard deviation \(\sigma_K\) of \(\{\bar A_{ij}\}\) are computed, and a threshold is set as
\[
T_s=\mu_K+\tau\cdot\sigma_K.
\]
The binary adjacency matrix is then defined by
\[
A_{ij}=
\begin{cases}
1, & \bar A_{ij}>T_s,\\
0, & \text{otherwise}.
\end{cases}
\]
The resulting graph therefore encodes edges \(E=\{(i,j)\mid A_{ij}=1\}\) through a joint criterion: local appearance resemblance and geometric proximity [2507.14608].

This construction yields a dynamic graph rather than a fixed anatomical template. The reported interpretation is that Eqns. (1)–(2) fuse local appearance and geometric proximity to capture the most salient facial-attribute relationships. A plausible implication is that the graph topology varies with both expression-dependent morphology and local visual evidence, rather than being prescribed a priori.

## 2. Vision-transformer embedding and graph-convolutional propagation

The patch encoder follows the standard ViT pipeline. Each patch \(I_i\) is tokenized into smaller sub-patches, each flattened and linearly projected to \(\mathbb{R}^d\). A learnable class token \([CLS]\) and fixed positional encodings \(E_{\text{pos}}\in\mathbb{R}^{(m+1)\times d}\) are prepended to form \(Z^0\). Transformer blocks are then applied:
\[
Z^l=\mathrm{MSA}(\mathrm{LN}(Z^{l-1})) + Z^{l-1},
\]
\[
Z^l=\mathrm{MLP}(\mathrm{LN}(Z^l)) + Z^l,
\]
for \(l=1,\dots,L\), and the final \([CLS]\) row of \(Z^L\) is used as the local descriptor \(x_i\in\mathbb{R}^d\).

Given the graph \(G=(P,X,A)\), Exp-Graph applies a GCN over the landmark graph. Self-loops are added and the adjacency is symmetrically normalized as
\[
\hat A=\tilde D^{-1/2}(A+I)\tilde D^{-1/2},
\qquad
\tilde D_{ii}=\sum_j (A_{ij}+\delta_{ij}).
\]
With \(H^0=X\), propagation at layer \(l\) is
\[
H^{(l+1)}=\sigma(\hat A H^{(l)}W^{(l)}),
\qquad
W^{(l)}\in\mathbb{R}^{d_l\times d_{l+1}}.
\]
After \(L\) layers, node outputs are pooled, for example by mean- or max-pooling over nodes, to obtain a graph-level embedding \(h\in\mathbb{R}^{d_L}\), which is then passed through an MLP and softmax for \(C\)-way expression classification [2507.14608].

The architectural division of labor is explicit. The Vision Transformer supplies rich local patch descriptors; the GCN integrates these descriptors according to the learned graph topology. The reported conclusion is that the vision transformer and graph convolutional blocks help the framework exploit the local and global dependencies among the facial attributes that are essential for the recognition of facial expressions.

## 3. Objective function, preprocessing, and evaluation protocol

Training uses the standard cross-entropy objective on predicted expression probabilities \(\hat y_j\in[0,1]^C\) and one-hot labels \(y_j\):
\[
\mathcal L_{\mathrm{CE}}=
-\frac{1}{N}\sum_{j=1}^N\sum_{i=1}^C y_{j,i}\log \hat y_{j,i}.
\]
No additional structural or regularization terms were added beyond standard weight-decay on network parameters [2507.14608].

The reported preprocessing pipeline consists of face detection and alignment with Dlib, resizing the full face to \(224\times 224\), extracting patches around each of 68 landmarks, and normalizing pixel intensities. Optimal patch size is dataset-dependent; examples given are \(70\times 70\) on Oulu-CASIA and \(30\times 30\) on eNTERFACE05. The implementation uses PyTorch 2.1.2, CUDA 12.8, NVIDIA RTX A6000, and random seed \(1000\). Optimization uses Adam with initial learning rate \(10^{-3}\), decayed to \(10^{-4}\), weight decay \(5\times 10^{-4}\), batch size typically \(32\)–\(64\), dropout \(0.2\) in GCN layers, hidden units \(256\), and GeLU as the primary activation, although ReLU and ELU were also reported to give comparable results.

Threshold selection is explicitly treated as a model-selection variable. The threshold parameter \(\tau\) is grid-searched over \(\{0.2,0.25,\dots,0.9\}\), with best values reported as around \(0.5\) on Oulu-CASIA and \(0.3\) on eNTERFACE05 and AFEW. Patch size is also grid-searched over \(\{10,20,30,50,70,90\}^2\). Evaluation metrics are Accuracy (Acc), Weighted Average Recall (WAR), Unweighted Average Recall (UAR), and \(F_1\)-score [2507.14608].

These design choices establish Exp-Graph as a dynamically constructed landmark graph pipeline rather than a static GCN on predefined facial topology. A plausible implication is that much of the method’s behavior is controlled not only by backbone capacity but also by graph sparsification and receptive-field calibration through \(\tau\) and \(H\times W\).

## 4. Benchmark performance and comparative results

Exp-Graph was evaluated on three benchmark datasets spanning controlled and unconstrained conditions.

| Dataset | Exp-Graph accuracy | Best prior |
|---|---:|---:|
| Oulu-CASIA | 98.09% | 92.03% (AT-ViG) |
| eNTERFACE05 | 79.01% | 54.35% (Vnet) |
| AFEW | 56.39% | 52.74% (DSAN-VGG) |

On Oulu-CASIA, backbone and graph-operator ablations show large differences. For GCN-based variants, ResNet18+GCN achieved \(59.93\) Acc, \(34.00\) \(F_1\), \(60.00\) WAR, and \(87.06\) UAR; EfficientNet+GCN achieved \(67.03\) Acc, \(36.79\) \(F_1\), \(67.03\) WAR, and \(89.25\) UAR; ViT+GCN achieved \(91.09\) Acc, \(91.00\) \(F_1\), \(91.09\) WAR, and \(95.55\) UAR. For GAT-based variants, ResNet18+GAT achieved \(67.00\) Acc, \(26.77\) \(F_1\), \(67.89\) WAR, and \(92.77\) UAR; EfficientNet+GAT achieved \(50.30\) Acc, \(19.29\) \(F_1\), \(50.30\) WAR, and \(86.28\) UAR; ViT+GAT achieved \(66.18\) Acc, \(25.51\) \(F_1\), \(66.18\) WAR, and \(91.17\) UAR. The reported conclusion is that ViT+GCN outperforms the alternatives by a large margin [2507.14608].

The same section also reports that t-SNE visualizations confirm that well-tuned \((\tau, H\times W)\) yield clearly separated clusters for each expression. This suggests that the graph-construction hyperparameters materially affect representation geometry, not merely end-task classification.

## 5. Ablation studies and operational sensitivities

The threshold ablation on Oulu-CASIA gives a sharply non-monotonic pattern: \(\tau=0.2\rightarrow 85.4\%\) Acc, \(\tau=0.3\rightarrow 69.9\%\), \(\tau=0.5\rightarrow 91.1\%\), \(\tau=0.7\rightarrow 36.9\%\), and \(\tau=0.9\rightarrow 27.6\%\). The best result occurs at approximately \(\tau=0.5\), which is described as a balance between connectivity and sparsity. Patch-size ablation on the same dataset gives \(10\times 10\rightarrow 69.9\%\), \(20\times 20\rightarrow 63.3\%\), \(30\times 30\rightarrow 75.5\%\), \(50\times 50\rightarrow 95.6\%\), and \(70\times 70\rightarrow 98.1\%\). On eNTERFACE05, the best patch size is \(30\times 30\) [2507.14608].

The paper’s interpretation is explicit on both forms of tuning. Too low a threshold produces an overly dense graph, whereas too large a threshold fragments the graph; both harm performance. Likewise, patches that are too small miss contextual cues, while overly large patches conflate distinct landmarks. These observations place graph sparsification and patch sampling at the center of the method rather than treating them as secondary implementation details.

The broader explanatory account given in the study attributes Exp-Graph’s performance in both laboratory and in-the-wild settings to three factors: synergy of global-local encoding in the ViT and relational reasoning in the GCN, dynamic edge learning rather than fixed predefined graphs, and robustness to illumination, pose, and occlusion through the geometry-aware graph and attention in ViT. Since these are interpretive claims from the study rather than formal guarantees, they should be understood as the authors’ account of the empirical behavior.

## 6. Position within adjacent graph-representation literature

The name “Exp-Graph” is potentially ambiguous because closely related labels designate distinct graph-learning frameworks in other subfields. “Towards Expressive Graph Representation” introduces ExpGNN, a graph neural network framework built around continuous injective set functions for neighborhood aggregation, with the goal of matching the discriminative power of the \(1\)-WL test; despite the similarity in naming, that work addresses expressive graph representation learning in general GNNs rather than facial expression recognition [2010.05427]. “ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents” uses an experience graph to organize reusable skills and failure lessons for frozen and replaceable LLM executors, which is again unrelated in task domain and graph semantics [2605.30712]. A further nearby term is “Expander Graph Propagation,” which superimposes expander templates on input graphs to mitigate bottlenecks and oversquashing in whole-graph learning [2210.02997].

Within this naming landscape, Exp-Graph denotes a specific facial-attribute graph framework in which graph vertices are facial landmarks, node features are ViT-encoded landmark patches, and graph edges are induced by joint appearance-geometry criteria. A common misconception would be to read the name as referring to “expressive graphs” or “expander graphs”; the literature indicates that these are separate research lines with distinct formal objects, objectives, and evaluation regimes.

Source: https://www.emergentmind.com/topics/exp-graph