Exp-Graph: Facial Expression Recognition

• Exp-Graph is a graph-based modeling framework for facial expression recognition that uses facial landmarks as dynamic graph vertices.
• It combines Vision Transformer patch encoding with graph convolutional networks to integrate local appearance and geometric features.
• Careful hyperparameter tuning and dynamic edge construction yield superior performance on datasets like Oulu-CASIA, eNTERFACE05, and AFEW.

Searching arXiv for the topic and related papers.

Exp-Graph is a graph-based modeling framework for facial expression recognition in which facial landmarks serve as graph vertices, edge formation combines landmark proximity with local appearance similarity encoded by a vision transformer, and graph convolutional networks integrate these structural dependencies into the facial-attribute representation. The framework is designed to exploit both local and global dependencies among facial attributes that are essential for the recognition of facial expressions, and it was evaluated on Oulu-CASIA, eNTERFACE05, and AFEW, where it achieved recognition accuracies of $98.09\%$, $79.01\%$, and $56.39\%$, respectively (Sharma et al., 19 Jul 2025).

1. Facial-attribute graph construction

The core object in Exp-Graph is a facial-attribute graph $G=(V,E)$ derived from a detected and aligned face image $I$. A fixed set of facial landmarks $P=\{p_1,\dots,p_n\}$, with $p_i\in\mathbb{R}^2$, becomes the vertex set. Around each landmark $p_i$, a local patch $I_i$ of size $H\times W$ is cropped, resized, and encoded by a pre-trained Vision Transformer into a $79.01\%$0-dimensional feature vector $79.01\%$1. The feature vectors are $79.01\%$2-normalized, and pairwise appearance affinity is computed by

$79.01\%$3

Geometry is incorporated through the Euclidean landmark distance

$79.01\%$4

Exp-Graph combines appearance and geometry in an un-thresholded adjacency score

$79.01\%$5

To retain only the most significant connections, the global mean $79.01\%$6 and standard deviation $79.01\%$7 of $79.01\%$8 are computed, and a threshold is set as

$79.01\%$9

The binary adjacency matrix is then defined by

$56.39\%$0

The resulting graph therefore encodes edges $56.39\%$1 through a joint criterion: local appearance resemblance and geometric proximity (Sharma et al., 19 Jul 2025).

This construction yields a dynamic graph rather than a fixed anatomical template. The reported interpretation is that Eqns. (1)–(2) fuse local appearance and geometric proximity to capture the most salient facial-attribute relationships. A plausible implication is that the graph topology varies with both expression-dependent morphology and local visual evidence, rather than being prescribed a priori.

2. Vision-transformer embedding and graph-convolutional propagation

The patch encoder follows the standard ViT pipeline. Each patch $56.39\%$2 is tokenized into smaller sub-patches, each flattened and linearly projected to $56.39\%$3. A learnable class token $56.39\%$4 and fixed positional encodings $56.39\%$5 are prepended to form $56.39\%$6. Transformer blocks are then applied: $56.39\%$7

$56.39\%$8

for $56.39\%$9, and the final $G=(V,E)$0 row of $G=(V,E)$1 is used as the local descriptor $G=(V,E)$2.

Given the graph $G=(V,E)$3, Exp-Graph applies a GCN over the landmark graph. Self-loops are added and the adjacency is symmetrically normalized as

$G=(V,E)$4

With $G=(V,E)$5, propagation at layer $G=(V,E)$6 is

$G=(V,E)$7

After $G=(V,E)$8 layers, node outputs are pooled, for example by mean- or max-pooling over nodes, to obtain a graph-level embedding $G=(V,E)$9, which is then passed through an MLP and softmax for $I$0-way expression classification (Sharma et al., 19 Jul 2025).

The architectural division of labor is explicit. The Vision Transformer supplies rich local patch descriptors; the GCN integrates these descriptors according to the learned graph topology. The reported conclusion is that the vision transformer and graph convolutional blocks help the framework exploit the local and global dependencies among the facial attributes that are essential for the recognition of facial expressions.

3. Objective function, preprocessing, and evaluation protocol

Training uses the standard cross-entropy objective on predicted expression probabilities $I$1 and one-hot labels $I$2: $I$3 No additional structural or regularization terms were added beyond standard weight-decay on network parameters (Sharma et al., 19 Jul 2025).

The reported preprocessing pipeline consists of face detection and alignment with Dlib, resizing the full face to $I$4, extracting patches around each of 68 landmarks, and normalizing pixel intensities. Optimal patch size is dataset-dependent; examples given are $I$5 on Oulu-CASIA and $I$6 on eNTERFACE05. The implementation uses PyTorch 2.1.2, CUDA 12.8, NVIDIA RTX A6000, and random seed $I$7. Optimization uses Adam with initial learning rate $I$8, decayed to $I$9, weight decay $P=\{p_1,\dots,p_n\}$0, batch size typically $P=\{p_1,\dots,p_n\}$1–$P=\{p_1,\dots,p_n\}$2, dropout $P=\{p_1,\dots,p_n\}$3 in GCN layers, hidden units $P=\{p_1,\dots,p_n\}$4, and GeLU as the primary activation, although ReLU and ELU were also reported to give comparable results.

Threshold selection is explicitly treated as a model-selection variable. The threshold parameter $P=\{p_1,\dots,p_n\}$5 is grid-searched over $P=\{p_1,\dots,p_n\}$6, with best values reported as around $P=\{p_1,\dots,p_n\}$7 on Oulu-CASIA and $P=\{p_1,\dots,p_n\}$8 on eNTERFACE05 and AFEW. Patch size is also grid-searched over $P=\{p_1,\dots,p_n\}$9. Evaluation metrics are Accuracy (Acc), Weighted Average Recall (WAR), Unweighted Average Recall (UAR), and $p_i\in\mathbb{R}^2$0-score (Sharma et al., 19 Jul 2025).

These design choices establish Exp-Graph as a dynamically constructed landmark graph pipeline rather than a static GCN on predefined facial topology. A plausible implication is that much of the method’s behavior is controlled not only by backbone capacity but also by graph sparsification and receptive-field calibration through $p_i\in\mathbb{R}^2$1 and $p_i\in\mathbb{R}^2$2.

4. Benchmark performance and comparative results

Exp-Graph was evaluated on three benchmark datasets spanning controlled and unconstrained conditions.

Dataset	Exp-Graph accuracy	Best prior
Oulu-CASIA	98.09%	92.03% (AT-ViG)
eNTERFACE05	79.01%	54.35% (Vnet)
AFEW	56.39%	52.74% (DSAN-VGG)

On Oulu-CASIA, backbone and graph-operator ablations show large differences. For GCN-based variants, ResNet18+GCN achieved $p_i\in\mathbb{R}^2$3 Acc, $p_i\in\mathbb{R}^2$4 $p_i\in\mathbb{R}^2$5, $p_i\in\mathbb{R}^2$6 WAR, and $p_i\in\mathbb{R}^2$7 UAR; EfficientNet+GCN achieved $p_i\in\mathbb{R}^2$8 Acc, $p_i\in\mathbb{R}^2$9 $p_i$0, $p_i$1 WAR, and $p_i$2 UAR; ViT+GCN achieved $p_i$3 Acc, $p_i$4 $p_i$5, $p_i$6 WAR, and $p_i$7 UAR. For GAT-based variants, ResNet18+GAT achieved $p_i$8 Acc, $p_i$9 $I_i$0, $I_i$1 WAR, and $I_i$2 UAR; EfficientNet+GAT achieved $I_i$3 Acc, $I_i$4 $I_i$5, $I_i$6 WAR, and $I_i$7 UAR; ViT+GAT achieved $I_i$8 Acc, $I_i$9 $H\times W$0, $H\times W$1 WAR, and $H\times W$2 UAR. The reported conclusion is that ViT+GCN outperforms the alternatives by a large margin (Sharma et al., 19 Jul 2025).

The same section also reports that t-SNE visualizations confirm that well-tuned $H\times W$3 yield clearly separated clusters for each expression. This suggests that the graph-construction hyperparameters materially affect representation geometry, not merely end-task classification.

5. Ablation studies and operational sensitivities

The threshold ablation on Oulu-CASIA gives a sharply non-monotonic pattern: $H\times W$4 Acc, $H\times W$5, $H\times W$6, $H\times W$7, and $H\times W$8. The best result occurs at approximately $H\times W$9, which is described as a balance between connectivity and sparsity. Patch-size ablation on the same dataset gives $79.01\%$00, $79.01\%$01, $79.01\%$02, $79.01\%$03, and $79.01\%$04. On eNTERFACE05, the best patch size is $79.01\%$05 (Sharma et al., 19 Jul 2025).

The paper’s interpretation is explicit on both forms of tuning. Too low a threshold produces an overly dense graph, whereas too large a threshold fragments the graph; both harm performance. Likewise, patches that are too small miss contextual cues, while overly large patches conflate distinct landmarks. These observations place graph sparsification and patch sampling at the center of the method rather than treating them as secondary implementation details.

The broader explanatory account given in the study attributes Exp-Graph’s performance in both laboratory and in-the-wild settings to three factors: synergy of global-local encoding in the ViT and relational reasoning in the GCN, dynamic edge learning rather than fixed predefined graphs, and robustness to illumination, pose, and occlusion through the geometry-aware graph and attention in ViT. Since these are interpretive claims from the study rather than formal guarantees, they should be understood as the authors’ account of the empirical behavior.

6. Position within adjacent graph-representation literature

The name “Exp-Graph” is potentially ambiguous because closely related labels designate distinct graph-learning frameworks in other subfields. “Towards Expressive Graph Representation” introduces ExpGNN, a graph neural network framework built around continuous injective set functions for neighborhood aggregation, with the goal of matching the discriminative power of the $79.01\%$06-WL test; despite the similarity in naming, that work addresses expressive graph representation learning in general GNNs rather than facial expression recognition (Mao et al., 2020). “ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents” uses an experience graph to organize reusable skills and failure lessons for frozen and replaceable LLM executors, which is again unrelated in task domain and graph semantics (Feng et al., 29 May 2026). A further nearby term is “Expander Graph Propagation,” which superimposes expander templates on input graphs to mitigate bottlenecks and oversquashing in whole-graph learning (Deac et al., 2022).

Within this naming landscape, Exp-Graph denotes a specific facial-attribute graph framework in which graph vertices are facial landmarks, node features are ViT-encoded landmark patches, and graph edges are induced by joint appearance-geometry criteria. A common misconception would be to read the name as referring to “expressive graphs” or “expander graphs”; the literature indicates that these are separate research lines with distinct formal objects, objectives, and evaluation regimes.