HFUT-VUT: Collaborative Micro-Gesture Research

• HFUT-VUT is a collaborative research identity at HFUT that spans benchmark construction, challenge competitions, and open-source dataset releases in micro-gesture recognition.
• The research trajectory evolved from skeleton-based methods to advanced multimodal fusion architectures, achieving notable improvements in Top-1 accuracy and F1 scores.
• Its contributions include the development of competitive challenge systems and standardized datasets that enhance reproducibility and innovation in human behavior analysis.

Searching arXiv for HFUT-VUT-related papers to ground the article in current sources. HFUT-VUT is a recurrent team and collaboration identifier centered on Hefei University of Technology (HFUT) that appears across arXiv papers on micro-gesture classification, online micro-gesture recognition, micro-action datasets, video motion magnification, and multimodal behavior analysis. In the MiGA challenge series it denotes a competition team; in the VUT-HFUT GitHub organization it also functions as a collaboration identity for releasing datasets and code. Across these contexts, HFUT-VUT is not the name of a single model but of a research line spanning benchmark construction, challenge systems, and task-specific architectures (Li et al., 2023, Li et al., 2024, Wang et al., 2023).

1. Institutional identity and naming

HFUT-VUT was introduced in the MiGA 2023 micro-gesture classification paper as the team from the School of Computer Science and Information Engineering / School of Artificial Intelligence, Hefei University of Technology, the Key Laboratory of Knowledge Engineering with Big Data (HFUT), Ministry of Education, and the Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China (Li et al., 2023). Subsequent challenge papers retain the same team identity while expanding the author list to include Zhejiang University and, in the 2024 online-recognition work, Anhui Zhonghuitong Technology Co., Ltd. (Liu et al., 2024, Chen et al., 2024).

In the dataset and repository literature, the hyphenated form also marks a collaboration identity rather than a method name. The MMAD paper states that “HFUT–VUT” appears in the project URL https://github.com/VUT-HFUT/Micro-Action and denotes a joint micro-action research effort, with HFUT as the authors’ institution and “VUT” as a partner institution in the VUT-HFUT organization used to host the Micro-Action project (Li et al., 2024). The same VUT-HFUT organization hosts the code release for EulerMormer, indicating continuity between the challenge-facing HFUT-VUT label and the repository-facing VUT-HFUT label (Wang et al., 2023).

This pattern suggests an umbrella designation for a sustained HFUT-centered research program in human behavior analysis, particularly micro-actions and micro-gestures, rather than a one-off challenge alias.

2. Competition trajectory and reported results

A defining feature of HFUT-VUT is repeated participation in challenge benchmarks, especially MiGA and MultiMediate. The published record shows a sequence of systems with increasing task coverage and improving leaderboard performance across both classification and temporal recognition settings (Li et al., 2023, Chen et al., 2024, Liu et al., 2024, Gu et al., 11 Jul 2025, Liu et al., 13 Jul 2025, Li et al., 2023).

Venue and track	System or focus	Reported outcome
IJCAI 2023 MiGA, Micro-gesture Classification	Joint skeletal and semantic embedding loss	1st place, Top-1 accuracy 64.12%, 1.10% above second place
IJCAI 2024 MiGA, Micro-gesture Classification	Cross-modal fusion and prototype learning	1st place, Top-1 accuracy 70.254%, 6.13 percentage points above MiGA 2023 winner
IJCAI 2024 MiGA, Micro-gesture Online Recognition	Learnable query points with Mamba-MHSA	2nd place, F1 score 14.34
IJCAI 2025 MiGA, Micro-gesture Classification	MM-Gesture multimodal fusion	1st place, Top-1 accuracy 73.213%
IJCAI 2025 MiGA, Micro-gesture Online Recognition	Data augmentation and spatial-temporal attention	1st place, F1 score 38.03, 37.9% above previous state of the art
ACM MM 2023 MultiMediate	Bodily behavior, eye contact, next speaker	mAP 0.6262, accuracy 0.7771, UAR 0.5281

Within MiGA alone, the sequence from 64.12% Top-1 in 2023 to 70.254% in 2024 and 73.213% in 2025 for classification indicates a steady upward trajectory in the team’s reported micro-gesture recognition performance (Li et al., 2023, Chen et al., 2024, Gu et al., 11 Jul 2025). For online recognition, the jump from F1 14.34 in 2024 to 38.03 in 2025 reflects a similarly large change in reported effectiveness on the SMG benchmark (Liu et al., 2024, Liu et al., 13 Jul 2025).

3. Micro-gesture classification systems

HFUT-VUT’s classification work shows a clear methodological progression from skeleton-centered recognition to multi-branch multimodal fusion.

The MiGA 2023 solution, “Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification,” addresses skeleton-based classification from 2D pose data. A clip is represented as $X \in \mathbb{R}^{T \times K \times C}$, with $K=22$ upper-body keypoints for iMiGUE and $K=25$ whole-body keypoints for SMG, and is converted into a 3D heatmap volume $\mathbb{R}^{C' \times T \times H \times W}$ following the PoseC3D setup. The backbone is SlowOnly, the pooled clip feature is a 512-dimensional skeletal embedding $Z$, and the core loss is

$$\mathcal{L} = \mathcal{L}_{class} + \alpha \cdot \mathcal{L}_{emb},$$

with $\mathcal{L}_{emb} = \| Z_{emb} - E_{emb} \|^2$, where $E_{emb} \in \mathbb{R}^{300}$ is a GloVe embedding of the class label. The paper reports that $\alpha=20$ gives the best Top-1 result, and that a joint–limb score ensemble with ratio $2:3$ yields the final 64.12% Top-1 and 91.10% Top-5 on iMiGUE (Li et al., 2023).

The MiGA 2024 classification system, “Prototype Learning for Micro-gesture Classification,” adds RGB and explicitly tackles ambiguous samples. It uses a two-pathway PoseConv3D backbone with an RGB branch and a pose branch, clip lengths $K=22$0 and $K=22$1, a channel-wise cross-modal fusion block, and a prototypical refinement module. The prototype mechanism computes TP, FN, and FP sets per class inside the mini-batch, updates class prototypes with EMA using $K=22$2, and optimizes

$K=22$3

At inference, the network-level prediction is $K=22$4 RGB softmax $K=22$5 Pose softmax; the final challenge submission additionally ensembles Video Swin Transformer on RGB, the MiGA 2023 skeleton method on $K=22$6, and the team’s own PoseConv3D-based system, reaching 70.254% Top-1 on iMiGUE (Chen et al., 2024).

The MiGA 2025 system, “MM-Gesture: Towards Precise Micro-Gesture Recognition through Multimodal Fusion,” further extends the design to six modalities: joint skeleton, limb skeleton, RGB video, Taylor-series video, optical-flow video, and depth video. Joint and limb modalities are derived from 36 selected keypoints out of 137 OpenPose points; flow uses MemFlow, and depth uses Video Depth Anything. The framework combines a PoseConv3D cross-modal fusion module, Video Swin Transformer uni-modal encoders, transfer learning from the MA-52 dataset for RGB, and a probability-level weighted ensemble

$K=22$7

The paper reports 73.213% Top-1 on iMiGUE, with ablations showing successive gains from $K=22$8, then Taylor, flow, and depth, before the full six-modality ensemble (Gu et al., 11 Jul 2025).

Taken together, these systems indicate a transition from a skeleton-dominant 3D CNN with semantic alignment to multimodal architectures combining pose, appearance, motion, and geometry. This suggests that HFUT-VUT treated micro-gesture recognition increasingly as a heterogeneous-signal fusion problem rather than as pose classification alone.

4. Online recognition and temporal detection

HFUT-VUT’s online-recognition work targets continuous-video settings in which classification and temporal localization must be solved jointly.

The MiGA 2024 paper “Micro-gesture Online Recognition using Learnable Query Points” formulates online recognition as set prediction over a continuous RGB stream:

$K=22$9

The backbone is I3D initialized from Kinetics-400, operating on frames downsampled to 10 fps. The action decoder uses $K=25$0 learnable queries, each with $K=25$1 learnable query points, and refines them layer by layer via

$K=25$2

where $K=25$3. The decoder combines a Mamba-MHSA block with a Multi-Level Interactive Module, uses a sliding window of $K=25$4 frames, overlap ratio 0.75 during training and 0 during inference, and achieves F1 14.34 on SMG, ranking second (Liu et al., 2024).

The MiGA 2025 paper “Online Micro-gesture Recognition Using Data Augmentation and Spatial-Temporal Attention” replaces the query-based detector with a DyFADet-based architecture strengthened by a multi-scale spatial-temporal attention head and a category-frequency-based label augmentation scheme. Input RGB is processed at 28 fps by VideoMAEv2-g with a 16-frame sliding window and stride 4. DynE constructs a temporal pyramid, and the head applies temporal attention

$K=25$5

and spatial attention

$K=25$6

For rare classes with $K=25$7, the annotation replication factor is

$K=25$8

The reported F1 is 38.03, ranking first and exceeding the previous state of the art by 37.9%; ablations show 27.78 for the DyFADet baseline and 33.44 when both augmentation and spatial-temporal attention are enabled (Liu et al., 13 Jul 2025).

HFUT-VUT’s temporal-detection agenda extends beyond challenge submissions. The MMAD paper defines Multi-label Micro-Action Detection as set prediction over overlapping temporal intervals,

$K=25$9

with predicted set $\mathbb{R}^{C' \times T \times H \times W}$0, and evaluates PointTAD and MS-TCT as baselines. This establishes a broader detection setting in which micro-actions may repeat and overlap in time, rather than appearing as single trimmed events (Li et al., 2024).

5. Datasets, benchmarks, and open resources

HFUT-VUT’s strongest non-challenge contribution is the construction of public micro-action benchmarks.

“Benchmarking Micro-action Recognition: Dataset, Methods, and Applications” introduces MA-52, a whole-body micro-action dataset collected from psychological interviews. MA-52 contains 205 participants, 22,422 video instances, 52 micro-action categories, seven coarse-grained body-part labels, 1920×1080 resolution, 30 fps, and clips of 1–7 seconds with mean duration about 1.9 seconds and total time about 12.29 hours. The instance-level split is 2:1:1, giving train 11,250, validation 5,586, and test 5,586. The benchmark model MANet inserts squeeze-and-excitation and temporal shift modules into ResNet and adds a joint-embedding loss with GloVe label embeddings; with $\mathbb{R}^{C' \times T \times H \times W}$1, the paper reports the best $\mathbb{R}^{C' \times T \times H \times W}$2, ahead of the compared baselines including TSM, I3D, SlowFast, Video Swin Transformer, TimeSformer, UniFormer, and others (Guo et al., 2024).

The MMAD paper extends this line by introducing MMA-52 for temporal detection. MMA-52 contains 6,528 videos, 203 subjects, 52 action categories, 19,782 action instances, total duration 18.67 hours, average video length 10.30 seconds, average instance length 4.07 seconds, and average 3.1 instances per video. The cross-subject split is 4,534 training videos, 1,475 validation videos, and 519 test videos. Interval-level labels support overlapping instances, and the initial Detection-mAP baselines are low: MS-TCT averages 3.51 and PointTAD 4.51 across tIoU thresholds $\mathbb{R}^{C' \times T \times H \times W}$3 to $\mathbb{R}^{C' \times T \times H \times W}$4, underscoring the difficulty of the task (Li et al., 2024).

These datasets are tied together by the VUT-HFUT/Micro-Action repository, which the papers identify as the practical entry point for dataset access and code release (Li et al., 2024, Guo et al., 2024). MA-52 also feeds back into later challenge systems: MM-Gesture pretrains its RGB Video Swin Transformer on MA-52 before fine-tuning on iMiGUE, raising the RGB branch from 65.629% to 66.615% Top-1 in the reported ablation (Gu et al., 11 Jul 2025).

A plausible implication is that HFUT-VUT’s challenge results and benchmark releases are mutually reinforcing: the datasets provide pretraining and evaluation substrates, while the challenge systems test architectural ideas under competitive conditions.

6. Broader research scope, ambiguity of the label, and significance

Although micro-action and micro-gesture analysis dominates the HFUT-VUT record, the label also appears in adjacent vision tasks. In “EulerMormer: Robust Eulerian Motion Magnification via Dynamic Filtering within Transformer,” the VUT-HFUT organization releases a first Transformer-based learning framework for video motion magnification. EulerMormer separates texture and shape, defines motion through inter-frame shape differences, and uses a dynamic filter composed of DMF and MGR. The paper reports training on a synthetic dataset of 100,000 pairs and improvements over prior methods on Synthetic-I, Synthetic-II, Synthetic-III, and real-world MANIQA evaluations, positioning the project as part of the same collaboration ecosystem (Wang et al., 2023).

In “Data Augmentation for Human Behavior Analysis in Multi-Person Conversations,” HFUT-VUT appears as the team in the MultiMediate Grand Challenge 2023. There the group uses Swin Transformer backbones with central video cropping, OpenFace face crops, and augmentation strategies such as ColorJitter, RandomErasing, Lighting, and RandAugment. The reported outcomes are mAP $\mathbb{R}^{C' \times T \times H \times W}$5 for bodily behavior recognition, accuracy $\mathbb{R}^{C' \times T \times H \times W}$6 for eye contact detection, and UAR $\mathbb{R}^{C' \times T \times H \times W}$7 for next speaker prediction (Li et al., 2023).

A common misconception is to read any occurrence of “VUT” on arXiv as referring to the HFUT-VUT team. That is not generally correct. In automated-driving evaluation, “VUT” can denote “Vehicle Under Test”; in “Real-world Troublemaker,” for example, the VUT is the central decision-making agent on a physical test track within a 5G cloud-controlled testing framework, and the acronym is unrelated to the HFUT-VUT team identity (Zhang et al., 20 Feb 2025).

Across the cited literature, HFUT-VUT is best understood as a stable HFUT-centered research identifier spanning competition systems, benchmark datasets, and open-source releases. Its published work is characterized by repeated attention to subtle human movement, multimodal representation, semantic or prototype-based supervision, and evaluation under cross-subject or challenge-benchmark protocols.