MA-Bench-Train: Micro-Action Corpus

• MA-Bench-Train is a training corpus designed for micro-action understanding, annotating 20,510 video clips with structured motion descriptions.
• It employs a semi-automatic pipeline that integrates optical-flow, skeleton data, LLM synthesis, and human verification for high-quality captions.
• Fine-tuning with Qwen3-VL-8B using LoRA strategies boosts performance in both closed-ended accuracy and open-ended reasoning tasks.

MA-Bench-Train is a large-scale training corpus introduced with MA-Bench to address the scarcity of training corpora tailored for micro-action understanding in Multimodal LLMs (MLLMs). It contains 20 510 videos annotated with structured micro-action captions for fine-tuning MLLMs, and is intended to provide high-quality, fine-grained supervision bridging optical-flow and skeleton-based motion descriptors with natural-language captions. The corpus is positioned around subtle body-part movements and is designed to facilitate both perceptual recognition and interpretive reasoning, complementing the 1 000-video MA-Bench evaluation set and its three-tier evaluation architecture (Li et al., 27 Mar 2026).

1. Position within the MA-Bench framework

MA-Bench-Train is the training component associated with MA-Bench, a benchmark for micro-action understanding in MLLMs. The benchmark itself comprises 1,000 videos and 12,000 structured question-answer pairs, and its evaluation architecture progressively examines micro-action perception, relational comprehension, and interpretive reasoning (Li et al., 27 Mar 2026). MA-Bench-Train was constructed to address the training-side gap identified by that benchmark: the absence of specialized supervision for motion granularity and fine-grained body-part dynamics.

The training corpus is explicitly separated from the evaluation benchmark. Its base videos are drawn from the Micro-Action-52 (MA-52) corpus, and the construction enforces a cross-subject split with no overlap with the 1 000-video MA-Bench evaluation set. This separation is methodologically important because the paper also recommends maintaining cross-subject splits to avoid overfitting to specific individuals. A plausible implication is that the corpus is intended not merely as additional data volume, but as a controlled resource for studying transfer from structured micro-action supervision to held-out human subjects.

2. Source data and construction pipeline

MA-Bench-Train consists of 20 510 clips, each 1–5 s long, sampled from 166 distinct participants in psychological interview settings, with one structured micro-action caption per video, totaling 20 510 captions (Li et al., 27 Mar 2026). The stated design goals are to provide high-quality, fine-grained supervision, bridge optical-flow and skeleton-based motion descriptors with natural-language captions, and support subtle body-part movement analysis in MLLMs.

Its construction uses a semi-automatic pipeline. First, a micro-motion tracker combining CoTracker3 and YOLOv8x-Seg extracts dense optical-flow vectors and 2D coordinates for seven body parts, including examples such as the left hand and head. Second, alignment with skeleton data yields per-body-part motion descriptors. Third, LLMs, including DeepSeek-v3.2 and GPT-4o, synthesize these descriptors into structured micro-action captions. Fourth, human annotators perform multi-round verification and correction to ensure factual and linguistic accuracy.

This pipeline establishes a layered supervision process in which low-level motion evidence is transformed into linguistically structured labels only after geometric extraction and human review. This suggests that MA-Bench-Train is designed to preserve motion fidelity while still being compatible with generative multimodal modeling.

3. Annotation schema and formal representation

Each caption $C_i$ is organized around body-part motion descriptors and high-level action semantics. The schema includes three explicit components: a Key part, defined as the coarse-grained body-part label; a Micro-action, defined as the fine-grained action label; and Motion detail, defined as a natural-language description of amplitude, direction, and rhythm, with examples such as “the right hand moves upward from torso-height...” (Li et al., 27 Mar 2026).

The annotation space is grounded in a body-part set

$$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$

For video $i$, the motion descriptor for part $p$ is given as

$$d_{i,p} = \big[ (v_{i,p,t}, x_{i,p,t}, y_{i,p,t}) \big]_{t=1}^{T_i},$$

where $v_{i,p,t} \in \mathbb{R}^2$ is the optical-flow vector at frame $t$, and $(x,y)$ are normalized image coordinates.

The paper also gives a formal representation of each video’s micro-action annotation as a sequence of triplets:

$$A_i = \{(t_{i,j},\, p_{i,j},\, a_{i,j})\}_{j=1}^{N_i},$$

where $t_{i,j} \in [0, T_i]$ is a temporal segment of the $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$0-th micro-action, $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$1 is the associated body part, and $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$2 is the fine-grained action label, with examples including “shaking head” and “curling left hand.”

This representation makes the corpus notable for combining temporal segmentation, body-part localization, and action naming within a single annotation formalism. A plausible implication is that it can support both caption generation and temporally grounded reasoning tasks without requiring separate labeling ontologies.

4. Statistical profile and lexical characteristics

The corpus contains 20 510 videos and 20 510 captions. Body-part labels are reported as balanced across 7 major regions, and action labels are drawn from the MA-52’s 52 categories (Li et al., 27 Mar 2026). The distribution is also described as long-tail, with approximately 10% of videos covering the rarest micro-actions.

Caption statistics indicate an average caption length of approximately 25 words. Reported high-frequency terms include “right hand,” “leftward,” “upward,” “contact,” “phase,” and “segment.” The associated word cloud is said to indicate emphasis on directional adverbs and temporal markers.

These properties show that MA-Bench-Train is not only large in clip count but also explicitly structured around directional and temporal language. This suggests a training signal that prioritizes motion trajectory, rhythmic patterning, and part-specific semantics over broader scene description.

5. Training objectives and fine-tuning configuration

The prescribed data split uses all 20 510 videos of MA-Bench-Train for training, while validation and test are conducted on the 1 000 videos of MA-Bench as a cross-subject held-out set (Li et al., 27 Mar 2026). Two learning objectives are specified. The first is generative captioning, which maximizes the log-likelihood of the reference caption. The second is classification for multiple-choice QA, using cross-entropy on the correct option $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$3. The total loss combines $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$4 and $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$5, with $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$6 in the experiments.

The base model for fine-tuning is Qwen3-VL-8B, described as a vision encoder plus LLM decoder. The paper evaluates three LoRA-based strategies: B2, LoRA on the vision encoder only; B3, LoRA on the language decoder only; and B4, LoRA on both, described as full fine-tuning. The reported configuration uses LoRA rank $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$7, $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$8, learning rate $$P = \{ \text{head}, \text{upper\_arm}, \text{lower\_arm}, \text{upper\_leg}, \text{lower\_leg}, \text{body}, \ldots \}.$$9, batch size $i$0, AdamW as optimizer, 50 000 training steps, and frame sampling of 8 frames per clip, which is stated to be optimal per ablation in Table 3.

In practical guidance, the paper recommends domain-adaptive pretraining with MA-Bench-Train, using approximately 8 frames per video because too few frames under-sample motion and too many introduce redundancy. It also recommends fine-tuning both the vision-encoder and language-decoder submodules for the best trade-off of performance versus computation.

6. Empirical effects, interpretation, and limitations

The paper reports that Qwen3-VL-8B fine-tuned on MA-Bench-Train improves on both closed-ended and open-ended tasks (Li et al., 27 Mar 2026). For closed-ended evaluation, defined as average accuracy across CMAR, FMAR, SAD, MAD, MAS, and PPR, performance increases from 46.97% before fine-tuning to 50.68% after MA-Bench-Train fine-tuning with full LoRA, a gain of 3.71 percentage points. For open-ended evaluation, defined as average LLM-as-judge score across MADU and MARE on a 0–5 scale, performance increases from 0.75 to 1.69, a gain of 0.94.

Task-level gains are also reported from B1 to B4. SAD increases from 56.10 to 60.30. MAD, identified as Multi-Action Detail, increases from 53.69 to 59.65. MARE, identified as Reasoning and Explanation, increases at three levels: L1 from 0.50 to 1.50, L2 from 0.51 to 1.67, and L3 from 0.47 to 1.54.

These results are presented as evidence that MA-Bench-Train improves both micro-action reasoning and explanation tasks. At the same time, the paper states a clear limitation: the data setting is interviews with limited background variation, and models should be generalized with caution to in-the-wild scenarios. It also advises ensuring that QA templates match downstream tasks, including adjustments for multiple-choice versus generative prompts. Together, these constraints indicate that MA-Bench-Train is a targeted supervision resource for subtle human-related behaviors rather than a universal video-understanding corpus.