MA-Bench: Micro-Action Benchmark
- MA-Bench is a benchmark for fine-grained micro-action understanding in videos, assessing subtle body movements using a three-tier evaluation framework.
- It employs a detailed taxonomy of 52 micro-actions across seven body-part categories and leverages techniques like dense optical flow and LLM-generated captions.
- The evaluation spans perceptual recognition, relational comprehension, and interpretive reasoning to measure both closed-ended accuracy and open-ended reasoning.
Searching arXiv for the benchmark and closely related work to ground the article. {"query":"MA-Bench micro-action understanding arXiv", "max_results": 5} MA-Bench is a benchmark for fine-grained micro-action understanding in multimodal LLMs (MLLMs), designed to evaluate whether video-capable models can recognize subtle, low-intensity human body movements, reason about relations among body parts over time, and produce evidence-grounded interpretations of those movements. Introduced together with the training corpus MA-Bench-Train, it comprises 1,000 benchmark videos, 12,000 structured question-answer pairs, and a three-tier evaluation architecture spanning perceptual recognition, relational comprehension, and interpretive reasoning. Its central premise is that micro-actions—such as slight head turns, finger fidgets, and micro-postural adjustments—are diagnostically important for emotion and behavior analysis, yet had lacked a dedicated benchmark at micro-action granularity prior to this work (Li et al., 27 Mar 2026).
1. Scope, source data, and taxonomy
MA-Bench is built from videos sampled from Micro-Action-52 (MA-52), described as a large human-centered micro-action recognition dataset collected from professional psychological interviews and containing 22,422 videos, 205 participants, 7 body-level categories, and 52 action-level categories. The benchmark subset contains 1,000 videos covering all 52 action categories from MA-52 and is selected from 20 participants so as to diversify action patterns while maintaining a non-overlapping cross-subject setting with the training corpus. MA-Bench-Train contains 20,510 videos from 166 participants with detailed structured micro-action captions (Li et al., 27 Mar 2026).
The benchmark is explicitly whole-body in scope. Its body-level taxonomy comprises seven categories: Body, Head, Upper limb, Lower limb, Body-hand, Head-hand, and Leg-hand. Its action-level taxonomy comprises 52 fine-grained micro-actions, including examples such as Shaking head, Scratching or touching neck, Turning head, Adjusting clothing, and Curling legs. The clips are short: the average duration is 2.12 s, the maximum is 5.01 s, most clips fall in the 1–3 s range, and few exceed 4 s. This temporal scale places emphasis on motion sensitivity and temporal precision rather than long-horizon scene understanding.
MA-Bench is vision-only in the reported evaluations. No audio modality is used. Inputs are videos or sampled frames, with 8 frames used by default. This choice aligns the benchmark with motion-centric visual reasoning rather than multimodal speech or soundtrack cues.
2. Annotation schema and data construction
The benchmark adopts a motion-centric annotation strategy. Its micro-motion tracker combines CoTracker3 for dense optical flow with four directional components, YOLOv8x-Seg for human-centric ROI segmentation, and skeleton alignment to associate flow with body parts. The resulting body-part motion descriptors store part-wise motion vectors and normalized coordinates over time. An example descriptor structure includes entries such as Left_Hand, Left_Lower_Arm, Head, and Right_Lower_Arm, each represented as sequences of {"vector": [...], "coord": [...]} tuples (Li et al., 27 Mar 2026).
These descriptors are then converted into structured micro-action captions. LLMs, including DeepSeek-v3.2 and DeepSeek-R1, are prompted with the motion descriptors to produce body-part motion summaries and coherent, part-aware captions. The captions encode parts involved, motion types, directions, amplitude or rhythm, and stage-wise temporal evolution. They serve two roles: they support QA generation for MA-Bench and provide supervision for MA-Bench-Train.
QA generation is semi-automatic and follows three stages. First, questions are automatically generated from captions using task-specific templates, either multiple-choice or yes/no. Second, an LLM performs reflection verification to check consistency between the generated QA items and the underlying descriptors. Third, the items are manually rectified and subjected to multi-round human verification for clarity, exclusivity of options, and evidence grounding.
The benchmark contains 12,000 QA items in total. The paper’s conclusion specifies that these comprise 10,000 closed-ended QA pairs across six tasks and 2,000 open-ended items across two tasks. MA-Bench functions as an evaluation benchmark, whereas MA-Bench-Train functions as a fine-tuning corpus.
3. Three-tier evaluation architecture
MA-Bench organizes evaluation into three tiers and eight tasks, progressing from direct perception to structured reasoning. The design is intended to separate failures of recognition from failures of relational modeling and from failures of explanation (Li et al., 27 Mar 2026).
| Tier | Tasks | Output form |
|---|---|---|
| Micro-action perception | CMAR, FMAR | Multiple choice |
| Relational comprehension | SAD, MAS, MMAD, PPR | Yes/No |
| Interpretive reasoning | MADU, MARE | Open-ended text |
The first tier, micro-action perception, contains CMAR and FMAR. CMAR, or coarse-grained recognition, asks the model to identify the correct body-level category. FMAR, or fine-grained recognition, asks it to discriminate among the 52 action-level labels and therefore places weight on localized spatiotemporal cues.
The second tier, relational comprehension, is more explicitly temporal and part-aware. SAD, or Single Action Detail, evaluates execution details such as direction, amplitude, and rhythm for one atom action. MAS, or Micro-Action Sequence, tests temporal order and transitions among local movements. MMAD, referred to as MAD in some figures, targets relations among multiple parts, including simultaneity, transitions, and directional consistency. PPR, or Part-Proximity Relation, evaluates whether a model can reason about changes in spatial proximity between two parts over time.
The third tier, interpretive reasoning, comprises open-ended generation tasks. MADU, or Micro-Action Descriptive Understanding, requires faithful and objective descriptions of visible movement without inference. MARE, or Micro-Action Reasoning and Explanation, requires a coherent reasoning chain that links observed evidence to both coarse and fine labels and justifies those choices causally.
Representative prompt forms make the benchmark’s granularity concrete. CMAR asks which coarse-grained behavior category a clip belongs to. FMAR asks which fine-grained action label best matches the behavior. SAD asks questions such as whether the right hand moves rapidly upward in the initial phase. MAS asks whether one movement occurred before another. PPR asks whether one part noticeably approaches another in an early stage. MADU asks for a description of changes in movement, and MARE asks for a detailed reasoning process culminating in both coarse and fine labels.
4. Evaluation protocol and metrics
All reported baseline evaluations are zero-shot, using a standard prompt on sampled frames, with 8 frames per video by default. Few-shot evaluation is not used. A separate fine-tuned setting evaluates Qwen3-VL-8B after supervised fine-tuning on MA-Bench-Train with LoRA (Li et al., 27 Mar 2026).
Closed-ended tasks are scored by accuracy:
Open-ended tasks are scored by GPT-4o as an LLM-as-a-judge on a 0–5 scale. MADU is evaluated along three dimensions: L1 for core action semantics, L2 for spatial directions and relations, and L3 for temporal order and structure. MARE is also evaluated along three dimensions: L1 for coarse body-level label correctness, L2 for fine action-level label correctness, and L3 for reasoning chain consistency. The overall open-ended score is the average across these six dimension scores.
The paper does not report BLEU, METEOR, CIDEr, F1, calibration, significance testing, or inter-annotator agreement. This omission is important for interpreting the benchmark: its open-ended evaluation relies on structured LLM judgment rather than conventional n-gram overlap metrics or reported human agreement statistics.
A frame-sampling ablation on zero-shot Qwen3-VL-8B compares 4, 8, and 16 frames. The reported results are 46.61% closed and 0.71 open for 4 frames, 46.97% closed and 0.75 open for 8 frames, and 46.71% closed and 0.72 open for 16 frames. The benchmark therefore recommends 8 frames as the best accuracy-latency trade-off.
5. Baselines and empirical findings
MA-Bench evaluates 23 representative MLLMs: three proprietary models—GPT-4o, Gemini-2.5-Flash, and Grok-2-Vision—and twenty open-source models, including LLaVA-NeXT-Video-7B, VideoLLaMA2-7B, VideoLLaMA2.1-7B, VideoLLaMA3-2B, VideoLLaMA3-7B, VideoChat-Flash-Qwen2-7B, LLaVA-OneVision-7B, Phi-3.5-Vision, Phi-4-Multimodal, Pixtral-12B, InternVL2-8B, InternVL2.5-8B, InternVL3-8B, InternVideo2-Chat-8B, InternVideo2.5-Chat-8B, H2OVL Mississippi-2B, Qwen2-VL-7B, Qwen2.5-VL-3B, Qwen2.5-VL-7B, and Qwen3-VL-8B (Li et al., 27 Mar 2026).
On the six closed-ended tasks, the best proprietary average is Gemini-2.5-Flash at 50.70% average accuracy. GPT-4o records 44.87%, and Grok-2-Vision records 43.25%. Among open-source zero-shot models, the best average is VideoChat-Flash-Qwen2-7B at 49.87%, followed by InternVideo2.5-Chat-8B at 45.39%. Qwen3-VL-8B records 46.97% zero-shot and 50.68% after fine-tuning on MA-Bench-Train, effectively matching Gemini-2.5-Flash on the benchmark’s aggregate closed-ended score.
Taskwise results reveal uneven difficulty. For CMAR, Qwen3-VL-8B + Train reaches 47.90%, Gemini-2.5-Flash reaches 43.00%, and GPT-4o reaches 20.50%. For FMAR, Qwen3-VL-8B + Train reaches 32.60%, while zero-shot Qwen3-VL-8B reaches 37.11% and Gemini-2.5-Flash reaches 31.40%. Several models exceed 55–60% on SAD, MAD, and PPR, but FMAR and MAS remain particularly challenging. This suggests that fine-grained label discrimination and temporal ordering are harder than some binary relation queries.
On the two open-ended tasks, proprietary averages are low: Gemini-2.5-Flash scores 0.76, GPT-4o scores 0.73, and Grok-2-Vision scores 0.65. Qwen3-VL-8B zero-shot scores 0.75, whereas Qwen3-VL-8B fine-tuned on MA-Bench-Train reaches 1.69, with MADU scores of 1.50, 1.67, and 1.54 and MARE scores of 1.98, 1.78, and 1.67 across the three sub-dimensions. Several open-source models obtain near-zero scores on reasoning-heavy open-ended tasks despite nontrivial closed-ended accuracy, indicating a clear gap between recognition and explanation.
The qualitative error analysis is consistent with these numbers. Without domain fine-tuning, models often confuse head-related and limb-related micro-actions, miss subtle amplitude and direction cues, and generate reasoning chains that are inconsistent or unsupported. In one MARE example whose ground truth is Head and Shaking head, GPT-4o, Gemini-2.5-Flash, Grok-2-Vision, and zero-shot Qwen3-VL-8B instead produce leg- or upper-limb-centric labels such as Crossing legs or Curling legs, whereas Qwen3-VL-8B + MA-Bench-Train correctly identifies Head | Shaking head with an evidence-based chain referencing continuous left-right head motion and torso coordination.
6. MA-Bench-Train, fine-tuning, and limitations
MA-Bench-Train is a 20.5K-video training corpus constructed from 166 participants who do not overlap with the benchmark’s subjects. Each video is paired with structured micro-action captions synthesized from motion descriptors into part-aware, temporally organized narratives, and the pipeline also produces task-aligned prompts such as FMAR open-ended labeling and SAD yes/no judgments with rationales. Fine-tuning experiments use Qwen3-VL-8B with supervised fine-tuning via LoRA in LLaMA-Factory (Li et al., 27 Mar 2026).
The ablation over trainable components distinguishes three settings. Tuning the vision encoder only raises the closed-ended average from 46.97% to 48.62% and the open-ended average from 0.75 to 1.11. Tuning the LM decoder only yields 50.63% closed-ended and 1.60 open-ended. Tuning both modules yields the best results: 50.68% closed-ended and 1.69 open-ended.
In aggregate terms, Qwen3-VL-8B improves from 46.97% to 50.68% on closed-ended tasks, a gain of +3.71 percentage points or +7.9% relative, and from 0.75 to 1.69 on open-ended tasks, a gain of +0.94 absolute or +125% relative. The especially large gains on MARE indicate stronger interpretive reasoning after domain-adaptive supervision. At the same time, FMAR decreases from 37.11% in zero-shot Qwen3-VL-8B to 32.60% after fine-tuning, even though the overall closed-ended average improves. This suggests that gains from MA-Bench-Train are not uniform across all subtasks.
The benchmark’s limitations are also explicit. It is vision-only, with no audio in evaluation. The paper does not report inter-annotator agreement. Ethics, privacy, and licensing details are not described beyond the MA-52 sourcing and the cross-subject split, with the project page indicated as the source for usage terms. More broadly, the reported results show that many current MLLMs still fail on fine-grained, short-horizon dynamics and produce weak reasoning chains. A plausible implication is that micro-action understanding remains bottlenecked by motion-sensitive temporal modeling and by explicit modeling of inter-part relations, rather than by generic video-question answering capacity alone.