Directional Motion Blindness in Video-LLMs

• Directional Motion Blindness is defined as the inability of video AI models to accurately recover motion direction when subtle frame-to-frame changes are lost.
• Research shows that despite high internal decodability of motion signals (over 95% accuracy), the final output remains near chance levels (25–30%), exposing a critical direction binding gap.
• Mitigation strategies like DeltaDirect, which reinforce frame-to-frame displacement cues, improve performance yet underscore broader challenges in physically grounded video understanding and accessibility.

Directional Motion Blindness (DMB) denotes a systematic failure of contemporary video-capable multimodal models to recover the direction of motion, even when they can correctly recognize the moving object or describe the scene. In the accessibility literature, DMB is introduced as a special case of Implicit Motion Blindness (IMB), a broader inability of frame-based systems to perceive continuous, low-signal motion because temporal continuity is discarded during sampling (Zhang, 11 Aug 2025). In the Video-LLM literature, the same phenomenon is characterized more narrowly as failure on signed image-plane motion questions such as left, right, up, and down, with many models performing near chance despite retaining direction information in internal representations (Lee et al., 21 May 2026). Taken together, these accounts identify DMB as a foundational weakness at the interface between visual perception, temporal representation, and language-conditioned answer generation.

1. Definition and conceptual scope

In the accessibility framing, IMB is the inability of frame-based AI models to perceive continuous, low-signal motion because temporal continuity is lost at sampling time. DMB is its special case: failure to recover the direction of motion when the signal between frames is uniform, low-contrast, or highly repetitive (Zhang, 11 Aug 2025). This formulation emphasizes motion as a physical attribute of the scene rather than a semantic label attached to an object.

The underlying conceptual model treats a video as a continuous signal $V(t)$ with an underlying motion field $M(t)$, identified with optical flow. Humans are described as perceiving $M(t)$ directly via optical-flow mechanisms, whereas frame-sampling models convert $V(t)$ into a sparse discrete sequence $\{f_1, f_2, \dots, f_N\}$ at times $t_i = i / f_s$, with $N \ll T_v$ (Zhang, 11 Aug 2025). Under this account, the fine-grained temporal derivatives that encode motion are discarded before downstream stages such as patchification, encoding, or fusion.

In the Video-LLM diagnosis, DMB is defined operationally as systematic failure on basic signed image-plane motion questions. On simple videos of a single object moving left, right, up, or down, many state-of-the-art models perform near random, approximately $25\%$ accuracy for four-way direction classification (Lee et al., 21 May 2026). Here, DMB is treated as failure on a minimal perceptual primitive that underpins navigation, physical interaction, and richer temporal reasoning.

These two definitions are complementary but not identical. One emphasizes loss of temporal continuity in frame-based architectures; the other emphasizes failure to map internally available direction information onto the correct verbal answer. This suggests that “Directional Motion Blindness” names a family of related failure modes rather than a single mechanistic defect.

2. Formal characterization of the failure

The accessibility account formalizes video input as

$$V \in \mathbb{R}^{T_v \times H \times W \times C}, \qquad \mathrm{Sample}(V, N) \to V_s = (f_1 \dots f_N), \qquad N \ll T_v.$$

Each sampled frame is then patchified:

$$\mathrm{Patch}(f_i) \to K \text{ patches } p_{i,j} \in \mathbb{R}^{P \times P \times C}.$$

The central claim is that temporal derivatives $M(t)$0 at times $M(t)$1 are irrecoverably lost (Zhang, 11 Aug 2025). The paper further states that equations from fusion onward cannot re-create $M(t)$2 or the flow field $M(t)$3.

A more specific formal intuition is given for the directional case. If the true frame-to-frame displacement satisfies $M(t)$4 and this displacement is smaller than pixel-level noise, or if successive changes repeat identically as with escalator treads, then sampled frames satisfy $M(t)$5 (Zhang, 11 Aug 2025). Under those conditions, the model effectively receives the same static image repeatedly and cannot infer the sign or vector of $M(t)$6.

The same source invokes a Nyquist analogue: if the sampling frequency $M(t)$7 is below twice the highest frequency of motion change, aliasing or undetectability occurs. A concrete example is given as “64 frames for 5 min video $M(t)$8 frame/5s,” under which low-signal continuous movement is effectively invisible (Zhang, 11 Aug 2025).

The Video-LLM diagnosis begins from a standard pipeline. A video is sampled to $M(t)$9 frames; a vision encoder such as SigLIP produces patch features

$M(t)$0

a projector transforms them into visual tokens

$M(t)$1

and an LLM consumes these visual tokens together with the text prompt to generate an answer (Lee et al., 21 May 2026). Unlike the accessibility account, this analysis argues that signed motion direction remains linearly accessible through much of the pipeline, which locates the failure not in total absence of directional signal but in how that signal is used at readout time.

3. The escalator problem and accessibility-specific significance

A canonical illustration of DMB is the “Escalator Problem”: a first-person video of two escalators moving in opposite directions, where a blind or visually impaired (BVI) user must know which escalator is ascending (Zhang, 11 Aug 2025). The reported model behavior is consistent across the examples described in the paper: the model correctly labels “escalator,” but then either denies seeing motion or guesses a random direction, often incorrectly.

The mechanics of this failure are linked to the dominant frame-sampling paradigm. The pipeline described in the paper is: sample $M(t)$2 frames to obtain $M(t)$3; patchify each $M(t)$4 into patches $M(t)$5; encode patches to features $M(t)$6; fuse with text features $M(t)$7; and decode to an answer (Zhang, 11 Aug 2025). Because successive frames are near-identical under low-signal, uniform texture, the visual encoder $M(t)$8 sees no meaningful delta, and no semantic or attentional mechanism flags directional flow.

The importance of this failure is not limited to escalators. The paper explicitly lists revolving doors, crowd flow, moving water, sliding doors, and baggage carousels as additional “implicit motion” challenges (Zhang, 11 Aug 2025). These examples delimit a class of scenes in which semantic object recognition is insufficient for safe action because the decisive variable is dynamic state rather than category identity.

The accessibility significance is framed in terms of trust and safety. The paper identifies a “Crisis of Trust”: if a tool fails predictably on motion tasks, BVI users are forced into constant vigilance about whether an object is safe to trust. It further connects unpredictability to increased mental burden through Sweller’s cognitive-load theory, and argues that once a fundamental task such as escalator direction is unreliable, users cannot safely depend on the system for dynamic guidance such as crossing a street (Zhang, 11 Aug 2025). A plausible implication is that DMB is not merely an evaluation anomaly but a deployment-limiting safety defect for assistive systems.

4. Internal representation, readout, and the direction binding gap

The diagnostic study on Video-LLMs traces direction information through four stages of the model pipeline. Using four-way linear probes for left, right, up, and down on frozen representations, it reports $M(t)$9 motion-direction accuracy from the vision-encoder output $V(t)$0, $V(t)$1 from projector output $V(t)$2, approximately $V(t)$3 from LLM visual-token hidden states $V(t)$4, and approximately $V(t)$5 in late layers for the LLM readout hidden state $V(t)$6 (Lee et al., 21 May 2026). These values indicate that signed direction remains linearly decodable throughout the internal computation.

Despite this, final multiple-choice answer accuracy remains near chance, approximately $V(t)$7–$V(t)$8 (Lee et al., 21 May 2026). The paper names the discrepancy between internal decodability and answer performance the “direction binding gap”: the model fails to bind the internal direction signal to the correct verbal output token, whether that token is an answer option such as A/B/C/D or a word such as “left” or “right.”

Across nine Video-LLMs, the reported gap between probe performance and multiple-choice accuracy exceeds $V(t)$9 percentage points, which the paper describes as a universal failure (Lee et al., 21 May 2026). On that basis, the authors conclude that in these systems DMB is “not a perception failure but a readout binding failure.”

This conclusion does not nullify the accessibility account; rather, it introduces a mechanistic distinction. In one account, temporal sampling destroys motion-relevant information before later processing. In the other, enough information survives to remain linearly decodable, but the language-conditioned decision process fails to align it with the answer vocabulary. This suggests that DMB may emerge at different loci depending on the architecture, task format, motion regime, and evaluation protocol.

5. Benchmarks, datasets, and empirical findings

The principal benchmark suite introduced for diagnosing and mitigating DMB in Video-LLMs is MoDirect, a dataset family for motion direction instruction tuning and evaluation (Lee et al., 21 May 2026). It includes three components.

MoDirect-Inst contains $\{f_1, f_2, \dots, f_N\}$0 K videos built from the simplest synthetic domain, Primitive-on-Syn, defined as colored geometric shapes on a flat background moving linearly. It includes mixed QA formats: four-way multiple-choice questions, open-ended questions, and appearance questions. MoDirect-SynBench contains $\{f_1, f_2, \dots, f_N\}$1 K videos and uses a $\{f_1, f_2, \dots, f_N\}$2 factorial design over foreground and background: geometric primitives versus cut-out real objects, and synthetic solid versus real scene backgrounds. Each video contains a single object moving exactly left, right, up, or down, with randomized start and end positions, and evaluation uses balanced four-way multiple-choice questions. MoDirect-RealBench contains approximately $\{f_1, f_2, \dots, f_N\}$3 K videos collected from Something-Something-V2, KTH, and TOMATO, reformulated into left/right/up/down multiple-choice questions with chance levels of $\{f_1, f_2, \dots, f_N\}$4 or $\{f_1, f_2, \dots, f_N\}$5 (Lee et al., 21 May 2026).

The reported results are substantial. On MoDirect-SynBench, LLaVA-Video-7B zero-shot obtains $\{f_1, f_2, \dots, f_N\}$6 average accuracy across the four synthetic splits. After MoDirect-Inst instruction tuning, this rises to $\{f_1, f_2, \dots, f_N\}$7, but falls to $\{f_1, f_2, \dots, f_N\}$8 on the hardest Cutout-on-Real domain, which the authors interpret as reopening of the binding gap. Adding DeltaDirect raises the synthetic average to $\{f_1, f_2, \dots, f_N\}$9, and raises Cutout-on-Real to $t_i = i / f_s$0, an improvement of $t_i = i / f_s$1 percentage points (Lee et al., 21 May 2026).

On MoDirect-RealBench, LLaVA-Video-7B zero-shot achieves $t_i = i / f_s$2 average accuracy, while DeltaDirect improves it to $t_i = i / f_s$3, a gain of $t_i = i / f_s$4 points, without using real videos in training (Lee et al., 21 May 2026). The paper also reports that standard video-understanding performance is preserved on MVBench, NExT-QA, Perception-Test, EgoSchema, TGIF-QA, TempCompass, VinoGround, FAVOR, and MotionBench, with examples including MVBench $t_i = i / f_s$5 points and TempCompass $t_i = i / f_s$6 points.

The same study states that DeltaDirect is backbone-agnostic in the tested settings: applying it to LLaVA-OneVision-7B and Qwen3-VL-4B yields comparable gains, reported as Avg. Dir $t_i = i / f_s$7 points and $t_i = i / f_s$8 points, respectively (Lee et al., 21 May 2026).

For accessibility-oriented evaluation, the position paper does not introduce a benchmark implementation, but it specifies a benchmark agenda. The proposed human-centered benchmarks prioritize primary tasks such as escalator direction, crowd flow, and door state; metrics that measure trustworthiness, specifically consistency and uncertainty calibration rather than only accuracy; and data based on egocentric, continuous first-person streams from wearable cameras, co-designed with the BVI community (Zhang, 11 Aug 2025). This agenda shifts evaluation from abstract VQA toward real-world dynamic guidance.

6. Mitigation strategies and broader research agenda

The principal mitigation proposed for Video-LLMs is DeltaDirect, a projector-level auxiliary objective designed to reinforce signed two-dimensional displacement cues before they enter the LLM (Lee et al., 21 May 2026). The method is defined by adjacent-frame feature deltas:

$t_i = i / f_s$9

A lightweight linear head $N \ll T_v$0 maps the feature delta to an unnormalized motion vector:

$N \ll T_v$1

The prediction is normalized and compared to the true unit-direction target $N \ll T_v$2:

$N \ll T_v$3

Supervision uses mean-squared error on the unit circle:

$N \ll T_v$4

and the full loss is

$N \ll T_v$5

where $N \ll T_v$6 is the cross-entropy on next-token prediction and $N \ll T_v$7 is typically $N \ll T_v$8 (Lee et al., 21 May 2026). The MVP branch is removed after training, so inference proceeds unchanged.

The accessibility paper advances a broader research program rather than a specific model. Its first demand is a shift from Recognition to Perception: from the question “What is this object?” to “How is the scene behaving?” in terms of direction, velocity, and physical state (Zhang, 11 Aug 2025). This distinction recasts failure analysis away from purely semantic competence.

The paper lists three example research directions. The first is hybrid architectures based on two-stream fusion, pairing a classic optical-flow network such as RAFT operating on dense or event-based frame pairs with a semantic MLLM, where the flow stream provides motion priors or acts as a verification module. The second is novel sensors, specifically event cameras, which provide asynchronous, pixel-level brightness-change signals that directly encode motion with microsecond resolution; the stated challenge is developing architectures or tokenizers that can natively process sparse event streams alongside frame-based data. The third is physics-informed learning, imposing continuity and rigid-body constraints during training so that models learn laws of motion rather than memorizing appearances (Zhang, 11 Aug 2025).

The same paper closes with a broader call to re-evaluate foundational assumptions in video AI for accessibility, to prioritize end-user safety and trust over benchmark scores on static datasets, and to co-develop benchmarks, metrics, and models with accessibility experts and the BVI community (Zhang, 11 Aug 2025). In the Video-LLM paper, the broader implications extend beyond signed direction to possible analogous binding gaps for speed, depth motion, rotation, and object interactions (Lee et al., 21 May 2026). A plausible synthesis is that DMB functions both as a narrow diagnostic target and as a proxy for deeper limitations in physically grounded video understanding.