MoDirect: Multi-domain Motion-Direction Framework
- MoDirect is a multi-domain research motif characterized by explicit motion-direction signaling and direct routing strategies in complex systems.
- The Video-LLM dataset family—including MoDirect-SynBench, MoDirect-Inst, and MoDirect-RealBench—provides controlled settings and concrete metrics to evaluate directional binding accuracy.
- DeltaDirect, paired with MoDirect, significantly enhances motion-direction accuracy by addressing the 'direction binding gap' through targeted training interventions.
MoDirect most concretely denotes a dataset family for signed image-plane motion direction in Video-LLMs, introduced to diagnose “directional motion blindness” and to support direction-focused instruction tuning and evaluation (Lee et al., 21 May 2026). In adjacent literatures, the same label also appears as an informal shorthand for several direct-routing, direction-oriented, or direct-preference formulations rather than for a single canonical method. This suggests that “MoDirect” is best understood as a family of research motifs centered on explicit direction, expert, or preference selection, with the Video-LLM dataset family providing the most literal and fully specified usage (Lee et al., 21 May 2026).
1. Scope and nomenclature
The term is used across multiple technical contexts. In one case it is the formal name of a benchmark suite; in others it is used to describe a design pattern or methodological family.
| Context | Usage of “MoDirect” | Source |
|---|---|---|
| Video-LLMs | Dataset family specifically for signed image-plane motion direction | (Lee et al., 21 May 2026) |
| LLM systems | Design pattern “very close to the design pattern that a system called ‘MoDirect’ would follow” | (Simonds et al., 2024) |
| Preference alignment | “A good shorthand” for a mixture-of-experts / mixture-based extension of DPO | (Bohne et al., 9 Oct 2025) |
| Motion forecasting | “MoDirect-style” separation of directional intentions and dynamic states | (Zhang et al., 2024) |
| Multi-objective optimization | Can be understood as a MoDirect-type direction-oriented method | (Xiao et al., 2023) |
| Direct NMT | Task-level MoE systems discussed as “MoDirect systems built on Task-level MoE” | (Tourni et al., 2023) |
The most important misconception is that these usages refer to one unified framework. They do not. The shared pattern is structural rather than taxonomic: explicit routing or regularization toward a direction, domain, task, or preference component. This suggests that “MoDirect” is presently a cross-domain research label rather than a standardized architecture.
2. MoDirect as a motion-direction benchmark for Video-LLMs
In the Video-LLM literature, MoDirect is a dataset family introduced to study signed image-plane motion direction under controlled and real-video settings (Lee et al., 21 May 2026). It contains three components. MoDirect-SynBench is a controlled synthetic benchmark with 8-frame videos at , exactly one moving object, constant-velocity linear motion, and four signed directions: left, right, up, and down. It uses a factorial design over foreground type and background type, yielding Primitive-on-Syn, Cutout-on-Syn, Primitive-on-Real, and Cutout-on-Real domains, with 1500 videos per direction and 6000 per domain. MoDirect-Inst is a synthetic instruction-tuning corpus of about 100K video–question–answer triples derived from the Primitive-on-Syn domain, with perturbed linear, zigzag, circular, roundtrip, and static motions; 8 canonical directions plus stationary; and diverse question types spanning direction, motion description, appearance, location, motion existence, trajectory, and rotation. MoDirect-RealBench reformulates subsets of SSv2, KTH, and TOMATO into motion-direction multiple-choice problems, with 722 samples from SSv2, 899 from KTH, and 403 from TOMATO (Lee et al., 21 May 2026).
The benchmark is designed around a multiple-choice binding requirement rather than pure perceptual decoding. In MoDirect-SynBench, options are randomized per sample, so a model must infer direction and then bind that internal representation to the correct answer option. This design exposed the central empirical finding of the paper: many Video-LLMs exhibit “directional motion blindness.” For LLaVA-Video-7B, MoDirect-SynBench average accuracy is 25.9%, essentially chance for a 4-way task, with domain scores of 27.6 on Primitive-on-Syn, 23.4 on Cutout-on-Syn, 26.9 on Primitive-on-Real, and 25.8 on Cutout-on-Real. MoDirect-RealBench average accuracy is 43.1%, with 52.2 on SSv2, 50.3 on KTH, and 26.8 on TOMATO. Other off-the-shelf Video-LLMs are similar; Qwen3-VL-4B is reported as the strongest baseline but still only reaches about 49.7% on SynBench and about 52.7% on RealBench (Lee et al., 21 May 2026).
The paper’s diagnostic result is that this failure is not primarily perceptual. On the simple synthetic domain, a linear classifier can decode the true direction from vision encoder outputs at about 99.8%, from projector outputs at about 96.5%, from visual tokens inside the LLM at , and from the readout token at , while the actual multiple-choice answer remains only about 27.6% (Lee et al., 21 May 2026). The authors therefore identify a “direction binding gap”: direction remains linearly accessible throughout the model pipeline, but the readout fails to bind that signal to the correct verbal answer option. This directly counters the simpler interpretation that Video-LLMs merely lack motion-direction features.
3. DeltaDirect and the supervised strengthening of motion direction
MoDirect is paired with DeltaDirect, a training-only objective applied at the projector output of a Video-LLM (Lee et al., 21 May 2026). Let the projector output be . DeltaDirect forms mean-pooled adjacent-frame deltas
and regresses a normalized 2-D motion vector target
through a linear head
The Motion Vector Prediction loss is
and the total training loss is
The MVP branch is discarded at inference, so DeltaDirect changes training but not deployment-time architecture (Lee et al., 21 May 2026).
This intervention targets the motion signal before it is weakened by later multimodal processing. The ablations in the paper report that applying the supervision after the projector is best; doing so at the vision encoder, LLM layers, or final readout is weaker or harmful. The target choice also matters: 2-D unit motion vector regression is stronger than frame-order prediction, delta equivariance, or concatenated deltas, and a linear head is sufficient (Lee et al., 21 May 2026).
Empirically, the gains are large. The abstract reports that on MoDirect-SynBench, instruction tuning with DeltaDirect improves motion-direction accuracy from 25.9% to 85.4%, and on MoDirect-RealBench DeltaDirect improves real-world motion-direction accuracy by 21.9 points over the vanilla baseline without real-world tuning data (Lee et al., 21 May 2026). For LLaVA-Video-7B specifically, vanilla performance is 25.9% on SynBench and 43.1% on RealBench; MoDirect-Inst instruction tuning alone raises these to 78.9% and 58.1%; adding DeltaDirect raises them further to 85.4% and 65.0%. The largest synthetic-domain gain is on Cutout-on-Real, from 60.5% with MoDirect-Inst alone to 71.7% with DeltaDirect. The same paper also reports that standard video-understanding benchmarks are preserved or slightly improved, with “Standard” averages rising from 69.4 to 70.1 and “Fine-grained” averages from 47.3 to 48.7 for LLaVA-Video-7B (Lee et al., 21 May 2026).
A second diagnostic layer concerns concept geometry. After instruction tuning on Primitive-on-Syn, cross-domain direction concept vectors achieve cosine similarity above 0.9 in late layers, so orientation generalizes, but magnitude decays from about 28.9 on Primitive-on-Syn to about 20.5 on Cutout-on-Syn, about 18.5 on Primitive-on-Real, and about 14.2 on Cutout-on-Real (Lee et al., 21 May 2026). This suggests that out-of-domain degradation is not chiefly an axis-rotation problem; it is a signal-strength problem.
4. MoDirect as direct routing and direct preference composition
A distinct usage appears in LLM systems. The MoDEM framework is presented as a concrete example of what may be called “direct routed experts” for LLMs: instead of embedding experts inside one large transformer, it uses a separate router model to choose among several full-fledged domain-specific LLMs (Simonds et al., 2024). The router is DeBERTa-v3-large with 304M parameters, trained as a 5-way classifier over Math, Health, Science, Coding, and Other. Given a prompt 0, routing is top-1 and hard: 1 Only one expert model is executed per query, plus the small router. The paper explicitly notes that this is “very close to the design pattern that a system called ‘MoDirect’ would follow” (Simonds et al., 2024).
MoDEM differs from classical sparse MoE in two ways that matter for terminology. Its experts are entire models rather than sub-networks, and routing is prompt-level domain classification rather than token-level gating. The router reaches about 97% average accuracy on held-out test sets of its training data, and on an out-of-distribution MMLU domain-mapping evaluation it achieves 81.00% overall, with 96.63% on Math, 81.18% on Health, 83.02% on Science, 77.42% on Coding, and 52.94% on Other (Simonds et al., 2024). In the medium configuration, MoDEM improves over Llama 3.1 70B on MMLU from 86.0% to 87.7%, on HumanEval from 80.5% to 86.5%, on MATH from 65.7% to 85.9%, and on Olympiad Bench from 27.7% to 49.0%, while the paper reports an MMLU inference cost of \$2\times2$25.00 for Llama 3.1 405B (Simonds et al., 2024). In this usage, “MoDirect” denotes direct semantic routing to specialized experts.
A related preference-alignment usage is explicit in “Mix- and MoE-DPO,” which states that “MoDirect” is a good shorthand for a mixture-of-experts / mixture-based extension of Direct Preference Optimization (Bohne et al., 9 Oct 2025). The model defines a mixture policy
$2\times2$3
introduces a latent expert variable $2\times2$4, models preferences with a Mixture-of-Bradley–Terry likelihood, and optimizes a variational ELBO. Mix-DPO uses fixed mixture weights, whereas MoE-DPO uses input-dependent gating $2\times2$5. The paper reports that on a multi-task sentiment setting with book versus movie reviews, baseline DPO achieves $2\times2$6 on movie sentiment and $2\times2$7 on book sentiment, while MoE-DPO with frozen gating reaches $2\times2$8 and $2\times2$9, and MoE-DPO with learnable gating reaches $98.1\%\pm1.3\%$0 and $98.1\%\pm1.3\%$1 (Bohne et al., 9 Oct 2025). Here, the “MoDirect” reading is direct preference optimization with explicit expert assignment.
A related but distinct direct-preference line is MODPO, which generalizes DPO to multiple alignment objectives and trains policies as implicit collective reward models for specific weight vectors $98.1\%\pm1.3\%$2 (Zhou et al., 2023). Its central loss augments the DPO logit with weight-dependent scaling and a reward-model margin term from auxiliary objectives. The paper reports that MODPO matches or outperforms MORLHF while using about three times fewer GPU hours per policy, with 4.0 $98.1\%\pm1.3\%$3 0.1 hours versus 13.8 $98.1\%\pm1.3\%$4 0.7 hours on safety alignment and 9.4 $98.1\%\pm1.3\%$5 0.2 versus 34.0 $98.1\%\pm1.3\%$6 0.5 on long-form QA (Zhou et al., 2023). Although that work does not name its method “MoDirect,” it belongs to the same direct-preference family invoked by the label.
5. MoDirect as explicit direction decomposition and direction-oriented optimization
In motion forecasting, DeMo is described as “exactly the kind of ‘MoDirect’-style approach” built around explicit separation of directional intentions and dynamic states (Zhang et al., 2024). Instead of the one-query-one-trajectory paradigm, it introduces mode queries for distinct directional intentions and state queries for dynamic states over time, then fuses them into hybrid queries. State queries are processed by scene cross-attention and Bidirectional Mamba, while mode queries undergo scene cross-attention and mode-to-mode self-attention. The resulting architecture separates multi-modality from temporal evolution before recombining them (Zhang et al., 2024).
This distinction is not merely conceptual. On Argoverse 2 validation for single-agent forecasting with $98.1\%\pm1.3\%$7, a one-query-one-trajectory baseline reports $98.1\%\pm1.3\%$8, $98.1\%\pm1.3\%$9, and $95.3\%\pm2.4\%$0, whereas full DeMo reports $95.3\%\pm2.4\%$1, $95.3\%\pm2.4\%$2, and $95.3\%\pm2.4\%$3 (Zhang et al., 2024). On the Argoverse 2 single-agent test leaderboard, DeMo reports $95.3\%\pm2.4\%$4, $95.3\%\pm2.4\%$5, $95.3\%\pm2.4\%$6, $95.3\%\pm2.4\%$7, and $95.3\%\pm2.4\%$8. It is also efficient: 5.9M parameters versus QCNet’s 7.7M, 22.8 GFLOPs versus QCNet’s 53.4 GFLOPs, training time of 9h versus QCNet’s 45h on 8× RTX 3090, and about 38ms inference per scene versus QCNet’s 94ms (Zhang et al., 2024). In this interpretation, “MoDirect” means explicit direction-state factorization.
A more abstract optimization-theoretic usage appears in “Direction-oriented Multi-objective Learning,” which can be understood as a MoDirect-type method because it regularizes a common descent direction around a target direction $95.3\%\pm2.4\%$9 (Xiao et al., 2023). The paper formulates the direction-oriented problem through
$\mathbf{F}\in\mathbb{R}^{T\times N\times D}$0
which reduces to minimizing $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$1 over simplex weights $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$2. Small $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$3 recovers MGDA-like behavior, while increasing $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$4 approaches gradient descent on a scalarized objective. The proposed SDMGrad and SDMGrad-OS achieve convergence to a Pareto stationary point for constant $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$5, and the paper reports strong empirical performance, including $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$6 and $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$7 on NYU-v2, and $\mathbf{F}\in\mathbb{R}^{T\times N\times D}$8 success on Meta-World MT10 for SDMGrad (Xiao et al., 2023). The “MoDirect” affinity here is explicit descent-direction control rather than routing.
6. Additional technical uses and broader interpretation
Further uses of the label or pattern are distributed across other domains. In direct neural machine translation, Task-level Mixture of Experts is discussed as a basis for “MoDirect systems built on Task-level MoE,” where routing is per translation task rather than per token (Tourni et al., 2023). The system uses top-2 routing, and the paper reports that the 16-expert Task-level MoE outperforms bilingual and pivot-based models for 7 language pairs, while pivot-based models still perform better in 9 pairs and directions (Tourni et al., 2023). This is another instance of direct task-to-expert association.
In transportation systems, a proposed decision-support methodology for Mobility-on-Demand design is explicitly described as a possible “MoDirect” tool: a framework combining a mode choice model, a supply-side MoD simulator, an equilibrium inner loop, and a Bayesian optimization outer loop over fleet sizes and discount factors (Liu et al., 2018). The Manhattan case study reports convergence of the multimodal supply-demand system and a best profit of about 145,015 under Bayesian optimization on the full-scale case, versus about 126,308 for random search under the same evaluation budget (Liu et al., 2018). In this usage, the label denotes direct design optimization for MoD systems rather than motion or LLM routing.
In geometric computing, “MoDirect” is used as a synonym for the Modified Direct Method (MDM), an iterative mesh-smoothing method for planar and surface meshes derived from Balendran’s Direct Method (Mei et al., 2012). MDM is reported to be effectively identical to Laplacian smoothing for planar triangular and quadrilateral meshes, to outperform Laplacian smoothing for tri-quad meshes, and to be convergent on tested planar and surface triangular, quadrilateral, and tri-quad meshes (Mei et al., 2012). This is historically unrelated to the Video-LLM usage but shows that the label also has an established meaning in mesh processing.
Taken together, these literatures indicate that “MoDirect” names either a specific dataset family for motion-direction understanding or a broader pattern of direct specialization: direct routing to domain experts, direct assignment of preference mass to expert policies, direct decomposition of directional intent from temporal state, or direct regularization of a descent direction. This suggests that the term’s research identity is currently strongest where the object being made explicit—motion direction, domain, task, or preference—is itself the main latent variable of interest.