---
title: '3MDBench: Robotics & Multi-Domain Benchmark'
url: https://www.emergentmind.com/topics/3mdbench
type: topic
---

# 3MDBench: Robotics & Multi-Domain Benchmark

3MDBench designates multiple distinct research benchmarks, each serving a specific domain in AI, robotics, medicine, or scientific simulation. This article surveys major “3MDBench” systems as referenced in leading literature: (1) a large-scale robotics benchmark for joint whole-body motion generation in mobile manipulation (“M3Bench”, also referred to as “3D Mobile-Manipulation Benchmark”), (2) a medical multimodal multi-agent dialogue benchmark for LVLM-driven telemedicine (“Medical 3MDBench”), and (3) a comprehensive instruction-following benchmark around multi-modal 3D scene prompts (“M3DBench”). Each variant sets specialized requirements, methodologies, and evaluation protocols.

## 1. Robotics: 3D Mobile-Manipulation Benchmark (M3Bench/3MDBench)

3D Mobile-Manipulation Benchmark—or “M3Bench” (hereafter, Editor’s term: 3MDBench (Robotics))—is a large-scale platform enabling rigorous evaluation of whole-body motion generation by mobile manipulators in 3D household scenes [2410.06678]. “Whole-body motion” stipulates joint trajectories for both the mobile base and manipulator arm (including end-effector), under kinematic, environmental, and task constraints.

Core tasks require an agent, given a partial 3D scan (point cloud), a target-object mask, and an abstract task instruction (“Pick that salt shaker”), to plan and execute continuous, coordinated base-arm motions for navigation, reaching into occlusion, stable grasping or placing, and persistent collision avoidance.

Key objectives:
- Standardized, physics-grounded evaluation of collision-free, kinematically feasible joint-space trajectories.
- Testing of generalization: new objects, new scenes, and unseen object–scene combinations.
- Support for multi-modal embodied-AI research via rich metadata: URDF annotations, natural language instructions, panoptic maps, egocentric videos.

## 2. Dataset, Coverage, and Scene Statistics

3MDBench (Robotics) comprises:
- 119 photo-realistic, physics-enabled household scans (from PhyScene), diversifying kitchen, living room, bedroom, and bathroom layouts.
- 32 object categories (e.g., cups, bottles, books), 588 distinct instances.
- 30,000 object rearrangement demonstrations: ≈20,000 pick, ≈10,000 place, each with 30 waypoints.
- Data splits:
  - **Base:** Train (75%), Validation (5%), Test (20%)
  - **Novel-Object:** Unseen objects in observed scenes
  - **Novel-Scene:** Seen objects in held-out scenes
  - **Novel-Scenario:** Unseen object–scene pairs

Splits allow component-wise assessment of object- and scene-level generalization.

## 3. M3BenchMaker: Automatic Trajectory Generation Pipeline

The M3BenchMaker toolkit orchestrates data synthesis through four modular stages:
1. **Task Builder**: Instantiates configuration for each manipulation episode, given scene/robot URDFs, target object, initial pose, and action type (pick/place).
2. **Conditional Scene Sampler**: Samples plausible object/robot poses by solving constraints on supporting surfaces. The plane $\pi_s$ maximizing $Area(U_s \cap proj_{o,s}(U_o))/Area(U_o)$ is selected, subject to mean surface proximity and angular alignment thresholds.
3. **Goal Configuration Generator**: Proposes a set of candidate 6-DOF end-effector poses using an SE(3) energy-based model. Adaptive KD-tree–based sampling iteratively computes “feasibility scores” to locate a pose with a feasible collision-free whole-body plan.
4. **VKC Problem Generator**: Treats the robot system as a Virtual Kinematic Chain, optimizing the continuous trajectory $q(\cdot)$ over time interval $[0, T]$:
   $$
   \min J(q(\cdot)) = \int_0^T [ w_1\|\dot{q}(t)\|^2 + w_2\|p_{ee}(q(t)) - p_{goal}\|^2 ] dt + w_3\|q(T) - q_{goal}\|^2
   $$
   subject to joint limits, collision avoidance ($dist(S_{robot}(q(t)), S_{scene}) \ge \epsilon$), and end-effector goal conditions. Sequential convex optimization (TrajOpt-style) yields final, smooth joint-space waypoints.

## 4. Evaluation Metrics and Protocols

All evaluation is conducted in Isaac Sim, simulating realistic contacts and friction. Used metrics:
- **Task Success Rate:** The primary metric; requires successful pick/place and stable holding of goal pose for ≥2 seconds.
- **Auxiliary metrics:**
  - Distance to goal: $\min_t \|p_{ee}(q(t)) - p_{goal}\|$
  - Collision rates: percentage of waypoints with robot–environment or self-collision.
  - Joint-limit violation: fraction of waypoints with $q(t)$ outside $[q_{min}, q_{max}]$.
  - Solve time: average planning/forward-pass compute time per trajectory.

## 5. Baseline Models and Performance Gaps

Three baseline approaches are systematically evaluated:
- **M3Bench-Planner ("modmp")**: Hybrid of VKC motion planning, SE(3) grasp prediction, and placement heuristics. Achieves ≈20% pick success, ≈2.8% place success (Base Test), degrades modestly in Novel splits, zero joint-limit/self-collision violations, but is slow (20–30s/plan).
- **Motion Policy Net (MPNet↑)**: Learning-based extension of MPNet with SDF collision losses. Yields ≈0.07% pick, ≈0.8% place success; high collision/joint-limit violations; fast inference time (<1s).
- **Mobile-Skill Transformer (MPTF)**: Decision transformer with PointNet++ scan encoder; nearly 0% pick success, 0.15–0.25% place, high environmental collision rates.
- Failure analysis demonstrates critical inability of learning systems to respect geometric constraints or coordinate base-arm motions. Even advanced hybrid planners exhibit low success and high computational latency, with sensitivity to grasp/placement proposal quality.

## 6. Open Challenges and Research Directions

The current state of the art reveals significant unsolved problems:
- Bridging sim-to-real gaps, including sensor noise and actuation uncertainty.
- Transitioning from discrete pick/place to temporally-extended, multi-object rearrangement.
- Richer natural language instruction understanding/generation.
- Data-efficient, tightly scene-aware models integrating perception, affordance reasoning, and continuous control.
- Dynamic manipulation and soft-contact modeling for heavy/articulated objects.
- Expanding to diverse robot morphologies and multi-robot scenarios.

By releasing 30,000 demonstrations, full scene annotations, and the M3BenchMaker pipeline, 3MDBench (Robotics) establishes a critical platform for benchmarking and advancing whole-body mobile manipulation in complex, real-world 3D environments [2410.06678].


---

*For unrelated instances of “3MDBench” in the literature, including Medical 3MDBench [2504.13861] and M3DBench for multimodal 3D instruction following [2312.10763], see the corresponding literature for domain-specific details and protocols. This entry details the robotics benchmark as per community usage and the cited primary source.*

Source: https://www.emergentmind.com/topics/3mdbench