InPlan3D: Indoor Task Planning Benchmark

• InPlan3D is a benchmark for indoor 3D task planning that converts detailed scene descriptions into grounded, multi-step action plans for service robots.
• It leverages the Text-Scene pipeline with ScanNet data, integrating object attributes, spatial relations, and BEV images to enrich planning evaluation.
• The evaluation employs multi-turn reasoning with task-level and step-level accuracy metrics, emphasizing the need for explicit, language-based scene representation.

InPlan3D is a benchmark for grounded long-horizon task planning in indoor 3D scenes, introduced alongside the scene-to-language framework Text-Scene in “Text-Scene: A Scene-to-Language Parsing Framework for 3D Scene Understanding” (Li et al., 20 Sep 2025). It is defined as a “comprehensive benchmark for 3D task planning,” and more specifically as a benchmark “to evaluate the model’s capability in indoor task planning based on ScanNet.” The benchmark contains 3,174 long-term planning tasks across 636 indoor scenes and is framed from the perspective of an indoor service robot, with the goal of testing whether multimodal LLMs can convert grounded scene understanding into feasible multi-step plans rather than only perform recognition-style 3D reasoning.

1. Benchmark scope and conceptual role

InPlan3D occupies a distinct role within the Text-Scene paper. Text-Scene is the scene-to-language representation framework; InPlan3D is the evaluation suite used to test whether models can reason over a 3D indoor environment and generate grounded, step-by-step plans. The paper explicitly contrasts this benchmark with prior 3D scene-language benchmarks such as 3D question answering, visual grounding, and dense captioning, which it characterizes as mainly focusing on specific tasks in a single-turn dialogue format. By contrast, InPlan3D introduces multi-task and multi-turn reasoning dialogue, requiring simultaneous scene understanding and action-sequence generation (Li et al., 20 Sep 2025).

The benchmark is organized around three linked abilities emphasized in the paper’s Figure 1 and surrounding discussion: problem definition, object retrieval, and action planning. This framing positions InPlan3D as a benchmark for embodied reasoning rather than for isolated perceptual competence. In the paper’s account, embodied AI in 3D scenes must handle object attributes, affordances, layouts, spatial relations, and physical context; the benchmark is therefore intended to test whether current MLLMs can operationalize those factors for long-horizon planning.

A useful clarification is that InPlan3D is not a generic “3D planning” term. It is distinct from “Planner3D,” which is a scene graph-conditioned indoor 3D scene synthesis method with joint layout-shape generation and explicit layout regularization, rather than an embodied planning benchmark (Wei et al., 2024).

2. Construction pipeline and dependence on Text-Scene

InPlan3D is tightly coupled to the Text-Scene representation pipeline. The benchmark construction prompt shown in the appendix uses System Prompt, Scene Information, and Rendered BEV Images, and the output is further refined to ensure conformance to a predefined format. The benchmark was constructed with GPT-4o from ScanNet scenes, using textualized scene information together with rendered BEV images (Li et al., 20 Sep 2025).

The underlying scene-information pipeline is object-centric. Text-Scene first identifies objects and geometric properties from 3D reconstructions and instance masks, computes spatial relations, and generates language descriptions. It produces three kinds of textual content: a system message specifying structure, an object-caption section describing object attributes, and a relationship-generation section that converts scene-graph triplets of the form $\{\text{obj}_1,\text{obj}_2,\text{rel}\}$ into natural language. A self-reflection module then refines this textual scene information. InPlan3D uses these outputs directly: generated scene descriptions, object attributes, spatial relationships, and scene-level summaries form the promptable substrate from which planning tasks are generated.

The preprocessing stack is inherited largely from Text-Scene rather than introduced as an independent annotation protocol. The scenes come from the point clouds provided by ScanNet. Instance labels are obtained by default from Mask3D predictions. Object captions are produced by projecting 3D boxes into multi-view images, captioning crops with BLIP-2, ranking candidate captions with CLIP similarity, selecting the top 10, and refining them with GPT-4o. Spatial relationships are then computed from geometry, camera-view directional reasoning, and semantic priors, and serialized into language. This implies that InPlan3D depends materially on the fidelity of scene parsing: planning quality is coupled to the quality of captioning, instance labeling, and relation extraction.

The most formal descriptions available for this benchmark are inherited from Text-Scene’s geometric parsing pipeline. The paper gives the scene reconstruction expression

$$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$

where $\mathcal{P}$ is the reconstructed point cloud, $\bm{I}_i$ are sampled images, $\bm{E}_i$ are camera extrinsics, and $\mathcal{R}$ is the reconstruction function. For relation parsing it gives the Euclidean distance

$d = \sqrt{(x_1-x_2)^2+(y_1-y_2)^2+(z_1-z_2)^2}.$

These are not formal definitions of InPlan3D itself, but they specify the geometric substrate from which the benchmark’s planning context is derived.

3. Task structure and input-output format

Each InPlan3D instance contains a high-level instruction and a ground-truth step-by-step low-level action sequence. The appendix states that “Each task contains a concise high-level instruction followed by a step-by-step breakdown of low-level actions.” The task horizon is described as long-term, and the clearest quantitative characterization in the appendix is that most tasks contain 4 to 6 steps and 30 to 60 words in total (Li et al., 20 Sep 2025).

The target format is natural-language planning rather than symbolic action sequences or single-step policy outputs. The paper specifies three stylistic constraints on the reference plans. First, every step begins with an action verb. Second, actions refer to specific objects using semantic and structural references such as “the central conference table” and “[table-0].” Third, many steps include spatial or contextual qualifiers such as “beside the desk.” This design makes object grounding explicit at the linguistic level.

The input format varies by evaluated system. In the reported text-only InPlan3D setting, the model receives only textual scene descriptions derived from scene parsing. In the full Text-Scene setting, those textual descriptions are augmented with one BEV image. Across baselines, input modalities differ more broadly: PQ3D, LEO, and ChatScene use point and vision encoders; GPT4Scene uses vision without a point encoder; Text-Scene text-only uses neither point nor vision input at inference beyond its text; full Text-Scene uses one BEV image but no point encoder.

The paper also states that InPlan3D involves multi-turn reasoning dialogue, although the exact dialogue schema is not specified in the visible text. This suggests a conversational interaction format rather than a single prompt-response exchange, but the precise turn structure remains underdocumented.

4. Evaluation protocol and metric structure

InPlan3D evaluates both planning correctness and language similarity. The planning-specific metrics are $G_\text{Acc}$ and $T_\text{Acc}$. Table 2 identifies $G_\text{Acc}$ as task-level accuracy and $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$0 as step-level accuracy. In addition, the benchmark reports standard text-generation scores: METEOR, ROUGE, BLEU-4, and CIDEr (Li et al., 20 Sep 2025).

The evaluation is therefore reference-based and automatic. The visible paper text does not describe simulator execution, robot rollouts, human preference studies, or LLM-as-judge evaluation for plan validity. It also does not mention exact match for full plans. Instead, it measures plan quality through task-level and step-level correctness together with language-overlap metrics.

A notable formal limitation is metric opacity. The paper states that more calculation details of $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$1 and $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$2 are shown in Appendix S4, but those formulas are not present in the supplied text. Accordingly, there is no explicit formal objective, notation, or scoring formula for InPlan3D beyond the metric names and their high-level descriptions. For a benchmark aimed at long-horizon embodied reasoning, this absence constrains reproducibility analysis and makes fine-grained metric interpretation less transparent than the benchmark scale and task design might suggest.

5. Reported empirical results

The main benchmark results appear in Table 2. Text-Scene achieves the best reported performance, with $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$3, $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$4, METEOR 15.04, ROUGE 44.96, BLEU-4 19.87, and CIDEr 80.17. The text-only version is already strong, reaching $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$5 and $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$6. The strongest non-Text-Scene baseline is GPT4Scene, with $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$7 and $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$8, plus lower language-similarity scores across the board. Among the remaining baselines, performance decreases from GPT4Scene to ChatScene, LEO, and PQ3D (Li et al., 20 Sep 2025).

These results support one of the paper’s central findings: a compact textual representation of 3D scene structure can outperform more vision-heavy baselines on long-horizon planning, and adding a single BEV image improves performance further. The paper interprets this as evidence that scene grounding and language representation quality matter more than simply scaling visual input volume.

The absolute scores also delimit the current difficulty of the benchmark. The best task-level accuracy remains below 50%, whereas step-level accuracy is materially higher. This suggests that current MLLMs often produce locally plausible or partially correct action sequences without achieving full-task correctness. The paper links this to the multi-component nature of embodied planning, which requires coordinated object retrieval, grounded scene understanding, subgoal ordering, and spatial reasoning.

Two indirect ablation findings from Text-Scene help explain why the benchmark favors explicit scene language. In the “Relationship Generation” ablation, raw coordinates perform poorly, simple directional labels improve results, and “Complex” relational language performs best, with SQA3D EM increasing from 16.5 to 39.6 to 61.2. A separate sentence-selection study shows that selecting relevant scene information improves downstream reasoning while reducing token cost and inference time. These ablations are not benchmarked directly on InPlan3D, but they suggest why relationally explicit, compact scene text is advantageous for grounded plan generation.

6. Limitations, interpretation, and research context

Several limitations are explicit or strongly implied in the paper. First, InPlan3D is restricted to indoor scenes from ScanNet. It therefore inherits a real scanned indoor distribution rather than a synthetic world model, but it does not cover outdoor, dynamic, or richly manipulable simulation settings. Second, benchmark construction relies on generated textual descriptions and LLM-assisted task generation. This means that failures in object labeling, captioning, or relation extraction can propagate into both task creation and model evaluation (Li et al., 20 Sep 2025).

Third, the benchmark evaluates textual plan quality rather than executable robot behavior. Good scores do not by themselves establish physical executability in a real or simulated environment. Fourth, the visible paper text does not include the exact formulas for $$\mathcal{P}=\mathcal{R}(\{(\bm{I}_i,\bm{E}_i)\}_{i=1}^N),$$9 and $\mathcal{P}$0, nor explicit split counts, nor a detailed human verification protocol for the generated tasks. Fifth, the task distribution is characterized statistically but not taxonomically: the paper does not provide a category breakdown such as retrieval, cleaning, navigation, or manipulation.

In broader context, InPlan3D belongs to a line of research that treats indoor spatial structure as a first-class representation problem, but its emphasis is planning rather than reconstruction or synthesis. Floorplan-oriented systems such as FloorNet reconstruct 2D vector floorplans from RGB-D scans (1804.00090), and Plan2Scene converts floorplans plus sparse photos into textured architectural shells (Vidanapathirana et al., 2021). By contrast, InPlan3D asks whether a model can generate a grounded, multi-step plan from a compact representation of a real indoor 3D scene. This suggests a shift from perception or visualization benchmarks toward evaluation of embodied reasoning over explicitly represented scene structure.

The benchmark’s main significance lies in making that shift measurable. It moves beyond perception-only 3D benchmarks by testing whether MLLMs can turn scene understanding into coherent action plans. Its main empirical conclusion is correspondingly specific: explicit textual decomposition of 3D scenes into object attributes and spatial relations is highly competitive for grounded long-horizon planning, and may be a particularly effective interface between 3D perception and embodied reasoning.