Physical AI Spatial Intelligence Warehouse

• Physical AI Spatial Intelligence Warehouse is a curated ecosystem integrating embodied 3D sensors, spatial grounding, and AI-driven planning for efficient, real-world warehouse operations.
• It leverages modular architectures that combine RGB-D perception, route planning, and deterministic evaluators to transform raw sensor data into actionable spatial intelligence.
• Benchmarks and asset banks, such as SpatialBench and SSI-Bench, offer measurable metrics like S1 scores and task success rates to validate system performance in complex warehouse environments.

Physical AI Spatial Intelligence Warehouse denotes a class of infrastructures for embodied spatial reasoning in warehouse-like physical environments and, in a broader repository sense, a curated stack of datasets, benchmarks, models, assets, and evaluation protocols for such reasoning. Across recent work, the central requirement is spatial grounding: metric understanding of 3D geometry, object relations, free space, dynamics, and action consequences, rather than symbolic grounding alone. In warehouse settings, this grounding is instantiated through RGB-D or 3D perception, shelf and bin topology, region-level object reasoning, route and manipulation planning, safety constraints, and interfaces to warehouse management systems and fleet control (Felicia et al., 2 Feb 2026, Bousetouane, 15 Jan 2025).

1. Conceptual scope and scientific framing

Physical AI in warehouses extends domain-specific cognitive systems into embodied operation. One line of work describes a modular three-block architecture composed of perception, cognition, and actuation, with a Physical Retrieval-Augmented Generation pattern that binds an industry-specific LLM to live physical context such as robot pose, maps, shelf topology, task queues, safety zones, human presence, battery state, and congestion maps (Bousetouane, 15 Jan 2025). A related survey frames the same problem at a higher level as the conjunction of spatial perception, structured representation, memory, planning, and learning, organized along Capability–Task–Scale axes. In that formulation, warehouse systems sit at the intersection of navigation, manipulation, mapping, scene understanding, and multi-agent coordination, typically at micro–meso scales (Felicia et al., 2 Feb 2026).

A recurring distinction in the literature is between symbolic grounding and spatial grounding. Symbolic grounding associates images with words and categories; spatial grounding requires explicit understanding of poses, distances, free space, contact, uncertainty, and the geometric consequences of action. Several works state this difference directly, arguing that perception or captioning alone does not confer agency, and that language-only reasoning remains insufficient for collision-free planning, manipulation feasibility, or long-horizon execution in physical environments (Felicia et al., 2 Feb 2026, Bousetouane, 15 Jan 2025).

Spatial cognition is also increasingly treated as hierarchical. SpatialBench decomposes spatial intelligence into five progressive levels: Observation, Topology and relation, Symbolic reasoning, Causality, and Planning. This hierarchy is especially relevant to warehouse settings, where object counting or distance estimation is only the first layer; higher competence requires relational understanding across shelving and aisles, rule-like interpretation of markers and affordances, causal prediction under motion or occlusion, and executable route or action sequencing (Xu et al., 26 Nov 2025).

2. Data resources, benchmarks, and asset banks

Recent work has produced several complementary resources that together approximate a spatial-intelligence warehouse: warehouse QA datasets, egocentric video benchmarks, constrained-geometry tests, interactive simulators, scale-aware 3D asset banks, and lifted 3D scene corpora.

Resource	Modality and focus	Reported scale
Physical AI Spatial Intelligence Warehouse benchmark	RGB-D, masks, QA for warehouse spatial reasoning	~95,000 RGB-D pairs; ~499,000 train QA; ~1,900 val; ~19,000 test (Muturi et al., 13 Oct 2025)
SpatialBench	Egocentric RGB video + synchronized 3D LiDAR	1,347 QA pairs across 50 first-person videos (Xu et al., 26 Nov 2025)
SSI-Bench	Real-world structural images for constrained-manifold reasoning	1,000 ranking questions; 1,160 unique images (Yang et al., 8 Feb 2026)
SpatialWorld	Vision-only interactive multimodal-agent benchmark	760 tasks across eight backends (Gao et al., 8 Jun 2026)
AmaraSpatial-10K	Deployment-oriented 3D asset bank	over 10,000 synthetic 3D assets (Salehi et al., 24 Apr 2026)
COCO-3D / Objects365-v2-3D	Single-view 2D-to-3D lifted datasets	117,183 train + 4,951 val scenes; ~2M scenes (Miao et al., 24 Jul 2025)

Warehouse-centered QA benchmarks are organized around four recurring task categories: Distance Estimation, Object Counting, Multi-choice Grounding, and Spatial Relation Inference. One such dataset uses paired RGB-D inputs, segmentation masks, region IDs, and normalized single-word or numeric answers, with scenes rendered in NVIDIA Omniverse/IsaacSim and characterized by clutter, occlusion, varied scales, and dense shelving and buffer layouts (Muturi et al., 13 Oct 2025). A separate agent-oriented system was evaluated on a closely related benchmark with approximately 499K training question–answer pairs, 1.9K validation questions, and 19K test questions (Huang et al., 14 Jul 2025).

Beyond warehouse QA, broader spatial-intelligence repositories emphasize explicit shortcut suppression and geometry-aware curation. SSI-Bench was built by ten researchers who spent over 400 hours curating images, annotating structural components, and designing questions to minimize pixel-level cues; it includes geometric, topological, and multi-view tasks over complex engineering structures, and was explicitly designed so that correct answers are not recoverable from simple 2D rankings (Yang et al., 8 Feb 2026). SpatialBench complements this with synchronized LiDAR-grounded egocentric videos, while SpatialWorld extends evaluation into active, simulator-agnostic interaction under vision-only partial observability (Xu et al., 26 Nov 2025, Gao et al., 8 Jun 2026).

The repository notion also includes 3D asset banks and scalable scene construction pipelines. AmaraSpatial-10K releases each asset as a metric-scaled, semantically anchored .glb with separated PBR material maps, a convex collision hull, a paired reference image, and multi-sentence metadata under a unified spatial convention of +X forward and +Z up; 91.4% of generated candidates pass its automated quality gates (Salehi et al., 24 Apr 2026). In parallel, 2D-to-3D lifting pipelines combine relative depth, metric depth, scale calibration, camera intrinsics, and extrinsics to generate depth maps, pseudo-RGBD, point clouds, and 3D annotations from single-view imagery, yielding COCO-3D and Objects365-v2-3D as large-scale sources for 3D perception and reasoning pretraining (Miao et al., 24 Jul 2025).

3. Representations and model architectures

Warehouse-oriented spatial models increasingly rely on explicit spatial tokens rather than latent caption-style scene summaries. A prompt-guided RGB-D framework fine-tunes a vision-language transformer on warehouse tasks by injecting bounding box coordinates into the prompt, using a SpatialBot visual backbone built on a SigLIP image encoder and a lightweight Phi-2 LLM; the training responses append a normalized answer string so that outputs match the evaluation interface (Muturi et al., 13 Oct 2025). In the same task family, SmolRGPT keeps RGB and depth separate up to language-space fusion, uses SigLIP2-base-patch16-256 with a SmolLM2-360M backbone, and inserts region embeddings into the token stream through <mask_rgb> and <mask_depth> placeholders. Its region representation is explicitly mask-pooled:

$$E(M, F) = \frac{1}{\sum_{x,y} M_{x,y}} \sum_{x,y} M_{x,y} F_{x,y,:}.$$

This design makes relation queries operate over designated regions rather than over undifferentiated scene tokens (Traore et al., 18 Sep 2025).

A contrasting architecture treats the LLM as a planner over tools. The SpatialAgent system uses Gemini 2.5-Flash as a frozen core and exposes callable functions such as dist(obj_1, obj_2), inside(obj_A, obj_B), left_of(obj_A, obj_B), and right_of(obj_A, obj_B). Distance is predicted by a trained 5-channel ResNet-50 over RGB plus two masks; inclusion is handled by a separate binary classifier; left/right uses centroid comparison in the image plane. Questions are preprocessed into object-aware identifiers like <buffer_0> or <transporter_1>, so the LLM reasons over explicit symbolic bindings and API outputs rather than over raw image-language embeddings alone (Huang et al., 14 Jul 2025).

Long-horizon warehouse autonomy has pushed research toward persistent spatial memory. BSC-Nav organizes memory into landmarks, route knowledge, and survey knowledge. Its landmark memory stores tuples $L_k = \{c_k, \theta_k, \rho_k, T_k\}$; its cognitive map projects DINOv2 patch features through depth and pose into an allocentric voxel grid; and its working memory retrieves candidate goals by combining confidence or similarity with travel distance:

$$H_i = \lambda p_i + (1-\lambda)\left(1 - \frac{d_i}{d_{\max}}\right).$$

The resulting plans are executed through a global planner such as A* and a local planner such as TEB, with explicit support for warehouse tasks such as picking, placing, and docking (Ruan et al., 24 Aug 2025).

At a more general level, surveys of spatial AI agents identify three architectural regularities: hierarchical memory for long-horizon tasks, GNN–LLM integration for structured spatial reasoning, and world models for safe deployment across scales (Felicia et al., 2 Feb 2026). A plausible implication is that a mature spatial-intelligence warehouse is less a single model than a compositional stack: explicit geometry encoders, structured map memory, language-mediated task orchestration, and planners or controllers that remain physically grounded at execution time.

4. Evaluation protocols, reported performance, and recurrent failure modes

Evaluation in this area is deliberately heterogeneous, because different benchmarks target different slices of spatial competence. The warehouse QA challenge aggregates success across task categories under an S1 final score, also described as a Weighted Average Success Rate. For quantitative tasks, relative error is defined as

$RE = \frac{|\hat{y} - y|}{y} \times 100\%,$

and a 10% tolerance criterion is used in the reported setup; categorical tasks use exact-match accuracy (Muturi et al., 13 Oct 2025). SpatialWorld instead formalizes interaction as vision-only POMDP control and reports Task Success Rate

$$TSR = \frac{1}{N}\sum_{i=1}^{N}\mathbf{1}[\mathcal{V}_i(s_T^{(i)}) = 1],$$

together with Step Efficiency relative to human reference trajectories (Gao et al., 8 Jun 2026).

On the warehouse benchmark itself, specialized systems have reported strong scores. Prompt-guided SpatialBot reaches a final S1 score of 73.0606 and places 4th overall on the public leaderboard (Muturi et al., 13 Oct 2025). SmolRGPT reports an S1 score of 90.68 and 3rd place overall, with 99.80% left-right accuracy, 92.76% counting accuracy, 88.02% multi-choice grounding accuracy, and 82.13% distance accuracy (Traore et al., 18 Sep 2025). SpatialAgent reports 95.8638% test accuracy and 1st place on the 2025 AI City Challenge Track 3 leaderboard (Huang et al., 14 Jul 2025). These numbers show that warehouse-specific tool use, coordinate prompting, and region-level RGB-D modeling can be highly effective under a fixed task protocol.

The same literature also shows that such performance does not imply general spatial competence. SSI-Bench evaluates 31 widely used VLMs on constrained-manifold structural reasoning; the best open-source model reaches 22.2% Taskwise Accuracy, the strongest closed-source model reaches 33.6%, and humans score 91.6%. The benchmark further reports that encouraging models to think yields only marginal gains, with error analysis attributing failure to weak structural grounding and constraint-consistent 3D reasoning (Yang et al., 8 Feb 2026). SpatialWorld makes a similar point in interactive settings: across 760 tasks and eight backends, GPT-5 reaches only 17.4% average TSR and the leading open-source model, Qwen-3.5, reaches 14.1%, with marked mismatches between task success and execution efficiency (Gao et al., 8 Jun 2026).

Hierarchical evaluation reinforces the same pattern. SpatialBench reports strong perceptual grounding at lower cognitive levels but clear degradation in symbolic reasoning, causal inference, and planning; Gemini-2.5-pro attains Ave. 71.86 and Score 75.79, while humans reach Human Level Ave. 93.89 and Score 96.40 (Xu et al., 26 Nov 2025). Common failure modes recur across benchmarks: occlusion and re-identification errors, misestimation of member extent or object size, confusion between camera-centric and agent-centric frames, over-reliance on 2D projected heuristics, long-horizon drift, and premature termination in interactive tasks (Yang et al., 8 Feb 2026, Gao et al., 8 Jun 2026, Xu et al., 26 Nov 2025).

5. Warehouse operations, planning, and systems integration

In deployment-oriented work, the spatial-intelligence warehouse is coupled directly to operational warehouse software and safety infrastructure. The Physical AI Agent architecture described for warehouse robotics combines 2D or 3D LiDAR, RGB-D or stereo cameras, IMU, wheel encoders, RFID or barcode readers, and optional short-range safety sensors; its cognition layer integrates SLAM, semantic detection, occupancy mapping, human and item tracking, planners, memory, and an LLM connected to enterprise data through Ph-RAG (Bousetouane, 15 Jan 2025). The data flow is explicit: sensors feed synchronization and calibration, then perception and mapping, then cognition and tool calls, then actuation and safety supervision.

Warehouse planning operates at several levels. For mobile autonomy, global path planning is typically performed on occupancy grids or topological graphs with A* or D* Lite, while local avoidance uses DWA or TEB, and manipulation uses MoveIt-compatible motion planning plus grasp synthesis and pose refinement (Bousetouane, 15 Jan 2025, Ruan et al., 24 Aug 2025). For facility design, automated warehouse layout generation treats the warehouse as a discrete grid and searches over aisle-carving operations with constrained beam search. Feasible layouts satisfy conditions such as aisle reachability from doors, minimum clearances, and access constraints; they are then scored by

$$\mathrm{Score} = \alpha T_s + \beta T_{pf} + c_1 T_o,$$

with storage capacity, pick faces, and orientation diversity as the core terms, and a connectivity score used for tie-breaking (Shahroudnejad et al., 2024). This links spatial intelligence not only to navigation within a warehouse, but also to the design of the warehouse topology itself.

At the representation level, warehouse systems increasingly use structured map or grid schemas. SIG introduces a 10×10 bird’s-eye-view Spatial Intelligence Grid, serialized as JSON with keys such as vehicles, traffic_lanes, traffic_signs, traffic_lights, and self, and paired with graph-based relation metrics and optional human-gaze weighting (Wu et al., 24 Oct 2025). Although developed in an autonomous-driving pilot, the same idea of explicit grid-grounded scene structure is directly compatible with warehouse labeling, where rack slots, pallets, lanes, signs, and moving agents can be encoded as typed objects under physical constraints. This suggests a practical convergence between task-specific warehouse QA, simulator state representations, and deployment telemetry.

6. Limitations, misconceptions, and likely development trajectories

A common misconception is that stronger language modeling or longer chain-of-thought is sufficient for spatial intelligence. Multiple benchmarks argue against this. SSI-Bench was explicitly designed to prevent 2D shortcut exploitation and reports only marginal gains from “thinking” prompts (Yang et al., 8 Feb 2026). SpatialImaginer goes further, arguing that text-only reasoning can abstract away the very geometric detail needed for multi-step spatial consistency; in its ablations, textual CoT harms geometry-heavy tasks, whereas interleaved visual imagination improves robustness across six spatial benchmarks (Li et al., 19 Apr 2026). The broader implication is that warehouse-scale physical intelligence is unlikely to emerge from text-only deliberation without explicit geometric state tracking.

Another limitation concerns supervision and verification. SpatialEvo identifies model-consensus self-training as unreliable for geometry-heavy reasoning and replaces it with a Deterministic Geometric Environment that computes ground truth exactly from point clouds and camera poses. Its DGE formalizes 16 task categories and provides zero-noise interactive oracles, showing that self-evolving training can be grounded in deterministic geometry rather than in pseudo-label agreement (Li et al., 15 Apr 2026). This is particularly relevant to warehouse repositories, where asset banks, maps, masks, point clouds, and simulator states can in principle provide executable ground-truth validators for placement, visibility, distance, or route feasibility.

A third open direction is how to encode intelligence in space itself. One recent proposal replaces explicit collision checking with a learned Riemannian metric field over the configuration manifold, such that action reduces to geodesic following under $G(s)=\exp(H(s))$. In that view, “the metric is the obstacle,” and navigation intelligence is embedded directly into spatial geometry (Xu, 17 Jun 2026). A plausible implication is that future warehouse systems may combine symbolic task orchestration with geometry-native planners whose cost structures are learned from layouts, obstacles, and traffic patterns rather than manually engineered.

Across surveys and benchmark papers, the long-term agenda is consistent: unified cross-scale spatial representations, stronger uncertainty handling, interactive evaluation beyond passive VQA, safer sim-to-real transfer, multi-agent coordination, and efficient edge deployment (Felicia et al., 2 Feb 2026, Gao et al., 8 Jun 2026). In repository terms, a mature Physical AI Spatial Intelligence Warehouse would therefore include at least four layers: physically grounded datasets and asset banks; deterministic evaluators and benchmark suites; memory- and world-model-based agents with explicit geometry interfaces; and deployment schemas linking maps, tasks, safety states, and enterprise systems. The literature suggests that only this full stack, rather than any single benchmark or model family, is sufficient for reliable spatial intelligence in real warehouse operations.