Object-Centric Mapping

• Object-centric mapping is a paradigm that decomposes complex environments into per-object representations for enhanced spatial and semantic reasoning.
• It leverages techniques like canonical coordinate reparameterization, volumetric segmentation, and robust place recognition to improve localization and mapping.
• It supports language-driven, cross-modal reasoning and scalable process mapping in dynamic, multi-object settings.

Object-centric mapping is a paradigm in spatial and semantic world modeling that decomposes complex environments into explicit, per-object representations—structuring map data, inference, and reasoning at the object or object-relation level, as opposed to purely point-, grid-, or scene-centric schemes. This approach enables compositionality, semantic grounding, and improved robustness across robotics, computer vision, process mining, and language-driven perception domains. Object-centric mapping subsumes a range of techniques: canonical 3D lifting for reconstruction, volumetric segment fusion, slot-based factorization for perception and reasoning, semantic instance-level occupancy grids, explicit object-relation modeling, and robust geometric localization tied to object coordinates.

1. Canonical Object-Centric Coordinates and Representations

A central tenet of object-centric mapping is the reparameterization of world structure and observation into object-local (rather than world-fixed or robot-fixed) frames. In interactive robot learning, it is demonstrably more effective to transform both robot actuation and observation data into each object’s local coordinate frame. This ensures that data obtained before and after object motion remains directly comparable, even as objects themselves are manipulated and displaced. Bayesian non-parametric models such as Gaussian processes fitted in these local, task-relevant object frames maintain data consistency and scalability throughout online interaction, in contrast to world-centric models which are sensitive to object displacements (Shinde et al., 2023).

In multi-view 3D reconstruction from images, object-centric mapping involves predicting a per-pixel mapping $f_\theta(I) = C$, where $C[u]\in\mathbb{R}^3$ determines each pixel’s position on a learned canonical 3D object surface. Crucially, this approach avoids reliance on externally estimated camera extrinsics, with the network responsible for resolving both viewpoint ambiguity and object symmetries, thereby enabling fusion and aggregation directly in the canonical object frame (Tulsiani et al., 2020).

2. Object-Centric Fusion, Segmentation, and Volumetric Mapping

A major trajectory in object-centric mapping is the shift from dense, independent occupancy grids to object-clustered, semantically fused volumetric representations. Classical grid-based occupancy mapping assumes cells are independent and updates occupancy using only direct measurements—limiting the system’s adaptivity to dynamic, multi-cell objects and resulting in “ghost” artifacts after object motion. Object-centric occupancy extensions introduce latent variables $\mathcal{D}_t$ to capture dependencies between all cells belonging to the same object, enabling joint updates and fast clearing of dynamic or occluded entities. Clustering via semantic segmentation informs the assignment of cells to object-centric groups, with intra-object update dependencies parameterized via membership strengths $\delta^i$ and region-growing or Pearson $\chi^2$ statistical consistency (Pekkanen et al., 2023).

Semantic object segmentation (using methods such as eSAM or language-image models) at the pixel or voxel level further enables feature aggregation into object-level volumetric submaps. Systems such as FindAnything form volumetric submaps $M_i$ with occupancy log-odds at the voxel scale, while representing instance segments by feature vectors $\bar f_k$ (e.g., CLIP embedding) and counts $N_k$, resulting in an efficient, object-centric, open-vocabulary geometry–semantics association (Laina et al., 11 Apr 2025).

3. Robust Correspondence, Place Recognition, and Localization

Object-centric mapping frameworks demonstrate pronounced benefits in structural place recognition and robust pose estimation. By anchoring descriptors to spatially prominent, semantically static objects (e.g., lamp posts, poles, traffic signs), and forming local descriptors (Object Scan Context, OSC) centered at each object’s centroid with canonical (e.g., 2D polar) coordinate frames, place recognition can robustly accommodate large viewpoint changes. Pairwise matching leverages column-shifted matrix comparisons and yields closed-form 3-DOF relative pose estimates $(\Delta x, \Delta y, \Delta \theta)$ using only object positions and orientation correction (Yuan et al., 2022). This contactless and deterministic approach surpasses LIDAR- or vehicle-centric methods on large-scale benchmarks such as KITTI Odometry and KITTI360, with best-in-class precision-recall and pose errors as low as 0.148 m, 0.168 m, and 1.248$^{\circ}$.

In room-scale mapping, Rooms from Motion (RfM) adopts explicit 3D box primitives as object-centric geometric anchors, removing reliance on keypoint-based SfM. Through learned object-level (CuTR, LightGlue) and corner-level correspondence, RfM estimates global camera poses, merges object detections into consistent tracks, and parameterizes scenes with a sparse, scalable set of oriented boxes. Bundle adjustment then optimizes both box geometry and pose, yielding state-of-the-art detection and localization accuracy scaling linearly with the number of objects (Lazarow et al., 29 May 2025).

4. Semantically Grounded, Language-Driven, and Open-Vocabulary Object Maps

Object-centric mapping explicitly supports semantic grounding and cross-modal reasoning. Volumetric submaps aggregate vision-language features for per-segment open-vocabulary semantic indexing. Natural language queries are mapped, via embedding similarity, to segments in the map and thus to associated 3D geometry, supporting zero-shot instance retrieval and language-driven robot exploration (Laina et al., 11 Apr 2025). Closed-set evaluations show that aggregation at the segment (object) level, rather than voxelwise, achieves state-of-the-art semantic accuracy and efficient scaling to resource-constrained platforms.

Language-mediated object-centric learning interleaves slot-based unsupervised object discovery (e.g., Slot Attention, MONet) with neuro-symbolic language executors, aligning per-object slot representations $C[u]\in\mathbb{R}^3$0 with learned concept embeddings $C[u]\in\mathbb{R}^3$1 via cosine similarity and objectness prediction. Joint optimization of reconstruction, segmentation, and language alignment losses reliably grounds perception in symbolic semantics, increasing ARI metrics and consistently improving downstream referring expression comprehension and symbolically-instructed tasks (Wang et al., 2020).

5. Temporal Consistency and Object Identity in Video Object-Centric Mapping

A persistent challenge in video-based object-centric mapping is temporal slot permutation—object-to-slot assignments changing unpredictably across frames, undermining downstream consistency. The Conditional Autoregressive Slot Attention (CA-SA) framework imposes temporal regularity through per-slot autoregressive GRU priors and an Objects Permutation Consistency (OPC) loss, which encourages each slot’s spatial attention to align temporally. This factorized prior over slot indices $C[u]\in\mathbb{R}^3$2 ensures identity-preserving trajectories in slot space, resulting in more stable slot identities and improved performance in video prediction and visual question-answering (VQA). On standard video benchmarks (CLEVRER, Physion), CA-SA consistently improves ARI, FG-ARI, mIoU, and downstream VQA metrics over baselines (Meo et al., 2024).

6. Object-Centric Process Mapping and Synchronized Interaction Graphs

In process mining, object-centric event logs and Petri net models (OCPN, OPID) explicate the evolution and interrelation of multiple object instances in complex workflows. OCPNs succinctly track tokens per object type and execution position, but lack explicit relationship tracking (e.g., synchronization constraints between object types). The mapping from OCPNs to OPIDs formalizes object identifiers and stable many-to-one relationships $C[u]\in\mathbb{R}^3$3, introducing explicit link places, creation transitions, and synchronized token flows. The extension guarantees conformance: only executions that respect the intended relationships are accepted. The mapping is polynomial time with strict model–execution equivalence, allowing robust conformance checking and violation detection in systems with intricate inter-object dependencies (Seidel et al., 18 Aug 2025).

7. Implementation, Scalability, and Limitations

Contemporary object-centric mapping systems are characterized by modularity and scalability. Memory and computational requirements are often proportional to the number of maintained objects or segments, not total geometric or pixel/voxel count. For example, FindAnything achieves real-time, open-vocabulary object-centric mapping on resource-limited drones by decoupling voxel occupancy and segment features in submap-local frames, supporting on-board aggregation and drift correction. However, limitations remain: such systems depend on accurate object segmentation and labeling, may underperform in object-sparse or highly dynamic scenes, and can face degraded recall if semantically stable objects are absent (Laina et al., 11 Apr 2025, Yuan et al., 2022, Lazarow et al., 29 May 2025).

A plausible implication is that future advances will center on improving object discovery, scaling to a broader diversity of object types and relationships, and addressing scenarios with limited object richness or frequent topology changes. Extensions to full 6-DOF geometric reasoning, improved dynamic object filtering, temporally-varying and many-to-many relationship modeling in process nets, and integration with foundation models for robust semantic alignment are ongoing research directions.