FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching

Abstract: Joint spatial and temporal understanding of 3D scenes is a crucial requirement for robots deployed in everyday household environments. Such agents must not only comprehend and navigate spatial layouts, but also reason about how these spaces evolve over time. In particular, humans interact with objects daily, causing them to change position throughout the environment and making it difficult for robots to reliably associate current observations with previously seen objects. However, these interactions are not random: human habits and routines induce spatio-temporally consistent patterns in object locations, which robotic agents can potentially learn and then exploit for downstream tasks such as navigation. To this end, we introduce FlowMaps, a latent flow matching model for estimating multimodal distributions over the future locations of dynamic objects in a continuous 3D space. By learning the implicit dependencies among objects and their temporal evolution, FlowMaps predicts likely changes in object locations conditioned on past human interactions, while supporting generalization across previously unseen environments that share similar object routines. To demonstrate the utility of this method, we deploy FlowMaps in a downstream dynamic Object Navigation task in both simulated and real-world environments. Across more than 600 episodes, FlowMaps outperforms state-of-the-art approaches, showing that modeling object dynamics through continuous, multimodal spatio-temporal distributions improves robotic search and navigation in changing household environments. Code and additional material is available at https://fra-tsuna.github.io/flowmaps/.

• The paper introduces a latent flow matching architecture that predicts multimodal 3D object trajectories in indoor environments.
• It integrates a VAE with a conditional diffusion transformer to capture spatial and temporal dependencies.
• Empirical evaluations reveal significant improvements in distributional fidelity and navigation success over existing baselines.

Modeling Multimodal 3D Object Dynamics via Flow Matching: An Expert Summary of FlowMaps

---

Motivation and Problem Setting

The modeling of temporally and spatially evolving 3D household environments is critical for embodied agents tasked with search and navigation. Traditional approaches to object relocation predict discrete receptacle-level states, overlooking the multimodal, continuous distributions induced by human routines. The "FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching" (2606.20209) introduces a latent flow matching architecture that recovers, for each object query, the conditional distribution over future locations in continuous 3D space, directly leveraging learned temporal dependencies.

---

Methodology: Flow Matching with Latent CDiT

The FlowMaps architecture fuses a VAE with a conditional diffusion transformer (CDiT) backbone for continuous multimodal distribution modeling. Object bounding box geometry and semantics are encoded via a VAE, exposing a compact latent space for probabilistic transport. The CDiT block stack parametrizes a velocity field transporting a standard Gaussian latent to the latent encoding of the queried object's future bounding box, conditioned on scene context, query label, and prediction horizon.

Figure 1: Training and inference pipeline for FlowMaps, showing VAE-based encoding and CDiT-based flow transport for multimodal posterior sampling.

The approach is instantiated in large-scale, highly dynamic indoor environments generated with ProcTHOR, capturing semantically consistent, multi-week object trajectories across diverse household layouts and human habits.

---

Representation and Conditional Modeling

Each environment snapshot encodes a set of dynamic and static object tokens, embedding 3D geometry, semantic label, and type. The map encoder, based on permutation-invariant Transformers, aggregates scene context with self-attention. Query time, object label, and flow integration step are fused for adaLN-zero conditioning in CDiT blocks. Sampling is performed by integrating the learned vector field from Gaussian noise to the target latent in fixed steps, with each forward run producing a plausible multimodal hypothesis.

---

Empirical Evaluation: Distributional and ObjNav Metrics

The evaluation quantifies both the distributional fidelity and practical navigation benefit of FlowMaps. In joint accuracy and coverage metrics, FlowMaps attains minFDE $0.342$ m (7.6% lower than the strongest baseline), mode coverage $>99.9\%$, and Naeem density $0.886$, a 37% gain over FreqPrior. The total variation and Jensen-Shannon divergences (TV $0.829$, JS $0.482$) are reduced, demonstrating improved spatial mass allocation relative to ground-truth distributions. The model consistently outperforms single-point regression, mean collapse, and semantic frequency baselines, clearly substantiating the need for multimodal generators.

Figure 2: Overview of 30 diverse ProcTHOR validation environments used for robustness and generalization testing.

In dynamic object navigation (ObjNav), FlowMaps achieves mean success rate (mSR) $65.86\%$ and mean SPL $46.66\%$, with SR@1 exceeding SR@5 of competing methods. The model manifests stronger early-mode ranking and substantially fewer step-budget failures compared to GNN and scene-graph baselines.

Figure 3: Success/failure decomposition for ObjNav, with FlowMaps concentrating more first-mode successes and reducing navigation-induced failures.

---

Qualitative Case Studies

Visualization of successful episodes reveals FlowMaps' concentration of probability mass on correct future receptacles and the spatial efficiency of ranked mode proposals. For difficult behavioral routines, FlowMaps achieves robust generalization, enabling multi-modal search and efficient retrieval in both simulated and real-world environments.

Figure 4: Representative FlowMaps episode showing predicted samples, ranked clusters, executed search trajectory, and target confirmation.

---

Limitations and Implications

While FlowMaps is not open-vocabulary, and currently limited to 41 object classes and 17 receptacles, the method demonstrates strong cross-environment generalization with offline training—proving practical for robotic deployment in continuously varying indoor settings. The architecture benefits from offline-generated trajectory data but may require adaptation for more heterogeneous contexts or real-world online learning.

Theoretically, FlowMaps extends flow matching beyond action policy learning to posterior inference for geometric scene predictions—a novel use-case for FM in robotics and generative modeling.

---

Future Prospects

Open questions remain regarding open-vocabulary scaling, sample efficiency for online and interactive learning, and habit diversity. Further, flow matching for posterior inference in open-world scenarios, integration with vision-language-action models, and continual adaptation to novel routines represent directions with substantial impact for embodied AI planning and generalization.

---

Conclusion

FlowMaps establishes a robust latent flow matching methodology for continuous, multimodal object dynamics, yielding improved distributional fidelity and navigation performance in dynamic 3D scenes. The approach, combining geometric and semantic embedding with efficient conditional CDiT flow transport, is validated in simulated and real-world robotic contexts. It materially advances prior state-of-the-art in long-term object modeling, demonstrating effective transfer and practical viability for dynamic object search in everyday environments.

Figure 5: Additional representative successful ObjNav episode illustrating multimodal hypothesis generation and ranked trajectory execution.

Figure 6: Highlight of further robust episode outcomes with FlowMaps, showing trajectory diversity and effective search.

Figure 7: Visualization of high-confidence multimodal predictions and corresponding navigation efficiency in challenging scenes.

Figure 8: Continuous spatial distribution modeling and efficient retrieval paths with FlowMaps across habit variations.

Figure 9: Cross-habit generalization and accurate mode localization by FlowMaps in evolving ProcTHOR environments.

Figure 10: FlowMaps supports practical robotic navigation with precise ranked location predictions and adaptive search.

Figure 11: Demonstration of FlowMaps' ability to maintain localization fidelity and spatial efficiency in diverse real-world-like scenarios.

---