FlowMaps: Multimodal Object Relocalization

• FlowMaps is a latent flow matching model that estimates multimodal distributions over future object locations in dynamic 3D environments.
• It integrates a variational autoencoder with conditional flow matching, enabling prediction of multiple plausible object futures based on past human interactions.
• FlowMaps has demonstrated superior benchmark performance and real-world robotic navigation by effectively modeling spatio-temporal object dynamics.

Searching arXiv for papers on “FlowMaps” and closely related work to ground the article. FlowMaps is a latent flow matching model for estimating multimodal distributions over the future locations of dynamic objects in a continuous 3D space. It was introduced for robotic reasoning in changing household environments, where human interaction induces spatio-temporally consistent object-location patterns that are not captured by static scene understanding alone. In this formulation, FlowMaps learns implicit dependencies among objects and their temporal evolution, predicts likely changes in object locations conditioned on past human interactions, and supports generalization across previously unseen environments that share similar object routines (Argenziano et al., 18 Jun 2026).

1. Problem setting and conceptual scope

FlowMaps addresses a setting in which a robot must reason jointly about spatial layout and temporal change. The motivating difficulty is not merely object detection, but object persistence under human-induced motion: objects that were previously observed can later appear in different places, and the relocation process is neither arbitrary nor strictly deterministic. The target quantity is therefore a multimodal distribution over future object locations rather than a single best estimate.

In the 3D household formulation, the scene at time $\tau$ is represented as

$M_\tau = (O_\tau, O_{\text{BG}})$

where $O_\tau$ denotes the current dynamic objects and $O_{\text{BG}}$ the static background. ProcTHOR is used to generate long-term dynamics with three distinct human-like behavioral routines, described as location preferences, evenly balanced routine, and highly dynamic behavior (Argenziano et al., 18 Jun 2026). This makes the prediction problem explicitly distributional: the same queried object may plausibly appear in several future locations.

A related FlowMaps formulation targets dynamic object relocalization in changing environments using a semantic map, a text-object query, and a prediction interval, with the aim of inferring multimodal object locations over space and time (Argenziano et al., 19 Sep 2025). Taken together, these formulations establish FlowMaps as a model family for long-term object relocalization under habitual human activity, rather than as a static receptacle-ranking heuristic.

2. Generative formulation in latent space

The core generative mechanism combines a variational autoencoder with latent flow matching. The VAE provides compact latent representations of object tokens, while the flow model learns a conditional continuous-time transport from a Gaussian base distribution to the latent distribution of future object states. In the 3D model, the flow is written as an ODE

$$\frac{d}{dt}\psi_t(x) = u_t(\psi_t(x)), \qquad \psi_0(x_0) = x_0$$

with a time-dependent velocity field $u_t$ in latent space (Argenziano et al., 18 Jun 2026).

Training uses Conditional Flow Matching. The objective is given as

$$\mathcal{L}_{CFM}(\theta) = \mathbb{E}_{t, x, y} \left[ \| u_t^\theta(x) - u_t^{\text{target}}(x \mid y) \|_2^2 \right],$$

with expectations over $t \sim U[0,1]$, data $y \sim p_{\text{data}}$, and samples $x \sim p_t(\cdot \mid y)$ (Argenziano et al., 18 Jun 2026). In the latent variant, data $M_\tau = (O_\tau, O_{\text{BG}})$0 are first encoded to $M_\tau = (O_\tau, O_{\text{BG}})$1, the flow transports $M_\tau = (O_\tau, O_{\text{BG}})$2 to $M_\tau = (O_\tau, O_{\text{BG}})$3, and the decoder reconstructs the predicted future object state.

This latent construction is important because the model is not framed as direct regression of future coordinates. Instead, it estimates a conditional generative distribution over object futures. Multiple samples from $M_\tau = (O_\tau, O_{\text{BG}})$4 yield multiple plausible futures, which is the operational source of multimodality in FlowMaps (Argenziano et al., 18 Jun 2026).

3. Object, scene, and conditioning architecture

FlowMaps is organized as a two-part deep generative model composed of a VAE and a latent flow matching network parameterized by a Transformer-based flow network, specifically a Conditional Diffusion Transformer (CDiT) (Argenziano et al., 18 Jun 2026). The object token given to the VAE consists of a 3D bounding box $M_\tau = (O_\tau, O_{\text{BG}})$5 together with a label $M_\tau = (O_\tau, O_{\text{BG}})$6. The decoder has separate heads for bounding box regression and class label prediction, and the VAE is trained with a standard loss with linearly annealed KL term to prevent posterior collapse.

The contextual scene representation is built by a map encoder that treats scenes as sets rather than sequences. Background and dynamic entities are pooled together. Each token combines an object type indicator, a bounding box embedding with structure given as translation, size magnitude, and direction, a class embedding, and a time embedding. The encoder is a stack of $M_\tau = (O_\tau, O_{\text{BG}})$7 Transformer encoder layers with permutation-invariant encoding, allowing variable-length scenes (Argenziano et al., 18 Jun 2026).

The flow block then advances only the queried object latent while conditioning on the rest of the scene. Its attention is asymmetric: the query object latent is the only state updated by the flow, and the context is injected through cross-attention. The conditioning vector combines normalized learned embeddings of flow time $M_\tau = (O_\tau, O_{\text{BG}})$8, target prediction time $M_\tau = (O_\tau, O_{\text{BG}})$9, and query object label $O_\tau$0, with channel modulation performed through adaLN-Zero (Argenziano et al., 18 Jun 2026). A central architectural consequence is that FlowMaps learns object-to-object and object-to-scene dependencies without explicit scene graphs.

4. Training procedure and sampling pipeline

Training is carried out with a CFM loss on interpolated source-target latent pairs, and the pairing is improved by mini-batch optimal transport. The stated effect of this pairing is improved sample path straightness and integration efficiency (Argenziano et al., 18 Jun 2026). This is a structural choice rather than a post hoc refinement: the transport geometry is shaped during training, not only during inference.

At inference time, the map encoder computes the scene context once for a given observation $O_\tau$1, query label $O_\tau$2, and prediction time $O_\tau$3. For each sample, the model draws $O_\tau$4, integrates the learned ODE with Euler steps to obtain $O_\tau$5, and decodes $O_\tau$6 into a predicted future bounding box $O_\tau$7 (Argenziano et al., 18 Jun 2026). Repeating this procedure produces a multimodal posterior over continuous object locations.

The related relocalization-oriented FlowMaps formulation follows the same high-level pattern: a map encoder produces context, a CDiT block learns the vector field along the flow-matching path, and inference integrates the learned ODE from isotropic Gaussian noise conditioned on the current map, the query object, and the prediction interval (Argenziano et al., 19 Sep 2025). This continuity across formulations indicates that the defining property of FlowMaps is the use of conditional flow matching for multimodal future-location inference, not a single fixed input modality.

5. Benchmarks, navigation performance, and real-world deployment

The principal 3D evaluation uses large-scale dynamic scenes generated with ProcTHOR: 2,706 training and 918 validation environments per habit, with 41 object classes, 17 receptacle categories, and 4 weeks of simulated time sampled at hourly intervals, yielding $O_\tau$8 steps per scene (Argenziano et al., 18 Jun 2026). Distributional evaluation uses minFDE@K, Recall@1, Coverage & Density, and TV/JS divergence. The baselines include FreqPrior, LLMPrior, EmpiricalMean, and MeanFlowMaps.

On this benchmark, FlowMaps achieves the lowest minFDE, reported as $O_\tau$9 m and $O_{\text{BG}}$0 better than the best baseline, together with nearly perfect coverage, the highest density, and lower TV/JS divergences (Argenziano et al., 18 Jun 2026). The comparison with EmpiricalMean and MeanFlowMaps is particularly diagnostic: collapsing the predictor to a single-point estimate produces markedly worse results, which directly supports the claim that multimodality is essential for this task.

The downstream Object Navigation evaluation uses 25 validation environments, 5 objects, and 5 $O_{\text{BG}}$1 pairs per object, totaling more than 600 episodes. Metrics include SR@K, SPL@K, mean SR/SPL, and mean path length or steps. Baselines span zero-shot VLM- and LLM-based methods, probabilistic planners using object-location likelihoods, and scene-graph or GNN-based temporal models, including TAP-LGX, OSG, CEG, SGM, HOMER, and a Naive LLM (Argenziano et al., 18 Jun 2026).

Across these episodes, FlowMaps outperforms state-of-the-art approaches and consistently achieves the highest mean SR and SPL. The reported interpretation is that SR@1 is often better than the SR@5 of competing methods, indicating sharper ranking of likely destinations (Argenziano et al., 18 Jun 2026). The model is also deployed on a real TIAGo mobile robot, where it generates location proposals after time has elapsed and the robot visits high-probability locations one by one until the target object is found. In this sense, FlowMaps is not only a predictive model but also a search prior for dynamic object navigation.

6. Terminological ambiguity and relation to other “flow map” research

A recurrent source of confusion is that “FlowMaps” in robotics is not synonymous with the broader literature on flow maps. In computer vision, “flowmaps” can denote optical-flow-derived fields and blending masks reused across resolutions, as in low-resolution flow and mask upsampling for efficient space-time video super-resolution (Dutta et al., 2021). In computational fluid dynamics, Neural Flow Maps compute long-term, bidirectional flow maps and their Jacobians from a neural velocity buffer to support low-dissipation incompressible flow simulation (Deng et al., 2023), while Particle Flow Map methods use Lagrangian particles as samples of bidirectional flow maps for accurate long-range advection (Zhou et al., 2024).

Other uses are equally domain-specific. Trajectory Flow Map represents city-wide aggregated traffic flows as a sequence of time-evolving weighted directed graphs and applies GraphScope to detect change-points in mobility regimes (Kim et al., 2022). In reinforcement learning, flow map policies parameterize start-to-end jumps across generative policy dynamics and support optimal Q-guidance under a trust-region formulation (Ziakas et al., 12 May 2026). In discrete generative modeling, Categorical Flow Maps define continuous trajectories on the simplex for few-step generation of images, molecular graphs, and text (Roos et al., 12 Feb 2026), and cumulative flow maps have been proposed as a unified few-step framework for diffusion- and flow-based generation (Li et al., 5 May 2026).

This suggests that the shared term marks a family resemblance rather than a single method: across robotics, graphics, fluids, vision, traffic analysis, and generative modeling, a “flow map” is generally a finite-time transport operator or representation of state evolution. FlowMaps, in the robotics sense, is the specialization of that transport viewpoint to multimodal object dynamics and long-term relocalization in changing 3D environments (Argenziano et al., 18 Jun 2026).