INSPATIO-WORLD: Interactive 4D World Model

• INSPATIO-WORLD is a world modeling paradigm that integrates spatial and temporal dynamics to enable explorable, persistent 4D simulations from a single video input.
• It employs spatiotemporal autoregressive modeling via the STAR architecture, combining an implicit cache with explicit spatial constraints to ensure coherent scene evolution.
• The approach achieves interactive frame rates and long-horizon exploration, supporting applications such as panoramic navigation, urban simulation, and autonomous driving research.

INSPATIO-WORLD denotes a line of world-modeling research that treats the environment as a navigable spatiotemporal system rather than a sequence of disconnected images, and it also names a specific real-time 4D world simulator that reconstructs and generates dynamic interactive scenes from a single monocular reference video through spatiotemporal autoregressive modeling (Team et al., 8 Apr 2026). Across the associated literature, the defining objective is not merely next-frame prediction, but long-horizon roaming with camera control, spatial persistence, and controllable scene evolution; related systems extend the same agenda to panoramic exploration, frame-based rendering, explicit 3D memory, remote sensing extrapolation, and geometry-first urban navigation (Yin et al., 29 Sep 2025, Team et al., 12 Mar 2026, Wang et al., 1 Oct 2025, Lin et al., 2 Jun 2026).

1. Definition and conceptual scope

In this research context, an INSPATIO-WORLD system is a world model in which state, action, and observation are jointly grounded in space and time. The emphasis is on environments that can be explored, revisited, and queried under user or agent control, rather than on passive video continuation. This orientation is explicit in STRIDE and TARDIS, where the environment is represented as a graph of road-connected places with observation, state, and action tokens; the model learns relations among egocentric views, positional coordinates, movement commands, month, and year within a single autoregressive sequence (Carrión et al., 12 Jun 2025). A closely related robotics formulation appears in predictive world modeling from partial observations, where the goal is to infer complete plausible world states from accumulated but incomplete agent-centric evidence rather than to synthesize appearance alone (Karlsson et al., 2023).

This framing distinguishes INSPATIO-WORLD from ordinary controllable video generation. PanoWorld-X states the distinction directly: a digital world should behave more like the real world, where the scene changes continuously as the observer moves, and this requires a fully explorable $360^\circ$ panoramic world that is both high-fidelity and controllable by camera or exploration routes (Yin et al., 29 Sep 2025). The specific INSPATIO-WORLD simulator extends the same principle to monocular reference-video-driven 4D roaming, with explicit claims of long-horizon interaction, camera control, spatial persistence, and photorealistic rendering from a single input video (Team et al., 8 Apr 2026).

A recurrent misconception is to equate this family with generic text-to-video or view-conditioned rerendering. The surveyed systems instead couple generative modeling to persistent scene state, geometric constraints, action-conditioned transitions, or explicit memory. This suggests that the defining property is not the rendering modality, but the treatment of the environment as an explorable state space whose evolution must remain spatially coherent over time.

2. Canonical INSPATIO-WORLD architecture

The canonical realization of the term is the real-time 4D simulator introduced in "INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling" (Team et al., 8 Apr 2026). Its core is the Spatiotemporal Autoregressive, or STAR, architecture, which performs chunk-wise latent video generation conditioned on historical latent context, reference-video guidance, and explicit geometric constraints. The generative factorization is given as

$$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$

with denoising of each chunk expressed as

$$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$

Two tightly coupled components define the architecture. The Implicit Spatiotemporal Cache aggregates reference frames from the source video and historical generated chunks. The retrieved reference latent $\mathbf{z}^{\text{ref}}_i$ serves as a globally stable anchor, while the previous generated latent is stored in a sliding window, producing what the paper describes as a coupled long-and-short-range memory mechanism. The design also introduces a position index fixing strategy to stabilize long-horizon autoregressive generation when RoPE position indices would otherwise drift under long rollouts (Team et al., 8 Apr 2026).

The Explicit Spatial Constraint Module converts user interactions into explicit geometric controls. User inputs such as rotation, translation, and perspective shift are mapped into a $6$-DoF relative pose transform $\Delta \mathbf{T}_i$, accumulated into the global pose $\mathbf{T}_i$, and then combined with depth and intrinsics from Feed-Forward Reconstruction. The warped guidance and valid-pixel mask are computed by

$$\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i = \mathrm{Proj} \bigl( \mathbf{z}^{\text{ref}} \mid \mathrm{FFR}(\mathbf{z}^{\text{ref}}), \mathbf{T}_i \bigr).$$

This module is explicitly intended to enforce deterministic spatial structure, physically plausible camera motion, and stronger 3D consistency.

A third component, Joint Distribution Matching Distillation (JDMD), addresses the synthetic-to-real gap. JDMD alternates between controllable video rerendering distillation and text-to-video distillation, using a synthetic-data teacher for motion control and the original Wan-T2V foundation model as a real-data teacher for appearance. The overall objective is

$$L_{\text{JDMD}} = L_{\text{vis}} + \lambda_{\text{ctrl}} L_{\text{ctrl}}.$$

The deployed system uses Wan2.1 as the backbone, specifically a $1.3$B real-time model, replaces the original VAE with Tiny-VAE, uses torch.compile, and reports 24 FPS on an H-series NVIDIA GPU and about 10 FPS on an RTX 4090 (Team et al., 8 Apr 2026).

3. Representational families within the broader research line

The broader INSPATIO-WORLD literature does not rely on a single representation. Instead, it spans chunked video latents, independent frame synthesis, panoramic video tokens, explicit 3D point-cloud memory, and geometry-only visibility fields. The common thread is spatially grounded controllability.

System	Primary representation	Distinctive mechanism
INSPATIO-WORLD	Chunk-wise latent video from a monocular reference video	STAR, implicit cache, explicit spatial constraints, JDMD
InSpatio-WorldFM	Independent target frames conditioned on reference image and target pose	Explicit 3D anchors, implicit memory, PRoPE, 2-step DMD
PanoWorld-X	Equirectangular panoramic video with route conditioning	Sphere-Aware DiT, exploration-aware attention, PanoExplorer
EvoWorld	Panoramic video plus explicit colored point-cloud memory	Evolving 3D memory and geometric reprojection
3D Isovist World Model	Spherical visibility-depth map of urban negative space	Residual next-isovist prediction and persistent latent BEV map

InSpatio-WorldFM is the clearest alternative architectural branch. Rather than sequential video generation, it adopts a frame-based paradigm in which each target frame is generated independently from a reference image, reference pose, target pose, and a point-cloud rendering at the target viewpoint. Its condition set is written as

$$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$0

The model uses explicit 3D anchors to preserve coarse geometry, implicit spatial memory from the reference frame to preserve fine appearance, a self-attention-only transformer, and PRoPE to inject camera geometry directly into attention. It is distilled into a 2-step real-time generator and reports ~10 FPS at 512×512 on a single NVIDIA A100, 7 FPS on RTX 4090 with single-step inference, and ~50–70 ms interaction latency (Team et al., 12 Mar 2026).

A different branch replaces RGB appearance with pure geometry. The 3D isovist world model encodes the environment as a spherical visibility-depth map

$$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$1

with $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$2 elevation bins over $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$3, $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$4 azimuth bins over $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$5, and clipped range $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$6 m. It predicts the next isovist via a residual decoder,

$$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$7

and augments temporal modeling with a persistent latent BEV spatial map anchored in world coordinates (Lin et al., 2 Jun 2026). This geometry-first formulation is a deliberate departure from appearance-first world models.

4. Memory, control, and spatial persistence

Persistent world behavior is the central technical challenge across the literature. PanoWorld-X approaches it through panoramic route control and spherical geometry. It introduces the PanoExplorer dataset with 116,759 panoramic videos, each paired with a 3D exploration route, generated in Unreal Engine from 504 high-fidelity scenes. Camera trajectories are built from walkable surfaces using Delaunay triangulation, Dijkstra’s algorithm, Laplacian smoothing, trajectory filtering shorter than 18 meters, collision detection, and motion normalization with a 10 cm inter-frame distance. The model itself uses an exploration-control branch that converts a $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$8-DoF route signal into dense pixel-level conditioning via Plücker embeddings, and a sphere-aware branch that builds attention masks from great-circle distance on the sphere rather than Euclidean distance in the flattened equirectangular image (Yin et al., 29 Sep 2025).

EvoWorld addresses the same persistence problem by alternating generation and reconstruction. Starting from a single panorama $$p(\mathbf{Z}_{1:I} \mid \mathbf{C}_{\text{ref}}, \mathcal{T}) = \prod_{i=1}^{I} p(\mathbf{z}_i \mid \mathbf{z}_{<i}, \mathbf{c}^{\text{ref}}_i, \tau_i),$$9, it recursively renders geometric cues from explicit memory, generates the next panoramic clip, and folds the new evidence back into memory:

$$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$0

Its explicit memory is a colored point cloud reconstructed with VGGT, updated incrementally, and used through a locality-aware retrieve-and-reproject strategy that caps the number of frames below 100 to keep memory usage constant over long sequences (Wang et al., 1 Oct 2025).

At inference time, persistent memory can also be managed without retraining. WorldKV treats the native KV cache of autoregressive video diffusion as world memory, stores evicted chunks in GPU or CPU memory, and retrieves scene-relevant chunks according to the current camera or action state:

$$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$1

It complements retrieval with World Compression, which uses key-key similarity to an anchor frame to retain the bottom $$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$2 of non-anchor tokens by similarity. In the main setting, a 3-frame chunk retains 25% of non-anchor tokens, compressing storage from roughly 3T tokens to 1.5T tokens and allowing about 2× more history under a fixed budget (Yi et al., 21 May 2026).

Taken together, these mechanisms show three distinct approaches to persistence: geometry-aware attention over panoramic fields, explicit 3D memory with reprojection, and training-free retrieval and compression of autoregressive latent caches. A plausible implication is that long-horizon world modeling increasingly depends on hybrid memory designs rather than on generation quality alone.

5. Benchmarks and empirical performance

The named INSPATIO-WORLD simulator is evaluated on WorldScore Benchmark, especially WorldScore-Dynamic, on RE10K-Long, and on camera-controlled rerendering datasets including OpenVid and Blender. It reports WorldScore Dynamic Overall: 68.72, 3D Consistency: 84.18, Motion Accuracy: 60.21, Smoothness: 71.91, Camera control: 81.51, Object: 71.63, Static Overall: 75.81, Photometric: 93.00, and Content: 54.50. On RE10K-Long, it reports FID: 42.68, FVD: 100.55, Rot: 2.8762, and Trans: 0.1398. On OpenVid, it reports Overall VBench: 0.8507, Rot: 1.6000, and Trans: 0.1240; on Blender, FID: 44.46, FVD: 110.11, Rot: 1.2386, and Trans: 0.0667. The paper states that it ranks first among real-time interactive methods on WorldScore-Dynamic and is the only world model on that leaderboard claimed to reach 24 FPS (Team et al., 8 Apr 2026).

Related systems use different benchmarks but converge on the same evaluation axes: fidelity, control, geometry, and long-horizon consistency. PanoWorld-X, built on CogVideoX-5B-I2V, fine-tuned for 49-frame videos and resized to 480 × 960, evaluates on 200 randomly selected panoramic videos and reports PSNR 19.34, SSIM 0.63, LPIPS 0.24, FID 28.01, FVD 467.18, with control metrics $$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$3 and $$\hat{\mathbf{z}}_i = \mathrm{Denoise}_\theta \bigl( \mathbf{z}_{i,\sigma} \mid \mathbf{z}_{<i}, \mathbf{z}^{\text{ref}}_i, [\mathbf{z}^{\text{warp}}_i, \mathbf{m}_i] \bigr).$$4 (Yin et al., 29 Sep 2025). EvoWorld introduces Spatial360, spanning Unity synthetic outdoor, UE5 synthetic outdoor, Habitat indoor scenes from HM3D and Matterport3D, and real-world outdoor captures from Insta360. On single-clip panoramic generation in Unity it reports FVD 106.81, LPIPS 0.167, PSNR 22.03, SSIM 0.826, MEt3R 0.0954, and AUC@30 0.8846, and in downstream tasks reports 93.3% target reaching with GPT-4o + EvoWorld versus 83.5% with GenEx, and 68.8% frame retrieval versus 50.5% (Wang et al., 1 Oct 2025).

The broader agenda also extends into non-RGB and non-panoramic settings. The 3D isovist model, trained city-blind on Manhattan and Paris, improves over copy-last on next-isovist prediction with MAE: 3.57 vs. 4.36 m, RMSE: 11.34 vs. 13.80 m, and Edge-F1: 0.719 vs. 0.689, while a logistic regression probe on its 256-D PathTransformer latent reaches 89.3% ± 3.0% city classification accuracy, above pooled-pixel and global-statistic baselines (Lin et al., 2 Jun 2026). In autonomy, WPT shows that world-model reasoning can be distilled into a real-time student policy, reporting 0.11 collision rate in open-loop and 79.23 driving score in closed-loop, with the student running at 64 ms latency versus 312 ms for the teacher with reward model, or about 4.9× faster (Jiang et al., 25 Nov 2025).

6. Limitations, misconceptions, and open directions

The literature is explicit that current systems do not yet constitute unrestricted, fully general world simulators. The original INSPATIO-WORLD notes two major limitations: it does not yet perfectly preserve fine-grained textural details of newly generated regions in long-term memory, and it still struggles with seamless 360-degree dynamic roaming, especially under wide-angle viewpoint changes where dynamic elements must remain multi-view consistent (Team et al., 8 Apr 2026). InSpatio-WorldFM identifies a different trade-off: dynamic content remains difficult, offline memory generation from multi-view or panoramic expansion is computationally heavy, and the frame-based paradigm can exhibit visual instability or jitter between consecutive frames (Team et al., 12 Mar 2026).

Panoramic systems surface complementary constraints. PanoWorld-X is explicit that it does not yet support long video generation, and that its current interface accepts only exploration routes, not richer interactive user commands; it is therefore described as a strong step toward explorable panoramic world generation, but not a full interactive world simulator (Yin et al., 29 Sep 2025). EvoWorld improves long-range exploration through explicit 3D memory, yet its own framing implies that video-only generation without geometric state eventually drifts on long or looping trajectories, which is precisely the failure mode it aims to mitigate (Wang et al., 1 Oct 2025).

Memory systems remain bounded by the underlying generator. WorldKV cannot fix fundamental visual artifacts in the pretrained backbone, and very long rollouts can still accumulate autoregressive error even when retrieval and compression are used (Yi et al., 21 May 2026). Geometry-first models are not exempt: the 3D isovist world model studies only two cities, includes a height-imputation confound for Paris, uses a lossy nearest-surface representation, and does not address dynamic objects or sim-to-real sensor issues (Lin et al., 2 Jun 2026).

Taken together, these limitations suggest that the central open problems are not only fidelity and speed, but also the joint treatment of memory, geometry, action, and interaction at scale. The trajectory of the field points toward richer semantic memory, stronger geometric-texture coupling, more expressive user or agent interfaces, and better handling of loop closure, dynamic content, and long-horizon revisitation.