WildCity: Urban-Scale AI Artifacts

• WildCity is a multifaceted term representing urban AI systems designed for sparse visual navigation, feed-forward 3D Gaussian Splatting, and simulation-based reconstruction.
• It encompasses a navigation benchmark that tests mapless routing with intersection-level images, a synthetic dataset that trains appearance-consistent 3D models, and a real-world testbed derived from autonomous fleet logs.
• The projects emphasize extracting stable spatial structures from noisy, incomplete data over long-horizon tasks, highlighting practical challenges in urban-scale AI applications.

WildCity is a research name that, in recent arXiv literature, denotes three distinct urban-scale AI artifacts rather than a single unified benchmark. In "City Navigation in the Wild: Exploring Emergent Navigation from Web-Scale Knowledge in MLLMs" (Dalal et al., 17 Dec 2025), it refers to a city-navigation framework centered on Sparsely Grounded Visual Navigation, the CityNav benchmark, and Verbalization of Path. In "Wild3R: Feed-Forward 3D Gaussian Splatting from Unconstrained Sparse Photo Collection" (Furutani et al., 10 Jun 2026), WildCity is a large-scale synthetic dataset for training feed-forward 3D Gaussian Splatting under multi-view, multi-illumination, and transient-rich conditions. In "WildCity: A Real-World City-Scale Testbed for Rendering, Simulation, and Spatial Intelligence" (Han et al., 7 Jul 2026), the term denotes a real-world multimodal dataset and simulator derived from autonomous-fleet logs. Across these uses, the common denominator is urban environments in the wild, but the operative goals differ: sequential navigation, appearance-consistent reconstruction, and simulation-ready digital twins.

1. Disambiguation and conceptual scope

The term "WildCity" currently spans three technical lineages with different data regimes, task definitions, and evaluation protocols.

WildCity usage	Primary objective	Reported scale
City navigation benchmark and prompting framework (Dalal et al., 17 Dec 2025)	Evaluate and improve MLLM-based mapless navigation	Four cities, 400 tasks
Synthetic 3DGS training dataset (Furutani et al., 10 Jun 2026)	Train feed-forward 3DGS on unconstrained sparse photo collections	200 scenes, 337,500 images
Real-world city-scale testbed and simulator (Han et al., 7 Jul 2026)	Urban reconstruction, rendering, and closed-loop simulation	18 trajectories averaging 83.7 km, total 1,507.1 km

This naming overlap is substantive rather than superficial. Each project operationalizes "wild" differently. In the navigation setting, "wild" means the absence of environmental annotations, maps, GPS, or explicit route instructions, forcing self-localization and planning from sparse visual evidence. In the synthetic reconstruction setting, "wild" refers to unconstrained photo collections with lighting diversity and transient objects. In the real-world city-scale testbed, it denotes long, noisy fleet logs containing dynamic objects, illumination changes, motion blur, and imperfect poses. This suggests that the shared label functions less as a standard benchmark family than as a recurring design principle for urban AI under realistic observational constraints.

2. WildCity for sparsely grounded visual navigation

In (Dalal et al., 17 Dec 2025), WildCity is defined through Sparsely Grounded Visual Navigation (SGVN), a task intended to evaluate whether an agent can navigate real-world cities relying only on visual observations at intersections and internal multimodal knowledge, without environmental annotations, maps, GPS, or language instructions that encode landmark names or positions. The city is formalized as an undirected graph $G=(V,E)$, where nodes are intersections and edges are street segments with length $\ell(e)>0$. At time $t$, the state is $s_t=v_t\in V$, actions are the incident edges $\mathcal{A}(v_t)$, and the observation consists only of per-edge images at intersections, $o_t=\{I_e \mid e\in \mathcal{A}(v_t)\}$. No observations are provided while traversing edges. The objective is zero-shot evaluation with success, SPL, and decision accuracy rather than training.

The instantiated benchmark, CityNav, covers four global cities selected for diversity in language, architecture, signage, and topology: New York, São Paulo, Tokyo, and Vienna. It uses Google Street View panographs, with intersection-only observations. For each candidate outgoing edge, the agent receives a single rectilinear image crop aligned to the edge’s heading. The benchmark contains 100 origin–destination pairs per city, for 400 tasks in total, with routes averaging about 2 km and more than 50 decision points. Reported city-level means per route are 1.8 km and 44 decisions for New York, 2.0 km and 55 decisions for São Paulo, 1.9 km and 80 decisions for Tokyo, and 2.1 km and 60 decisions for Vienna. Success is defined by entering a manually annotated destination polygon; SPL is computed as

$$\mathrm{SPL} = \frac{1}{N} \sum_{i=1}^N S_i \frac{L_i^*}{\max(L_i, L_i^*)};$$

decision accuracy is the percentage of decisions after which remaining walking distance strictly decreases.

The evaluated models are GPT-4o, GPT-4.1, Gemini-2.5 Flash, GPT-5, and Qwen-2.5VL-32B. Standard reasoning strategies include Chain-of-Thought and Self-Reflection. The action loop provides images tagged with option IDs and numeric headings, then requests the exact destination location, the current estimated exact location, walking directions from current location to destination, one option ID as the decision, and concise memory for the next step. Episodes are capped at max_decision_points=150 and max node transitions=2000, and explicit self-positioning is performed every third decision point to induce slight uncertainty.

The central intervention is Verbalization of Path (VoP), which probes the model’s latent cognitive map by requiring explicit statements of destination location, estimated current location, and walking directions before choosing an action. The paper represents the cognitive map at step $t$ as $M_t = \{ (L_i, \mathbf{d}_i, w_i) \}_{i=1}^k$, where $L_i$ are landmarks, $\ell(e)>0$0 are bearings or directions, and $\ell(e)>0$1 are confidences inferred from verbal cues. Action selection is then defined as

$\ell(e)>0$2

where $\ell(e)>0$3 scores alignment between visual evidence, verbalized directions, landmark cues, and heading consistency.

Empirically, the paper reports that current state-of-the-art MLLMs and standard reasoning techniques significantly underperform in this setting. For GPT-4.1 in New York, base prompting achieves 15% success, 0.097 SPL, and 42.27% decision accuracy; Chain-of-Thought achieves 21%, 0.173, and 44.59%; Self-Reflection with GPT-4.1 achieves 16%, 0.112, and 42.90%; Self-Reflection with a GPT-5 reflector achieves 22%, 0.168, and 48.14%; AgentNav with full VoP reaches 92% success, 0.557 SPL, and 75.27% decision accuracy. The ablation sequence on New York with GPT-4.1 further attributes gains to specific memory and grounding components: baseline 15%, + Markovian Memory 23%, + Decision History 29%, + Previous Visit Tracking 35%, + Partially Verbalized Path 66%, and full VoP 92%. Cross-model comparisons show consistent gains, though Tokyo, Vienna, and São Paulo remain substantially harder than New York.

The reported failure modes are localization failures in non-English signage and irregular topology, long-horizon planning breakdown with compounding errors and loops, and misrecognition of landmarks or facades combined with overreliance on initial wrong plans. In that sense, WildCity as SGVN is not only a navigation benchmark; it is also a stress test for the degree to which web-scale multimodal priors can be converted into persistent, self-correcting spatial behavior.

3. WildCity as a synthetic training corpus for feed-forward 3D Gaussian Splatting

In (Furutani et al., 10 Jun 2026), WildCity is a large-scale synthetic, scene-level dataset created to train feed-forward 3D Gaussian Splatting models for unconstrained sparse photo collections. Its motivating premise is that existing feed-forward 3DGS approaches underperform in real-world settings because training data rarely combines three ingredients simultaneously: multi-view coverage per scene, broad lighting diversity per viewpoint, and realistic transient-object variability. WildCity addresses this by offering urban outdoor scenes rendered under many HDRI-based illuminations and augmented with transient objects, so that models can learn appearance consistency across viewpoints conditioned on a reference view while suppressing transients in geometry and appearance.

The dataset comprises 200 distinct urban scenes across nine procedurally generated virtual cities. It uses 170 HDRI environment maps adapted from LightCity, with 30 lighting conditions randomly sampled per scene, and 50 views per scene, yielding 337,500 images in total. Transient augmentation is applied to 12.5% of rendered views, totaling 37,500 transient-augmented images. Scene diversity extends beyond the default SceneCity catalog through more than 130 Sketchfab assets, including temples, shrines, towers, and giant Buddha statues. Cameras are placed in a fan-shaped region centered on the target building, with heights 1–2 m, positions about 10–25 m from scene centers, and optical axes aimed within a rectangle around the scene center. Field of view is uniformly sampled between $\ell(e)>0$4 and $\ell(e)>0$5, with dynamic reduction when far cameras would otherwise see beyond the ground plane.

The acquisition pipeline uses SceneCity as the base, incorporates Sketchfab GLB assets with permissive licenses, and draws HDRI lighting from LightCity. Assets are normalized to match SceneCity’s 10 m grid, and nine distinct subsets of Sketchfab assets are integrated to create nine unique cities. Twenty to twenty-five target locations are sampled per city, producing the 200 scenes. Rendering is performed in Blender Cycles at $\ell(e)>0$6 resolution with 512 spp. The paper specifies a physically based decomposition

$\ell(e)>0$7

with diffuse and glossy terms separated into direct and indirect contributions. Transient objects are inserted in 2D via Gemini using text prompts for people, vehicles, signage, banners, and related content, and are intentionally not view-consistent across views. Ground-truth camera intrinsics, extrinsics, depth maps, and sky masks are available because the scenes are synthetic.

WildCity is used to train Wild3R, a camera-free, feed-forward 3DGS model built on AnySplat and VGGT. The model predicts anisotropic 3D Gaussian primitives, camera parameters, and depth maps directly from sparse, unconstrained images in a single forward pass, conditioned on a chosen reference view to enforce appearance consistency across viewpoints and illuminations. Training uses pretrained AnySplat weights; depth and camera heads are frozen; approximately 940M parameters remain learnable. The optimizer is AdamW for 30K iterations with cosine learning-rate scheduling, peak learning rate $\ell(e)>0$8, and 1K warmup, taking about one day on a single NVIDIA A100 (80 GB). Losses combine appearance consistency, LPIPS, depth supervision, and AnySplat’s Gaussian and camera losses.

The reported evaluation uses Photo Tourism and NeRF-OSR. On Photo Tourism with 16 context views in the camera-free setting, Wild3R achieves PSNR 15.87, SSIM 0.435, LPIPS 0.506, and about 0.95 s reconstruction time; AnySplat reports 13.72, 0.377, 0.546, and about 0.95 s; DA3 reports 14.03, 0.420, 0.586; YoNoSplat reports 13.04, 0.403, 0.640, and about 0.38 s. On NeRF-OSR with 16 views, Wild3R reports, for example, 13.27 PSNR, 0.388 SSIM, and 0.535 LPIPS on europa, and 12.22 PSNR, 0.401 SSIM, and 0.547 LPIPS on stjohann. Ablations indicate that fine-tuning AnySplat without Wild3R’s objectives improves over the base model but still trails Wild3R, that removing transient augmentation degrades performance, and that excluding Sketchfab assets harms generalization.

The limitations are explicitly tied to the synthetic design. The dataset focuses on urban outdoor scenes, excludes very dark night HDRIs, and inserts transients in 2D rather than as view-consistent geometry. It provides no explicit transient masks. The associated model lacks explicit material modeling, can fail when the reference view is heavily occluded by transients, and remains limited on very fine high-frequency detail relative to per-scene optimization. Even so, the dataset is positioned as the first training corpus sized and structured to teach feed-forward 3DGS models appearance-consistent in-the-wild reconstruction under joint multi-view, multi-illumination, and transient variation.

4. WildCity as a real-world city-scale testbed and simulator

In (Han et al., 7 Jul 2026), WildCity is a real-world, city-scale testbed that transforms long autonomous-fleet logs into multimodal data for urban reconstruction, photorealistic rendering, and closed-loop embodied simulation. The motivation is the absence of realistic city-scale street-view data that supports both reconstruction into digital twins and embodied evaluation over continuous long routes. Existing synthetic datasets offer control but inherit sim-to-real gaps, while real driving datasets are generally dominated by short clips rather than continuous coverage. WildCity is intended to operate at scales more comparable to human spatial memory, where agents must perceive, remember, and reason across tens of square kilometers.

The dataset contains 18 continuous trajectories across six U.S. cities: Atlanta, Arlington, Ann Arbor, Eden Prairie, Grand Rapids, and Miami. There are three logs per city, each averaging 83.7 km and about 2.5 hours, for a total traversed route length of 1,507.1 km. Per-city spatial coverage is about 40.18 km². The sensor suite includes a roof-mounted LiDAR, six surround-view RGB cameras, an IMU, and a GPS receiver. The cameras comprise three forward narrow-angle views and three lateral or rear wide-angle views; three are at $\ell(e)>0$9 resolution and three at $t$0, all at 10 Hz. LiDAR runs at 10 Hz with about 80k points per frame, downsampled to about 15k for reconstruction; GPS runs at 10 Hz and IMU at 100 Hz. The released data totals 3.01M keyframes, spatially subsampled at 0.5 m intervals. Each log is partitioned into contiguous 5 km chunks, and standardized sub-trajectories are provided at 50 m, 250 m, 500 m, 1 km, 2.5 km, and 5 km.

Processing begins by defining the released keyframe by the front-camera timestamp, associating other sensors by nearest timestamp, and interpolating ego poses from the localization stream. LiDAR points are deskewed by ego-pose interpolation. Ego poses are initialized from onboard SLAM and refined using GPS to improve global consistency and reduce drift, including loop closures when revisiting areas; all poses are then georegistered into a shared city-level coordinate system. Semantic masks for ground, sky, and potentially movable objects are generated using SAM3 with text prompts and refined with onboard 3D tracking cuboids so that only truly dynamic instances are masked, using the threshold $t$1. Quantitative validation on 100 hand-labeled images across four cities yields overall mIoU 91.58%, with dynamic 88.48%, sky 92.01%, and ground 94.24%.

The accompanying reconstruction baseline is an urban-tailored 3D Gaussian Splatting pipeline designed for long, narrow street-view trajectories with limited viewpoint overlap and sensor noise. It augments 3DGS with rig-aware pose optimization, sky modeling, ground regularization, depth supervision, multi-GPU scaling, and a post-repair extrapolation stage. Multi-camera rigidity is modeled through

$t$2

so ego poses and rig extrinsics are jointly optimized rather than each camera pose independently. Sky is handled through a lightweight view-dependent MLP composited as infinite background, and ground regularization constrains road geometry. Extrapolated-view repair progressively renders off-trajectory views, repairs them with Difix3D+, and re-inserts them as supervision. Depth supervision in disparity mode is weighted at 0.1. Densification begins at 10k steps, ends at 60k, occurs every 100 steps, and adds 0.5% of the current primitive count.

The system is explicitly scaled for multi-kilometer scenes. Reported Gaussian counts are 6M, 12M, 30M, and 60M for 0.5 km, 1 km, 2.5 km, and 5 km segments, respectively. Minimum VRAM is listed as 1×24G, 1×40G, 1×80G, and 2×80G for those same scales. Typical training uses two H200 GPUs and can run up to 600k steps. On Ann Arbor-0.5k, the method reports 29.99 dB PSNR, 0.917 SSIM, 0.240 LPIPS, and 15.158 m Depth L1. On Atlanta-5k, it reports 23.14 dB, 0.799, 0.477, and 6.622 m. The ablations show that removing ground regularization can preserve superficially strong 2D metrics while causing Depth L1 to explode to 148.54 m on Atlanta, and that removing rig pose optimization increases depth error. The paper therefore argues that photometric quality alone is insufficient for simulation readiness.

A further component is the closed-loop simulator, in which reconstructed Gaussian scenes become interactive digital twins. Agents query poses, receive rendered views, and act iteratively. Alpamayo-R1 is integrated as a proof of concept for navigation and multi-step reasoning. The stated research focus then extends beyond reconstruction quality toward simulation-critical properties: scalability, view extrapolation, and uncertainty under real-world dynamics, lighting changes, motion blur, and pose noise.

5. Cross-cutting technical themes

Although the three WildCity projects address different tasks, they share a common interest in urban intelligence under sparse, noisy, or weakly structured supervision. In the SGVN setting, the agent receives only intersection-level visual observations and destination text, with maps and GPS explicitly prohibited. In the synthetic 3DGS setting, the model must infer static scene structure from sparse views with varying illumination and transient objects. In the real-world city-scale testbed, reconstruction and simulation must cope with partial viewpoint overlap, imperfect poses, moving objects, and multi-sensor synchronization. This suggests that the recurring technical motif is not merely city scale, but the extraction of stable spatial structure from incomplete and contaminated evidence (Dalal et al., 17 Dec 2025, Furutani et al., 10 Jun 2026, Han et al., 7 Jul 2026).

A second commonality is the treatment of long horizon. For CityNav, the horizon is sequential decision-making over routes with more than 50 decision points and route means near 2 km. For the synthetic WildCity dataset, the horizon is implicit in appearance consistency across viewpoints and lighting conditions, where the model must preserve a scene-level representation across large observational variation. For the real-world testbed, the horizon is literal geographic continuity over trajectories averaging 83.7 km, along with progressive evaluation from 50 m to 5 km segments. In all three cases, short-clip or short-trajectory evaluation is treated as inadequate.

A third commonality is the inadequacy of purely photometric or language-only proxies. The navigation benchmark reports that standard reasoning techniques such as Chain-of-Thought and Self-Reflection do not solve self-localization and route execution. The synthetic reconstruction paper shows that feed-forward performance depends not only on architecture, but also on data structure that couples multi-view, multi-illumination, and transient suppression. The real-world testbed explicitly shows that high PSNR and SSIM can coexist with large geometric error, making Depth L1 and off-trajectory behavior crucial. A plausible implication is that WildCity-style evaluation consistently pushes models from surface-level plausibility toward operational spatial correctness.

The differences are equally important. The navigation WildCity is benchmark-centric and zero-shot; the synthetic WildCity is training-centric and fully supervised through rendering metadata; the real-world WildCity is infrastructure-centric, combining dataset, baseline, and simulator. Their modalities also differ sharply: Street View crops and destination strings; rendered images with exact cameras, depth, and sky masks; and fleet-scale RGB-LiDAR-GPS-IMU logs with region-level masks. As a result, the three lines are complementary rather than interchangeable.

6. Limitations, ambiguity, and prospective development

Each WildCity project identifies a different bottleneck. The SGVN benchmark remains constrained by dependence on web-scale priors, hallucinations in localization, brittle long-horizon planning in irregular graphs or ambiguous visuals, and residual Street View artifacts such as missing reverse links and orientation jumps. The synthetic WildCity dataset is limited by its urban outdoor focus, the exclusion of true nighttime extremes, 2D transient insertion without view consistency, and the absence of explicit transient masks. The real-world city-scale testbed reports residual pose errors, automatic-mask imperfections despite 91.58% mIoU, limited fine-grained semantic annotation, costly post-hoc repair for extrapolated views, and memory pressure from billion-primitive-scale reconstruction (Dalal et al., 17 Dec 2025, Furutani et al., 10 Jun 2026, Han et al., 7 Jul 2026).

The immediate future directions are correspondingly differentiated. For SGVN, the paper points to better cognitive map representations with uncertainty tracking $t$3, explicit re-localization triggers when decision accuracy drops, lightweight topology priors that do not violate the no-maps constraint, and stronger visual grounding through higher-resolution crops or targeted attention to signage and facades. For the synthetic dataset and Wild3R, the proposed path is expansion to indoor, suburban, and natural scenes, more extreme lighting and weather, transient masks or labels, and richer material attributes. For the real-world testbed, the agenda centers on tighter integration of geometric priors, improved dynamic modeling, broader adverse-condition coverage such as night and heavy rain, and scaling to more cities and more robust off-trajectory generalization.

There is also a bibliographic issue: "WildCity" is now a polysemous label spanning at least three technically unrelated artifacts. A plausible implication is that scholarly citation should rely on full titles and arXiv identifiers rather than the shorthand alone, because the same name can denote an MLLM navigation benchmark (Dalal et al., 17 Dec 2025), a synthetic 3DGS training corpus (Furutani et al., 10 Jun 2026), or a real-world city-scale reconstruction and simulation platform (Han et al., 7 Jul 2026). Within current literature, the name therefore functions best as a family resemblance term for urban AI in realistic conditions, not as a single canonical benchmark.

Taken together, these works establish WildCity as a recurring research motif at the intersection of embodied navigation, neural rendering, and spatial intelligence. One line studies whether MLLMs can self-localize and plan through real cities without maps; another studies how feed-forward 3DGS can be trained to maintain appearance consistency while suppressing transients; the third builds real-world digital twins from long fleet trajectories for reconstruction and closed-loop simulation. Their convergence around urban scale, imperfect observations, and long-horizon reasoning indicates a broader shift in evaluation: away from short, clean, highly scaffolded tasks, and toward spatial competence under the full messiness of cities in the wild.