SatSkylines: Efficient 3D Building Synthesis

• SatSkylines is a 3D building generation approach that synthesizes detailed models from satellite imagery and coarse geometric priors.
• It employs a sparse-structure variational autoencoder with cosine-scheduled interpolation to balance between strict geometric fidelity and creative, appearance-driven reconstructions.
• Validated on the Skylines-50K dataset, the method demonstrates improved accuracy and efficiency for urban digital twin creation and scalable 3D asset generation.

SatSkylines is an approach for 3D building generation that synthesizes detailed models from satellite imagery and coarse geometric priors. It addresses fundamental limitations in image-based and detailization methods, particularly the challenges of inferring accurate building structure from top-down satellite perspectives and the inefficiency of voxel-heavy techniques when only simple geometric cues are available. The framework is designed for flexible geometric control and efficient 3D asset generation, leveraging a new dataset, Skylines-50K, with over 50,000 unique 3D building assets to support training and benchmarking.

1. Methodological Framework

SatSkylines formulates the transformation from satellite imagery and coarse geometric priors to detailed 3D building models as a guided generative process in latent space. The method begins by encoding the coarse geometric prior, represented as a voxel grid $\mathcal{O} \in \{0,1\}^{N\times N\times N}$, into a latent code $Z_\mathcal{O}$ using a sparse-structure variational autoencoder (SS VAE). Since $Z_\mathcal{O}$ tends to have a mean and variance inconsistent with the standard Gaussian (empirically close to $\mathcal{N}(0,0.2)$), it is normalized channel-wise: $$Z_\mathcal{O}' = \frac{Z_\mathcal{O}-\mathrm{Mean}(Z_\mathcal{O})}{\mathrm{Std}(Z_\mathcal{O})}.$$

Rather than employing pure noise input, as is common in prior generative models such as Trellis, SatSkylines interpolates between geometric latent and standard Gaussian noise $(\epsilon)$ via a cosine-scheduled parameter $\lambda \in [0, 1]$: $$Z_\mathcal{O}'' = \cos(\lambda\pi/2) Z_\mathcal{O}' + \sin(\lambda\pi/2) \epsilon.$$ This enables continuous geometric control over the generation: $\lambda=0$ produces models closely aligned with the provided prior, $\lambda=1$ yields unconstrained, pure noise-based generations, and intermediate values allow interpolation between these extremes.

The enhanced prior $Z_\mathcal{O}''$ is processed by a Sparse Structure (SS) flow transformer to obtain a geometry latent $Z_{\text{ss}}$. Separately, a Structured Latent transformer encodes appearance cues ($Z_{\text{slatt}}$). Final textures and detailed geometry are conditioned on features from satellite images, which are fused via cross-attention layers to inject high-frequency appearance information.

2. Flexible Geometric Guidance

Unlike previous approaches that concatenate high-fidelity voxels or add overhead for more accurate priors, SatSkylines imposes geometric structure through its cosine interpolation. This mechanism allows for the geometric latent to be “blended” in, controlling the degree of prior influence at runtime without a computational penalty. The cosine scheduling provides a smoothly tunable path from rigid, reproducible reconstructions (even from crude geometric cues such as cuboids) to explorative, appearance-driven models for urban or simulation design contexts.

A plausible implication is that SatSkylines can dynamically balance fidelity vs. creativity, repairing or inventing plausible geometry in the absence of dense height or shape inputs, and restricting the solution space as more detailed priors become available.

3. Skylines-50K Dataset

SatSkylines is supported by Skylines-50K, a large and diverse 3D building dataset curated from the Steam Workshop for the simulation game “Cities: Skylines.” Each of over 50,000 assets is a hand-crafted 3D building model rendered in a variety of satellite-style top-down views, paired with automatically generated geometric priors at multiple levels of detail (LODs):

LOD Level	Geometric Prior	Description
LOD 0	Simple cuboid/bounding box	Minimal coarse guidance
LOD 1	Single cross-section, repeated height	2D outline extruded to 3D
LOD 2	Two cross-sectional profiles/height	Slightly richer geometric information

These priors mimic the sparse footprints and building heights found in OpenStreetMap datasets and similar sources. The large asset variety (architectural style, region, texture richness) ensures that the model’s learned priors capture a high degree of real-world plausibility and generalization.

4. Quantitative and Qualitative Performance

Evaluation is conducted on a 500-sample subset of the test split from Skylines-50K (with a 20-asset subset for baseline comparisons due to speed constraints). Key geometric and appearance metrics include:

• Intersection-over-Union (IoU)
• Chamfer Distance (CD)
• F Score (for 3D shape overlap)
• Peak Signal-to-Noise Ratio (PSNR)
• LPIPS (perceptual similarity)
• CLIP similarity score (image-text alignment)

SatSkylines demonstrates improved reconstruction accuracy compared to image-only approaches (e.g., Trellis) and outperforms voxel-dependent methods such as CLAY particularly when only minimal geometric priors are provided. Asset generation averages roughly 15 seconds per model, a notable efficiency gain compared to multi-minute runtimes for previous flows.

These results suggest that SatSkylines is well suited for scalable 3D urban asset generation where only basic footprint/heights and satellite/aerial images are available.

5. System Pipeline and Real-World Application

The SatSkylines pipeline is fully end-to-end and designed for integration into urban digital twin and city simulation systems. The process encompasses:

• Automatic extraction of building footprints and heights from platforms such as OpenStreetMap.
• Image enhancement through a super-resolution module based on “gpt-image-1,” counteracting the effect of low-resolution satellite images.
• Generation of 3D models using the previously discussed latent transformation, geometric control, and satellite image fusion.

Applications include:

• Urban planning and digital twins for large-scale city environments, producing detailed, realism-consistent 3D assets with minimal manual input.
• Augmentation or repair of incomplete geospatial data, using the interpolation parameter $\lambda$ to compensate for missing or low-detail priors.
• Simulation, gaming, or architectural pre-visualization where plausible building datasets need to be synthesized rapidly and flexibly.

6. Limitations and Future Directions

While SatSkylines provides substantial flexibility and efficiency, its reliance on satellite-style images and the specific structure of Skylines-50K mean that further work is necessary to extend to more complex, multi-modal inputs (e.g., text, segmentation masks, or multiple perspective images). The authors note several directions for future research:

• Training on a broader set of data (potentially with multi-view or oblique angle imagery).
• Enriching the appearance and geometric enhancement step to better handle artifacts in input satellite images, including developing higher-fidelity super-resolution models.
• Extension to support dynamic urban settings (e.g., time-evolving structures).
• Integration of auxiliary modalities (e.g., textual context) to facilitate finer control over model outputs.

This suggests that increasing both the variety of inputs and the capacity of the underlying models will be important for the next generation of scalable urban digital twin generation.

7. Comparative Positioning

SatSkylines is positioned within a recent line of research that addresses automated, scalable 3D asset creation for urban settings. Its closest predecessors include voxel-concatenative methods for 3D detailization and flow-based image-to-3D approaches for asset generation (e.g., Trellis, CLAY), but SatSkylines’ innovation lies in combining simple, real-world geometric cues (as from public GIS data) with satellite imagery, while controlling fidelity and computational cost via its latent interpolation mechanism. This places it as a practical and generalizable method for urban 3D synthesis from sparse observational data.

In summary, SatSkylines constitutes a methodological advance in the use of satellite imagery and geometric priors for detailed 3D building generation, leveraging novel architectural elements, flexible geometric mixing, and a purpose-built dataset to address the efficiency and scalability demands of large-scale urban digital twin applications (Jin et al., 25 Aug 2025).