Top2Pano: Indoor 360° Panorama Synthesis

• The paper introduces a multi-stage pipeline that fuses occupancy estimation, ray-marched rendering, and diffusion refinement to generate photorealistic 360° indoor panoramas.
• It leverages SAM-based segmentation and structural reinforcement to infer vertical cues and resolve ambiguous floorplan inputs for coherent scene synthesis.
• Empirical evaluations on Matterport3D and Gibson demonstrate superior PSNR, SSIM, FID, and LPIPS scores, outperforming previous cross-view and whole-house panorama methods.

Top2Pano denotes the task of generating 360° panoramas from top-view inputs and, more specifically, the indoor synthesis model introduced in "Top2Pano: Learning to Generate Indoor Panoramas from Top-Down View" (Zhang et al., 28 Jul 2025). In that formulation, the input is a single 2D top-down view rendered orthographically, optionally accompanied by a camera position inside the scene, and the output is a full 360° equirectangular panorama at that position. In subsequent work, PanoWorld defines Top2Pano as generating indoor 360° panoramas from a top-view floorplan and reframes it as autoregressive generation of node-based panoramas for whole-house VR tours rather than as continuous 6-DoF scene generation (Jia et al., 18 May 2026). Across these formulations, the central difficulty is unchanged: top-view inputs do not provide explicit vertical structure, appearance cues are incomplete, occlusions must be respected from an egocentric viewpoint, and the result must remain both photorealistic and geometrically coherent.

1. Definition and problem setting

Top2Pano, in the indoor sense established by the 2025 model, targets synthesis of immersive 360° indoor panoramas from a single 2D top-down view akin to a floorplan (Zhang et al., 28 Jul 2025). The top-down input lacks explicit heights for walls and furniture, does not directly reveal first-person occlusions, and provides only ambiguous evidence for room structure and object arrangement. The model therefore treats panorama generation as a joint geometry-and-appearance inference problem rather than as a direct 2D image translation problem.

PanoWorld generalizes the same problem to the whole-house regime. Given a 2D floorplan $F$, a style condition $s$, and target panorama poses $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$, it outputs a set of furnished 360° panoramas $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$ together with a dynamic 3D Gaussian Splatting cache $C$ that acts as renderable spatial memory (Jia et al., 18 May 2026). This formulation matches commercial VR tour products, where users jump between discrete camera nodes rather than traverse a continuous asset.

Formulation	Input	Output
Top2Pano	A single 2D top-down view of an indoor floor rendered orthographically, optionally accompanied by a camera position	A full 360° equirectangular panorama at that position
PanoWorld Top2Pano	A 2D floorplan $F$, a style condition $s$, and target panorama poses $V_{\mathrm{tar}}$	Furnished 360° panoramas $I_{\mathrm{tar}}$ and a dynamic 3DGS cache $C$

A common source of confusion is that earlier cross-view panorama synthesis also used the Top2Pano label for aerial-to-ground generation. That setting synthesizes a 360-degree ground-view panorama from a top-view aerial image of the same geographic location and is explicitly framed as a cross-view image synthesis problem with no field-of-view overlap between domains, rather than as indoor floorplan-conditioned synthesis (Wu et al., 2022).

2. Core architecture of the 2025 indoor Top2Pano model

The 2025 Top2Pano model uses a three-stage pipeline: volumetric occupancy estimation, volumetric rendering of coarse panoramas, and diffusion-based refinement (Zhang et al., 28 Jul 2025). Its input modalities are a rasterized orthographic top-down RGB image $s$0 and a segmentation map $s$1 obtained using SAM. The segmentation is used because indoor top-down views lack reliable semantic labels; SAM segmentation acts as a geometric structure proxy encoding room boundaries, furniture footprints, and shapes.

The first stage, OccRecon, outputs a volumetric occupancy grid $s$2, where $s$3 is the number of vertical voxels along height. Occupancies are normalized to $s$4 and queried by tri-linear interpolation during rendering. OccRecon is described as a diffusion-based encoder–decoder operating in 2D, conditioned on $s$5 and $s$6, with a final 3D convolution that lifts features into the volumetric grid. Structural reinforcement is applied by setting wall voxels to occupancy $s$7 after normalization and reinforcing the floor using top-down priors, which yields crisper wall geometry and helps infer furniture heights and placements indirectly through end-to-end optimization.

The second stage renders coarse depth and color panoramas by ray marching through $s$8. Equirectangular projection is defined by

$s$9

with $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$0 and $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$1. For discrete volume rendering, the model follows alpha compositing:

$V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$2

$V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$3

Rather than learning a radiance field, Top2Pano directly projects colors from the top-down image along rays and assigns $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$4 from $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$5 by bilinear interpolation. Uniform voxel sampling is used along fixed-length rays. To reduce banding artifacts in floor regions, the ray length for color sampling is halved while the original longer ray is retained for depth.

The third stage, PanoGen, is a Stable Diffusion plus ControlNet refinement module. Coarse color $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$6 and coarse depth $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$7 are provided as separate ControlNet conditions and then fused, optionally with a text prompt $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$8 for stylization. Depth conditioning preserves geometry, while color conditioning preserves furniture appearance and placement. Image-guided stylization is implemented with LoRA, using intrinsic rank $V_{\mathrm{tar}}=\{v_i\}_{i=1}^N$9 and the update

$I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$0

with only $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$1 and $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$2 trained. The paper states that this adds approximately $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$3 parameters.

3. Training objectives, data construction, and implementation

The Top2Pano model is trained end-to-end with a denoising objective, a depth-alignment term, and a color histogram loss (Zhang et al., 28 Jul 2025). The diffusion loss is

$I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$4

the alignment term is

$I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$5

and the histogram term is

$I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$6

The total objective is

$I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$7

The paper explicitly notes that no explicit voxel occupancy BCE or TSDF losses are reported, and that structural reinforcement substitutes explicit 3D labels for walls and floors.

Because no public dataset contains paired top-down views and panoramas, the authors import textured 3D meshes into Blender and render orthographic top-down views from Matterport3D and Gibson. Camera positions are clustered by floor using DBSCAN, and outdoor scenes and mismatched venues are excluded. SAM is used to produce the segmentation input. For reproducibility, the described workflow consists of rendering orthographic top-down views, computing SAM segmentation, training OccRecon and PanoGen end-to-end with the losses above, generating coarse depth and color panoramas via volumetric rendering, refining them with ControlNet, and optionally fine-tuning with LoRA for style control.

Implementation details are unusually explicit for model scale and hardware. The system uses approximately $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$8B parameters, is trained on an NVIDIA RTX A6000 with 48 GB, uses batch size $I_{\mathrm{tar}}=\{I_i\}_{i=1}^N$9, runs for $C$0 epochs, and requires approximately two days per dataset. Optimization uses Adam with $C$1, $C$2, $C$3, and learning rate $C$4. Code and model release are not specified.

4. Empirical results, ablations, and generalization

Top2Pano is evaluated on Matterport3D and Gibson using PSNR, SSIM, FID, and LPIPS, with ground-truth panoramas and depths rendered from the 3D assets (Zhang et al., 28 Jul 2025). The dataset splits are reported as follows: Matterport3D has train 61 scenes / 127 floors / 6177 panoramas and test 14 scenes / 29 floors / 1405 panoramas; Gibson has train 152 scenes / 203 floors / 5379 panoramas and test 39 scenes / 76 floors / 1672 panoramas.

On Matterport3D, Top2Pano achieves PSNR $C$5, SSIM $C$6, FID $C$7, and LPIPS $C$8. On Gibson, it achieves PSNR $C$9, SSIM $F$0, FID $F$1, and LPIPS $F$2. The paper states that these results outperform Sat2Density+LDM, Sat2Density+ControlNet, and PanFusion, with especially pronounced improvements in FID and LPIPS. Cross-dataset evaluation further reports PSNR $F$3, SSIM $F$4, FID $F$5, and LPIPS $F$6 when trained on Gibson and tested on Matterport3D, and PSNR $F$7, SSIM $F$8, FID $F$9, and LPIPS $s$0 in the reverse direction.

The ablation studies isolate the roles of SAM segmentation, floor reinforcement, wall reinforcement, coarse depth, and coarse color. The full model is best on both datasets. Removing coarse color or floor reinforcement degrades furniture placement and color accuracy; removing coarse depth or wall reinforcement weakens spatial understanding and room structure; segmentation input consistently helps. Qualitatively, the model is reported to generalize to textured, plain, and hand-drawn floorplans, and to support interactive edits such that adding a rectangle in the floorplan yields a washstand with a mirror in the panorama.

These results define the original Top2Pano contribution: not a generic text-to-panorama model, but an end-to-end system that couples inferred 3D occupancy, coarse physically motivated rendering, and diffusion refinement. A plausible implication is that the volumetric intermediate is less important as a final scene representation than as a structural scaffold for controllable panorama synthesis.

5. Relationship to adjacent paradigms and to whole-house Top2Pano

Top2Pano sits between earlier cross-view image synthesis and later whole-house floorplan-conditioned panorama generation. In "Cross-View Panorama Image Synthesis," the problem is to synthesize a ground-view panorama image from a top-view aerial image, with a large semantic and appearance gap between exocentric top-down imagery and egocentric panoramic output (Wu et al., 2022). That work introduces PanoGAN, a conditional GAN with a generator that jointly outputs panorama and segmentation, a dual-branch discriminator composed of an image discriminator and a segmentation discriminator, and an adversarial feedback module that injects discriminator features back into the generator for iterative refinement. It is therefore related in output format but materially different in domain, conditioning signal, and geometric assumptions.

PanoWorld repositions Top2Pano as a whole-house synthesis problem (Jia et al., 18 May 2026). Instead of generating a single panorama from a single top-down input, it treats whole-house synthesis as autoregressive generation of node-based panoramas organized in a topological graph $s$1, where edges encode navigation adjacency. The system uses a floorplan-derived 3D shell $s$2 as a global geometric proxy and a dynamic 3DGS cache $s$3 as renderable spatial memory. For the first node,

$s$4

and for subsequent nodes,

$s$5

where $s$6 is shell-derived geometry guidance, $s$7 is cache-rendered memory, and $s$8 is a nearby panorama carrying local appearance and style. A feed-forward panoramic LRM lifts generated panoramas into local Gaussian updates, while Room-aware Group Attention restricts attention to same-room or doorway-connected tokens. Progressive cache fusion, selective spherical harmonics blending, and cross-room depth gating are then used to limit leakage across walls.

The contrast with pure 2D panorama generators is stated directly in PanoWorld’s comparison to prior Top2Pano-style methods. PanFusion, PanoGen, DreamHome-Pano, Seedream-4.5-Edit, and Nano Banana 2 can produce attractive individual panoramas but lack persistent spatial memory and re-imagine geometry and materials across nodes. Pano2room reconstructs geometry from a single panorama but lacks persistent whole-house memory in the adapted setting. OmniRoam improves local temporal continuity but lacks topology-aware cache and room-aware reconstruction. PanoWorld reports HPSv3 $s$9, CLIP-I Style $V_{\mathrm{tar}}$0, and $V_{\mathrm{tar}}$1 $V_{\mathrm{tar}}$2 on its floorplan benchmark, with the best $V_{\mathrm{tar}}$3 among baselines; OmniRoam records $V_{\mathrm{tar}}$4, a deficit of $V_{\mathrm{tar}}$5 dB. In this sense, PanoWorld does not replace Top2Pano so much as extend the task from single-node synthesis to cross-node coherent tour generation.

6. Limitations, misconceptions, and open problems

The principal limitations of the 2025 Top2Pano model follow from the ambiguity of top-down inputs (Zhang et al., 28 Jul 2025). Reported failure modes include missing or hallucinated ceiling fixtures such as fans and lights, incorrect wall heights or decorations, false windows, furniture height errors, missing thin objects such as TVs, and incorrect stair direction. The paper also notes that the training data imposes limited vertical FoV and suggests that broader-FoV datasets would likely help. Future directions proposed by the authors include explicit geometric priors such as TSDF or plane constraints, multi-view or temporal top-down cues, semantic reasoning for object categories and heights, and better handling of thin structures and clutter.

PanoWorld identifies a different but related set of failure modes at whole-house scale (Jia et al., 18 May 2026). Floorplan-to-shell errors, misplaced walls, missing doorways, incorrect topology, or sparse node spacing can weaken cache guidance and consistency. Dynamic objects, mirrors, transparent materials, and heavy occlusions remain challenging for both LRM lifting and cache rendering. The 12-view LRM setting shows slightly reduced reconstruction metrics relative to 8 views because of larger scene extents and more cross-room visibility changes. Its ablations further show that removing Room-aware Group Attention significantly degrades LRM performance, removing visual memory or nearby view reduces cross-node $V_{\mathrm{tar}}$6, and removing CPRoPE induces seam artifacts and lowers cross-node alignment.

A recurrent misconception is to equate Top2Pano with monolithic 3D reconstruction. The indoor Top2Pano model uses volumetric occupancy and volumetric rendering, but its final objective is panorama synthesis, not construction of a reusable global 3D asset (Zhang et al., 28 Jul 2025). PanoWorld makes this distinction explicit: the cache is not a final photorealistic asset, but a spatial memory rendered at new nodes to preserve appearance and material identity across views (Jia et al., 18 May 2026). Another misconception is to treat all top-view-to-panorama work as a single category. The aerial-to-ground cross-view problem addressed by PanoGAN, the indoor single-node floorplan-conditioned problem addressed by Top2Pano, and the multi-node whole-house problem addressed by PanoWorld share a panoramic output space but differ substantially in topology, geometry priors, and consistency requirements.

Taken together, these works define Top2Pano less as a single fixed architecture than as an evolving research direction. The 2025 model establishes occupancy-to-rendering-to-diffusion synthesis for single indoor panoramas, while PanoWorld shows that whole-house deployment requires explicit topology, persistent visual memory, and cross-node consistency mechanisms. This suggests that future Top2Pano systems will likely continue to hybridize geometric proxies with high-fidelity 2D synthesis rather than rely exclusively on either pure 2D generation or monolithic scene reconstruction.