PolyLayout: Multi-room Manhattan Layout Estimation
This presentation explores PolyLayout, a method that reconstructs 3D indoor layouts from multiple posed images by representing rooms as Manhattan polygons rather than restrictive cuboids. The approach combines learned visual features with explicit geometric optimization to jointly estimate multiple room layouts while sharing orientation and height parameters, achieving strong cross-dataset generalization on both synthetic and real-world benchmarks.Script
Most layout estimation methods force every room into a cuboid, but real buildings have alcoves, corridors, and L-shaped rooms that break this assumption. PolyLayout represents rooms as Manhattan polygons, preserving the orthogonal structure of architecture while capturing the non-rectangular reality of indoor spaces.
The method initializes each room layout from camera positions rather than image semantics alone. It computes a concave hull of camera centers, buffers it, rasterizes it on a Manhattan grid, and traces the outline to obtain the starting polygon. This geometric initialization is critical because the optimization is local and topology-dependent.
PolyLayout combines three types of learned evidence with geometric constraints. Dense features are warped between views through the current 3D polygon, edge maps score projected wall boundaries, and vanishing points enforce Manhattan orientation. The network is trained end to end through the unrolled optimization process, learning features that produce useful residuals for geometry refinement.
The multi-room formulation is the paper's principal geometric extension. Rooms in the same building share a Manhattan orientation and can additionally share floor and ceiling heights, while wall offsets remain room specific. This parameter sharing couples underconstrained rooms through common structural variables and improves conditioning, raising volumetric accuracy from 93.5 to 94.3 on synthetic scenes.
On ScanNet plus plus, PolyLayout achieves 87.4 volumetric overlap and 0.20 meter Chamfer distance, outperforming the cuboid baseline. However, wall recall and room recall remain moderate at 69.3 and 36.3, reflecting incomplete scans, short wall segments, and deviations from shared height assumptions. The method does not dominate cuboid specialists on every metric but excels on non-cuboid and cross-dataset settings.
PolyLayout demonstrates that learning image-space evidence while retaining explicit geometric optimization yields better cross-dataset generalization than purely learned or point-cloud-dependent alternatives. Yet the method still struggles when complex room sections are never observed or when visual evidence is too weak to distinguish plausible wall configurations. To explore these ideas further and create your own research videos, visit EmergentMind.com.
