---
title: S23DR Challenge 2026
url: https://www.emergentmind.com/topics/s23dr-challenge-2026
type: topic
---

# S23DR Challenge 2026

Searching arXiv for the S23DR 2026 and related S23DR 2025 papers to ground the article in the cited literature.
S23DR Challenge 2026 is a benchmark for **structured 3D reconstruction of building roofs** from incomplete geometric and semantic inputs. The task is to predict a **3D wireframe** \(G=(V,E)\) whose vertices are metric 3D roof keypoints and whose edges encode straight roof-line connections, while preserving both topology and metric accuracy. In the available technical record, the challenge is characterized through the winning solution, which formulates wireframe prediction as conditional set generation over vertex slots and reports the top private-leaderboard result of **HSS \(= 0.654\)** [2606.06695].

## 1. Challenge formulation

The challenge targets **structured 3D reconstruction of building roofs** from **incomplete geometric and semantic information**. For each scene, corresponding to a single residential building, the required prediction is a **3D wireframe**
\[
G=(V,E),
\]
where \(V=\{v_i\in\mathbb{R}^3\}\) are metric 3D vertices corresponding to roof apexes, eave or ridge endpoints, building corners, and related structural keypoints, and \(E\subseteq V\times V\) are straight edges connecting these vertices. Each edge carries a semantic class determined by its adjacent faces, including categories such as rake, eave, flashing, and ground line [2606.06695].

The challenge departs from image-centric formulations by explicitly **not** providing RGB images. Instead, reconstruction must proceed from fused geometric and semantic evidence. This makes the benchmark a problem in direct 3D structural inference rather than 2D detection followed by lifting. A plausible implication is that the task emphasizes robustness to missing observations, calibration irregularities, and sparse sampling more strongly than conventional multi-view reconstruction pipelines.

The core technical requirement is not merely to place plausible vertices in 3D space, but to recover a graph whose **cardinality, geometry, and connectivity** jointly match the ground-truth roof structure. This dual emphasis is reflected in the official evaluation metric, which combines a vertex-detection term with an edge-overlap term [2606.06695].

## 2. Inputs, outputs, and evaluation metric

The input modalities for each scene are threefold. First, the challenge provides a **sparse SfM point cloud (COLMAP)** containing approximately \(40{,}000\) 3D points together with camera poses and intrinsics. These points are described as noisy and sparse, particularly on textureless roof surfaces. Second, it provides **fitted depth maps**, i.e. per-view metric depth scale-fitted against COLMAP; back-projected depth pixels yield additional 3D points, but depth frequently drifts at discontinuities and is often absent. The reported missingness is substantial: approximately **42% of training** and **69% of validation** views have no depth. Third, it provides **semantic segmentations** per view from both **Gestalt roof-specific classes** and **ADE20K general classes**, which are projected onto 3D points to produce semantic labels for COLMAP and depth points [2606.06695].

These heterogeneous sources are fused into a **semantic point cloud**. Each point carries normalized 3D position \(xyz\), a point-type indicator distinguishing COLMAP, depth, and camera-center points, Gestalt top-1 and top-2 labels, an ADE label, and confidence scores for both geometry and semantics. The normalization is defined by centering at the median COLMAP location and scaling by the 95th-percentile radius [2606.06695].

The output representation uses a fixed-capacity graph parameterization. The model predicts **64 vertex slots**, each associated with a 3D coordinate and an implicit validity flag, together with a dense matrix of **pairwise edge logits**. Thus the predicted graph has variable effective size but fixed representational capacity:
\[
v_i=(x_i,y_i,z_i)\in\mathbb{R}^3,
\]
with validity determining whether slot \(i\) corresponds to a real vertex, and edge logits determining connectivity between slot pairs [2606.06695].

Evaluation uses the **Hybrid Structure Score (HSS)**, defined as the harmonic mean of a vertex term and an edge term. Vertices are matched with the Hungarian algorithm under a metric threshold \(r\), producing vertex precision, recall, and \(F1_V\). Edges are rendered as cylinders in 3D, and the edge term is the volumetric IoU between the predicted and ground-truth unions of cylinders. The metric is
\[
\text{HSS}=\frac{2\,F1_V\,IoU_E}{F1_V+IoU_E}.
\]
This metric is permutation-invariant at the vertex level and jointly sensitive to vertex existence, vertex placement, connectivity, and geometric alignment of edges [2606.06695].

## 3. Dataset characteristics and representational constraints

The challenge dataset is described as **HoHo \(\rightarrow\) hoho22k\_2026\_trainval** and contains approximately **22k training** and **200 validation** buildings. Each building has **4–22 views** with mean **9**, building bounding boxes of **20–27 m diagonal**, and wireframes containing **6–36 vertices** with mean approximately **22** [2606.06695].

These statistics explain several architectural decisions in the reported winning system. The choice of **64 vertex slots** is explicitly justified as being larger than the maximum typical vertex count while remaining small enough to permit dense \(64\times 64\) pairwise edge modeling inside a transformer. The model handles variable graph size via **null slots** and a learned validity head. This yields a conditional-set formulation in which permutation invariance is enforced by **Hungarian assignment** during training rather than by imposing an arbitrary canonical ordering [2606.06695].

The scene representation is also constrained by scale. Stage 1 processes up to **8192** points, while Stage 2 processes up to **16384** points after a hull-based crop. Both stages encode the scene into **1024 scene tokens**, which then condition the vertex generator. This makes the benchmark simultaneously a problem in structural prediction and in controlled compression of large, heterogeneous point sets into a latent representation suitable for graph synthesis [2606.06695].

A plausible implication is that S23DR 2026 rewards methods that can preserve roof-specific semantic cues under aggressive subsampling. The winning method addresses this explicitly through priority-aware sampling that favors semantically important roof classes during token formation.

## 4. Winning solution architecture

The winning solution treats the wireframe as a **conditional set** and “denoises **64 vertex tokens** with a **flow-matching DiT** conditioned on **Perceiver-style scene tokens**” [2606.06695]. The pipeline has five stages: semantic point-cloud construction and normalization; Perceiver-style scene encoding; vertex-set generation by a DiT-like denoiser; a two-pass global and hull-cropped refinement scheme; and a final multi-sample consensus step.

The scene encoder first lifts each point to a \(320\)-dimensional token using normalized position, Fourier features, point type, Gestalt labels, ADE labels, and confidence scalars. It then applies two full self-attention layers over all points, followed by **anchored pooling** into **1024 learned pooling tokens** whose anchors are chosen by **priority-aware Gestalt FPS**. Two further self-attention layers over these pooled tokens produce the final **1024 scene tokens** [2606.06695].

The vertex generator is a **DiT-style** transformer operating on
\[
x_t\in\mathbb{R}^{64\times 3},
\]
the stack of 64 noisy vertex coordinates at flow time \(t\in[0,1]\). Each slot is embedded by concatenating raw coordinates with Fourier features and projecting to model dimension \(d=320\). The denoiser predicts three outputs from these tokens: a velocity field \(v_\theta(x_t,t,c)\) for ODE evolution, a validity logit per slot, and pairwise edge logits. The flow uses the linear interpolant
\[
x_t=(1-t)x_0+t x_1,
\]
where \(x_0\) is random initialization and \(x_1\) is the slot-assigned ground-truth vertex configuration. The target velocity is \(x_1-x_0\), and training uses a **SmoothL1** flow loss [2606.06695].

The architecture is explicitly two-stage. **Stage 1** performs a **global pass** over the entire scene to obtain a coarse wireframe and its convex hull. **Stage 2** computes an inflated convex hull around the Stage-1 vertices, crops the point cloud to that region, and reruns the same architectural template on denser local evidence, initializing the 64 slots from the Stage-1 prediction rather than from random noise. The stage-2 crop is designed to remove clutter while retaining the relevant roof structure [2606.06695].

The final system adds a **16-sample consensus** mechanism. Because initialization is random, multiple trajectories can yield slightly different wireframes. The method therefore runs 16 stochastic trajectories, defines a distance between predictions based on Hungarian-aligned vertex distances and edge disagreement, selects the **medoid wireframe**, and replaces each medoid vertex by a consensus aggregate over matched vertices from the remaining samples. This is explicitly described as a device to keep the stochastic sampler well behaved [2606.06695].

## 5. Training regime, inference procedure, and empirical performance

The training loss is
\[
\mathcal{L}=\mathcal{L}_{flow}+0.5\,\mathcal{L}_{end}+0.2\,\mathcal{L}_{valid}+0.2\,\mathcal{L}_{edge},
\]
where \(\mathcal{L}_{flow}\) is a SmoothL1 velocity loss, \(\mathcal{L}_{end}\) is a SmoothL1 endpoint loss on final vertex positions, \(\mathcal{L}_{valid}\) is focal loss on slot validity, and \(\mathcal{L}_{edge}\) is binary cross-entropy on pairwise edge logits. Null slots are retained during training with reduced weight [2606.06695].

Stage 1 uses a **34M-parameter** model with scene encoder width **320**, **10 attention heads**, **1024 scene tokens**, a denoiser of **12 DiT blocks**, and **64 slots**. It is trained for **2000 epochs** on **4 \(\times\) H200 GPUs** with Distributed Data Parallel, per-GPU batch size **32** for a global batch of **128**, **bf16 autocast**, fused AdamW, weight decay \(10^{-4}\), gradient clipping at **1.0**, and a learning-rate schedule consisting of **1000-step linear warmup** followed by cosine decay to a peak learning rate of \(5\cdot10^{-4}\). Stage 2 uses the same architecture, is trained first from scratch on hull-cropped scenes and then fine-tuned for an additional **1000 epochs** at learning rate \(10^{-5}\) [2606.06695].

Inference integrates the learned ODE with **50 Euler steps** from \(t=0\) to \(t=1\) in both stages. In local validation ablations, Stage 1 alone achieved **HSS \(=0.4764\)** with total runtime **3.14 s**; Stage 1 plus Stage 2 achieved **HSS \(=0.4977\)** with total runtime **4.30 s**; and the **16-sample ensemble** achieved **HSS \(=0.5025\)** with total runtime **5.52 s**. The reported interpretation is that Stage 2 adds approximately **0.0213 HSS** over Stage 1 alone, while the ensemble adds a further approximately **0.0048 HSS** and reduces sensitivity to random initialization [2606.06695].

On the official competition evaluation, the submitted system ranked **1st on the private leaderboard** with **HSS \(=0.654\)**, ahead of the **second-place entry at 0.648**, the **learned organizer baseline at 0.474**, and the **handcrafted baseline at 0.391**. It also achieved the highest reported **vertex F1 \(=0.791\)** among all submissions. The paper interprets this as evidence of strong recovery of vertex positions and counts, with edge and topology modeling still leaving room for improvement because HSS remains below vertex F1 [2606.06695].

## 6. Relation to S23DR 2025 and emerging research directions

The 2026 winning system represents a substantial methodological shift relative to the **S23DR Challenge 2025** winning approach. The 2025 system operated directly in 3D but used a staged discriminative pipeline: Gestalt-driven **vertex candidate generation**, an **Extended PointNet** for vertex refinement and classification on **4 m \(\times\) 4 m \(\times\) 4 m** cubic patches, and a second PointNet-like model for **edge classification** on **radius-1 m cylinders** around candidate vertex pairs. That system achieved **HSS \(=0.43\)** on the private leaderboard [2506.16421].

The comparison highlights several continuities. Both systems are explicitly **3D-centric**, both avoid fragile 2D-to-3D lifting, and both rely on fused geometric and semantic evidence derived from COLMAP, depth, and semantic segmentation. However, the 2026 winner replaces explicit candidate enumeration and pairwise classification with a **conditional-set generative model** that predicts the entire wireframe jointly. This suggests a transition from local patch classification to global graph synthesis as the dominant paradigm in the benchmark.

The 2026 paper also identifies likely failure modes, though not exhaustively. These include **missing fine structures**, **edge connectivity errors**, occasional **semantic misalignment**, and instability across runs without consensus. It further suggests several directions for improvement: more explicit graph priors, better treatment of edge semantics, adaptive slot counts or hierarchical representations for more complex roofs, and extensions of the Perceiver-style fusion architecture to additional modalities such as LiDAR, multispectral data, or temporal sequences [2606.06695].

In broader methodological terms, S23DR 2026 situates roof-wireframe reconstruction at the intersection of **set prediction**, **flow-based generative modeling in metric 3D space**, and **heterogeneous point-cloud encoding**. The challenge, as documented by the winning system, therefore serves not only as an evaluation task for roof reconstruction, but also as a concrete testbed for permutation-invariant structured prediction under severe geometric sparsity, missing depth, and semantically noisy evidence [2606.06695].

Source: https://www.emergentmind.com/topics/s23dr-challenge-2026