DeepC4: Census-Constrained Deep Clustering

• The paper introduces a deep learning framework that combines an autoencoder with census-derived constraints for high-resolution mapping of roof, wall, and height features.
• It leverages multisource Earth observation data and conditional probability tables to enforce pixel-level consistency in urban morphology classifications.
• The method demonstrates enhanced performance over traditional spatial disaggregation approaches, offering low error rates at national, provincial, and pixel scales.

Deep Conditional Census-Constrained Clustering (DeepC4) is a deep learning-based framework for large-scale spatial disaggregation of urban morphology, explicitly designed to overcome the limitations of classical spatial disaggregation methods by tightly incorporating local census statistics as cluster-level constraints. DeepC4 jointly models conditional label relationships within a multitask clustering architecture, leveraging multisource Earth Observation (EO) data and subnational census-derived priors to generate high-resolution, semantically consistent maps of critical built-environment indicators such as roof type, wall type, and building height. This method addresses key discrepancies and propagated uncertainties that arise when mapping urban characteristics from coarse to fine scales under weak label supervision, as motivated by documented challenges in global exposure mapping efforts such as the GEM Uniform African Exposure Dataset and the METEOR Project (Dimasaka et al., 30 Jul 2025).

1. Model Architecture and Disaggregation Pipeline

DeepC4 ingests multi-band satellite imagery (Sentinel-1 VV/VH backscatter, Sentinel-2 B2–B12 reflectances at 10 m resolution), pixel-level spatial coordinates, and an iteratively constructed “possible-building” mask. This mask is derived from four imperfect building footprint sources (OSM, Google, Microsoft, Overture) and refined using the Dynamic World V1 built-area probability map, with dynamic thresholding to enforce sector-specific census-derived pixel counts. Sector-level census statistics (household counts, urban/rural split, wall/roof/height distributions) are translated into pixel-level class constraints via expert-elicited conditional probability tables.

The core model consists of a fully-connected autoencoder $E$–$D$, which, for every masked pixel $j$ in sector $i$, transforms a 14-dimensional input vector $x^{(j)}$ (comprising 12 spectral/polarimetric bands and 2 log-coordinates) into a normalized representation. This is encoded into a $k$-dimensional latent embedding $z^{(j)}$, where $k = 9$ is partitioned into three 3-dimensional subspaces encoding features for roof, wall, and height indicators, respectively.

For each indicator $$t \in \{\mathrm{roof}, \mathrm{wall}, \mathrm{height}\}$$, DeepC4 introduces a constrained clustering head. Each pixel is assigned (via a binary indicator $T^{(j,h)}_t$) to exactly one of $D$0 cluster centers $D$1, with the cluster population matching census-implied pixel counts as enforced by an integer linear programming (ILP) constrained k-means objective (minimum-cost flow; see Bradley et al. 2000). The autoencoder parameters are trained to minimize both reconstruction error and task-specific clustering losses under these census-derived constraints.

2. Loss Functions, Constrained Clustering, and Census Integration

DeepC4’s optimization is governed by the following loss components computed per sector $D$2:

• Autoencoder Loss:

$D$3

• Constrained k-means Task Loss:

For each indicator $D$4,

$D$5

Class imbalance is compensated via $D$6, normalized across classes. The hard assignment constraint ensures $D$7, with $D$8 directly derived from conditional probability tables (e.g., $D$9, $j$0, $j$1) and census tabulations.

• Total Loss:

$j$2

The census-derived constraints serve as strict equality constraints in the clustering head. The optimization procedure iteratively updates latent codes and cluster assignments jointly; the ILP-backed constrained clustering guarantees that census priors are strictly respected.

3. Encoding Conditional Label Relationships in Multitask Learning

The method operationalizes conditional relationships formalized in census tables and expert knowledge by sequentially decomposing raw tabulations to pixel-level constraints:

• Urban/rural dwelling counts are estimated using the average household size per sector.
• Pixel counts for each roof, wall, and height class result from chained application of conditional probabilities.
• The final constraints are mapped onto the clustering assignment problem with each pixel as a supply and each class/cluster as a demand node with census-determined demand.

Through backpropagation of the clustering loss, the latent space $j$3 is shaped by multitask cross-influence, yielding shared representations that enforce consistency across morphological attributes. The model thus provides pixelwise predictions that, in aggregate, reproduce all known sector-level attribute marginals and their conditional dependencies encoded in the census tables.

4. Data Preparation and Feature Engineering

EO inputs comprise annual median (Sentinel-2) and mean (Sentinel-1) composites generated in Google Earth Engine, supplemented with normalized pixel coordinate features. The census-based computations include derivation of average household sizes, allocation of dwellings to urban/rural typologies, and iterative conversion of buildings to pixel counts using precise area conversion factors.

The building presence mask is constructed by rasterizing all footprint sources; mask pixel counts are iteratively adjusted via thresholding of built-area probabilities, ensuring alignment with census-implied totals. Final masks guarantee that only the precise number of relevant pixels are presented to the autoencoder and constrained clustering modules for each sector.

5. Training Strategy and Hyperparameterization

Training is conducted on 20 Kigali sectors with available groundtruth typology (2015), using five-fold cross-validation (four sectors per fold), with the remaining 396 sectors addressed in inference mode. Model optimization employs the Adam algorithm (learning rate $j$4, $j$5, $j$6), with each batch comprising all masked pixels from a sector. Hyperparameters include latent dimension per task $j$7 (total $j$8), number of clusters $j$9, $i$0, $i$1, and normalized task weights. Training typically converges after approximately 200 epochs, with early stopping determined by the flattening of $i$2 and minimization of $i$3. A single NVIDIA Tesla V100 (32 GB RAM) provides sufficient computational resources, with training requiring approximately six hours for 20 sectors at full epoch count.

Component	Setting/Value	Rationale
Optimizer	Adam ($i$4)	Adaptive learning, efficient convergence
Latent dims/task	3 ($i$5 total)	Partitioned for multitask separability
Clusters/task	Roof: 4; Wall: 8; Height: 6	Matches census/morphology category granularity
Epochs	~200 (optimal at 184)	Convergence observed empirically
Batch size	Full sector ($i$6 pixels)	Sector masks differ in size

6. Evaluation, Results, and Comparative Analysis

National, provincial, and sectoral metrics position DeepC4 as an advancement over prior approaches such as GEM and METEOR. On 2022 Rwandan census data:

• National Totals: 1.13% error in dwellings, 1.11% in occupant counts; GEM yields 2.03%/3.29%, METEOR exceeds 5% for both.
• Urban/Rural Splits: <1.8% error in dwellings per stratum, 0% in occupants; GEM shows –13% (urban) and +8% (rural) error.
• Province-level Errors: Consistently <1.4% for DeepC4; GEM reaches up to –19% (Kigali) and +13% (other provinces).
• Pixel-level Disaggregation: 32–49% of pixels display higher DeepC4 inferred building counts relative to 2022 EO data, with no spatially systematic bias.
• Clustering Precision: Five-fold cross-validation yields roof prediction true positive rates rising from 98.97% to 99.03%, wall 96.64% to 96.45%, and height 95.34% to 95.87%. Weighted class-balance versions of these metrics remain stable within ±2% across folds.

Qualitative outputs include pixelwise sectoral maps, province/district/sector pies showing morphology distributions (e.g., >60% tile roofs in Western Province; iron sheet/wood mud wall dominance in Eastern Province), and visualizations of how the building mask reconciles discrepancies between footprint sources via EO-informed refinement.

7. Limitations, Uncertainty, and Prospective Extensions

Several limitations are acknowledged:

• Groundtruth Validation: Only 20 Kigali sectors with 2015 typology are available, necessitating the assumption of static wall/roof/height proportions through 2022. Integrating updated labels via on-the-ground surveys or high-resolution Lidar is recommended for future work.
• Constraint Rigidity: Current ILP implementation enforces deterministic pixel assignment, ignoring sampling uncertainty in conditional probabilities. Relaxing constraints via soft penalties or probabilistic resampling is a theoretical extension for uncertainty quantification.
• Computational Scalability: While the ILP solver presently scales linearly with sector count, increased administrative resolution (<10 m grid) may induce a bottleneck; GPU-friendly relaxations such as Sinkhorn clustering are plausible solutions.
• Temporal Invariance: DeepC4 is presently static; introducing recurrent or time-aware encoders would permit explicit modeling of urban expansion over longitudinal datasets.
• Regional Generality: Application to new geographies requires re-specification of conditional probability tables $i$7, which could be addressed by learning these distributions empirically from cross-country surveys.

DeepC4 demonstrates the feasibility of embedding census-level constraints and expert-elicited priors into deep clustering frameworks, enabling more accurate and spatially coherent urban morphology mapping than previous EO-driven and classical disaggregation approaches, while highlighting future directions for uncertainty representation and temporal modeling (Dimasaka et al., 30 Jul 2025).