---
title: Multi-Scale Fusion Convexity Prior (MFCP)
url: https://www.emergentmind.com/topics/multi-scale-fusion-convexity-prior-mfcp
type: topic
---

# Multi-Scale Fusion Convexity Prior (MFCP)

The Multi-Scale Fusion Convexity Prior (MFCP) is a physically motivated soft constraint on local surface convexity, designed to enhance monocular shape-from-polarization (SfP) by fusing azimuth angle information at multiple spatial scales within locally convex regions. MFCP forms a central component of the Segmentation-Driven Monocular SfP (SMSfP) framework, addressing the intrinsic azimuth ambiguity of polarization analysis and yielding improved surface normal recovery that preserves both global structure and fine-scale texture [2601.04776].

## 1. Mathematical Definition and Formulation

MFCP operates on two per-pixel azimuth angle fields: $\varphi(x, y)$ estimated directly from polarization, and $\varphi_\mathrm{im}(x, y)$ obtained by convex-boundary propagation under a convexity assumption. The procedure comprises multi-scale decomposition, blockwise detail enhancement, and variance-weighted fusion.

1. **Multi-Scale Block Decomposition**: For a discrete set of tile sizes $S = \{A, B, C, \ldots\}$ (e.g., $A=32$, $B=16$, $C=8$), both $\varphi$ and $\varphi_\mathrm{im}$ are partitioned into non-overlapping $i \times i$ blocks at each scale $i \in S$, yielding $\varphi_i$ and $\varphi_{i,\mathrm{im}}$.

2. **Blockwise Range Mapping and Detail Enhancement**: Each block of $\varphi_i$ is linearly normalized to $[0,1]$, then $\gamma$-corrected ($\gamma=0.5$):
   $$ \varphi_{i,\mathrm{gam}}(p) = (\mathrm{Norm}(\varphi_i(p)))^\gamma $$
   The result is then mapped to the dynamic range of the corresponding $\varphi_{i, \mathrm{im}}$ block, ensuring the fused result maintains the coarse statistics of the implicit azimuth while injecting localized details.

3. **Variance-weighted Fusion**: The sample variance $\sigma_i^2$ of $\varphi_i$ over the object mask is computed. Weights $\omega_i$ are set as
   $$ \omega_i = \sigma_i^2 / \sum_{j \in S} \sigma_j^2 $$
   The fused implicit azimuth is:
   $$ \varphi_\mathrm{im}^\mathrm{out}(x, y) = \sum_{i \in S} \omega_i \cdot \varphi_{i, \mathrm{im}}(x, y) $$
   This fusion leverages the most informative spatial scale(s) per region.

4. **Prior Normals**: With per-pixel zenith angle $\theta(x,y)$, the fused implicit surface normal is:
   $$ n_\mathrm{im}(x, y) = \left[
   \sin \theta \cos \varphi_\mathrm{im}^\mathrm{out},
   \sin \theta \sin \varphi_\mathrm{im}^\mathrm{out},
   \cos \theta
   \right]^T $$

5. **Estimated Normals from Height Field**: For unknown height $z(x,y)$, finite differences $D_x, D_y$ yield
   $$ n_\mathrm{est}(x, y) = \left[
   -D_x z \cdot \cos \theta,
   -D_y z \cdot \cos \theta,
   \cos \theta
   \right]^T $$

6. **Convexity-Prior Energy Term**: With pixel-wise weights $\omega_\mathrm{con}(x,y)$ (unity at boundaries, exponentially decaying inward),
   $$ E_\mathrm{convex}(z) = \sum_{x,y} \omega_\mathrm{con}^2(x, y) \| n_\mathrm{im}(x, y) - n_\mathrm{est}(x, y) \|^2 $$
   This penalizes normal deviations from the multi-scale convexity prior.

## 2. Integration Within the SfP Energy Minimization

The overall 3D reconstruction is cast as a linear least-squares problem in the height field $z$, with fixed albedo $\alpha$, refractive index $\eta$, view $v$, and lighting $l$. The energy includes:
- Azimuth constraint
- Zenith/intensity constraint
- MFCP (convexity prior)
- Laplacian smoothness

The objective function is
$$ E_\mathrm{total}(z) = \|A D z - b\|^2 + \lambda_\mathrm{conv} E_\mathrm{convex}(z) + \lambda_\mathrm{lap} E_\mathrm{lap}(z) $$
where $A D z \approx b$ is the concatenated system, $E_\mathrm{convex}$ terms are added as extra rows weighted by $\omega_\mathrm{con}$, and sparse QR is employed for solution.

## 3. Rationale for Multi-Scale Fusion

Single-scale global convexity priors derived from object masks fail to resolve local detail and introduce quantization, undermining fine-structure recovery and introducing spatial artifacts. The multi-scale MFCP approach addresses these shortcomings:
- Large tile sizes capture coarse, globally convex azimuth trends.
- Small tiles inject local textural and structural details.
- Variance-based weighting prioritizes scales where the observed azimuth is most informative, enabling automatic adaptation to region complexity.
- $\gamma$-correction accentuates subtle variation in $\varphi$ prior to fusion, preserving feature contrast while remapping to the physical range of $\varphi_\mathrm{im}$ for plausibility.

## 4. Algorithmic Procedure and Segment Interaction

MFCP operates within each polarization-aided adaptive region growing (PARG) segment:
1. **Input**: per-pixel $\varphi$, implicit $\varphi_\mathrm{im}$, block sizes $S$.
2. **For each $i \in S$**:
   - Tile $\varphi \rightarrow \varphi_i$, $\varphi_\mathrm{im} \rightarrow \varphi_{i,\mathrm{im}}$.
   - Normalize and apply $\gamma$-correction, remap to $\varphi_{i,\mathrm{im}}$'s range.
   - Compute variance $\sigma_i^2$.
3. **Compute weights** $\omega_i$.
4. **Fuse** to obtain $\varphi_\mathrm{im}^\mathrm{out}$.
5. **Convert to normals** $n_\mathrm{im}$.
6. **Add $E_\mathrm{convex}$ rows** to the matrix system with per-pixel weights $\omega_\mathrm{con}$.

PARG segments, obtained via polarization-space region growing, enable imposition of a local convexity assumption by construction. Convexity weights are strongest at segment boundaries and decay inward. Independent reconstruction of each segment, followed by guided-filter stitching of local height fields, preserves both intra-segment consistency and global surface continuity.

## 5. Implementation Details

Key parameters and choices demonstrated in [2601.04776]:
- Initial values: $\alpha=0.8$, $\eta=1.15$, $v=[0,0,1]$, $l$ estimated per [14].
- MFCP $\gamma=0.5$.
- Block sizes: $S = \{32, 16, 8\}$ or $\{16, 8, 4\}$, to balance coverage with detail.
- Finite-difference gradients: central with Gaussian smoothing; forward/backward at edges.
- Segmentation: 5x5 local windows for feature variance, queue-based growing, hole filling, and boundary smoothing.
- Optimization: QR decomposition of the sparse normal equations.
- Iterative update: alternate solving for $z$ and updating $\alpha$, $\eta$, and $\theta$ until $\Delta z < 10^{-3}$ or 10 iterations.
- Typical runtime: several seconds per $10^5$ pixels on a modern CPU.

## 6. Quantitative and Qualitative Impact

Experimental results demonstrate substantial improvements attributable to MFCP within SMSfP. On Dataset A, mean angular error declines from 25.20° (with global convexity prior) to 16.99°; on Dataset B from 20.87° to 13.69°. Pixel accuracy at $< 11.25^\circ$ error increases from 25.33% to 47.56% (A) and from 42.54% to 59.83% (B). Qualitative evaluation shows MFCP substantially suppresses large coherent error regions, enhances local normal consistency, and recovers fine structural substrate, e.g., in textures such as car grilles and animal fur [2601.04776].

## 7. Significance and Contextualization

MFCP generalizes the convexity prior in SfP from a rigid global constraint to an adaptive, multi-scale, and segment-wise regularizer. By fusing information across spatial scales and weighting according to observed variance, it circumvents fundamental limits of polarization ambiguity and coarse mask-based propagation. Coupled with PARG segmentation, MFCP enables a physically principled, locally consistent, and detail-preserving shape reconstruction pipeline, providing empirical gains in both quantitative metrics and visual fidelity compared with existing monocular, physics-based SfP methods [2601.04776].

Source: https://www.emergentmind.com/topics/multi-scale-fusion-convexity-prior-mfcp