Crop Visibility & Mask Consistency Scores

• Crop visibility and mask consistency scores are quantitative metrics that assess observable areas and label reliability in 3D crop instance segmentation.
• They leverage occlusion-aware geometric projections and semantic NeRF to merge partial 2D mask evidence into a unified 3D crop count.
• Empirical results show that integrating these metrics reduces MAPE from 7.1% to 4.9%, demonstrating significant improvements in counting accuracy.

Crop visibility and mask consistency scores are quantitative image-analysis metrics introduced to address the challenges of 3D crop instance segmentation and counting in densely occluded agricultural environments. Developed within the CropNeRF framework, these scores integrate multi-view 2D instance mask evidence with occlusion-aware geometric projections derived from a semantic neural radiance field (NeRF), enabling robust post-hoc assessment of mask reliability for each candidate crop instance and view. This paired metric approach resolves ambiguities in 2D semantic instance labeling—caused by partial occlusions and label inconsistencies—when merging partial 3D clusters into unified crop counts (Muzaddid et al., 1 Jan 2026).

1. Formal Definitions and Motivation

The crop visibility score $v_{ij}$ and mask consistency score $c_{ij}$ operate on a given 3D crop subcluster $S_i$ and a camera view $C_j$. The visibility score quantifies the fraction of the projected area of $S_i$ that is visible (i.e., not occluded by other scene geometry) from $C_j$. Its role is to restrict the weight of 2D mask evidence to only those regions of $S_i$ observable in the current view, guarding against confounds due to occlusion by non-crop entities such as leaves, branches, or soil.

The mask consistency score $c_{ij}$ evaluates, for the visible portion of $S_i$ in $C_j$, the extent to which this region aligns with a unique instance label in the 2D mask $M_j$. This metric is motivated by the tendency of 2D instance segmentation tools—including human annotation or foundation models such as Segment Anything Model (SAM)—to inconsistently split single crops or merge two physical crops into a single label, especially in clustered environments. $c_{ij}$ thus down-weights viewpoints that exhibit ambiguous mask assignments.

2. Mathematical Formulation

Let $S_i$ denote the $i$-th 3D crop subcluster, $C_j$ the $j$-th camera pose, $M_{jl}$ the region of $M_j$ assigned label $l$, and $\Pi_e$ the full scene point cloud including occluders. Two geometric projection operators are defined for each view:

• $\mathcal{P}_j(S_i)$: The occlusion-free projection of all points in $S_i$ (ignoring scene depth ordering).
• $\mathcal{V}_j(S_i)$: The occlusion-aware projection (OpenGL-style z-buffered) of $S_i$, computed by first projecting $\Pi_e$ to build a depth map, then projecting $S_i$ and retaining only those pixels where the $S_i$ point is closer than any occluder.

The three core metrics are:

$v_{ij} = \frac{\Area\bigl(\mathcal{V}_j(S_i)\bigr)}{\Area\bigl(\mathcal{P}_j(S_i)\bigr)}, \quad v_{ij} \in [0,1]$

$c_{ij} = \frac{ \displaystyle\max_{l}\Area\left(\mathcal{V}_j(S_i) \cap M_{jl}\right) }{\Area\bigl(\mathcal{V}_j(S_i)\bigr)}, \quad c_{ij} \in [0,1]$

$\lambda_{ij} = \arg\max_{l}\Area\left(\mathcal{V}_j(S_i) \cap M_{jl}\right)$

$r_{ij} = v_{ij} c_{ij} = \frac{ \displaystyle\max_{l}\Area\left(\mathcal{V}_j(S_i) \cap M_{jl}\right) }{\Area\bigl(\mathcal{P}_j(S_i)\bigr)}, \quad r_{ij} \in [0,1]$

This yields the combined mask reliability $r_{ij}$, incorporating both the geometric and mask-based reliability for $S_i$ in $C_j$.

3. Computational Procedure and Pipeline Integration

The algorithmic steps for integrating these metrics into 3D segmentation and counting are as follows:

1. Train a semantic NeRF to obtain:
   • A density field $\sigma(x)$.
   • A semantic field $s(x)$ that discriminates crop from non-crop voxels.
2. Sample:
   • The environment point cloud $\Pi_e$ by uniform sampling from $\sigma(x)$.
   • The crop-specific point cloud $\Pi_t$ by sampling $s(x)$, filtered by $\sigma(x)$.
3. Cluster $\Pi_t$ into superclusters (DBSCAN, $\epsilon = 0.02$, min_points = 30), then decompose into subclusters $S_i$ (k-means, $K = 10$).
4. For each subcluster $S_i$ and each view $C_j$: a. Project $S_i$ occlusion-free to obtain $\mathcal{P}_j(S_i)$. b. Project $\Pi_e$ to build the z-buffer (depth map). c. Project $S_i$ with z-buffer test to obtain $\mathcal{V}_j(S_i)$. d. Compute $\Area(\mathcal{P}_j(S_i))$, $\Area(\mathcal{V}_j(S_i))$. e. Intersect $\mathcal{V}_j(S_i)$ with each label region $M_{jl}$, compute $c_{ij}$ and $\lambda_{ij}$. f. Compute $v_{ij}$, $c_{ij}$, $r_{ij}$.

A pseudocode outline describes this workflow, aggregating the reliability scores for each subcluster-view pair and constructing an affinity graph for clustering:

for i in range(K): for j in range(n): Pij_mask = project_occlusion_free(S_i, C_j) Vij_mask = project_with_zbuffer(S_i, Pi_e, C_j) A_free = area(Pij_mask) A_vis = area(Vij_mask) v[i][j] = A_vis / A_free for l in unique_labels(M_j): overlap[l] = area(Vij_mask & M_j[l]) c[i][j] = max(overlap) / A_vis lambda[i][j] = argmax(overlap) r[i][j] = v[i][j] * c[i][j]

4. 3D Instance Merging and Affinity Graph Construction

The visibility and mask consistency scores are used exclusively in the post-NeRF 3D instance segmentation step; there is no back-propagation of $v_{ij}$ or $c_{ij}$ through the NeRF itself. Affinity between pairs of subclusters ($S_i$, $S_{i'}$) is quantified as:

$$\alpha_{ii'} = \sum_{j=1}^{n} r_{ij} r_{i'j} \times (-1)^{\mathbf{1}\{\lambda_{ij} \neq \lambda_{i'j}\}}$$

Positive terms indicate consistent mask assignment across views; negative terms penalize disagreement. The affinity matrix is used to construct a weighted graph on subclusters, and label propagation or simple thresholding ($\alpha_{ii'} > 0$) merges subclusters into unified 3D crop instances, each of which is enumerated as a single crop.

5. Empirical Evaluation and Ablation

Empirical results on the cotton dataset demonstrate the quantitative contributions of each score and pipeline stage:

Configuration	MAPE (%)
Baseline (raw mask agreement)	7.1
+ Visibility	6.3
+ Mask Consistency	6.5
+ Both (no label propagation)	5.4
Full CropNeRF (visibility, consistency, propagation)	4.9

The largest incremental improvement is attributable to the visibility score, while mask consistency provides a complementary but smaller gain. The inclusion of graph-based merging (label propagation) further reduces the mean absolute percentage error (MAPE), underscoring the importance of each component (Muzaddid et al., 1 Jan 2026).

6. Hyperparameters and Practical Considerations

Key method parameters anchored in the pipeline include:

• DBSCAN $\epsilon = 0.02$, $\mathtt{min\_points} = 30$ for supercluster formation.
• $K=10$ for k-means subclustering, chosen to exceed maximum expected cluster cardinality.
• Occlusion handling leverages a standard z-buffer constructed from the environment point cloud.
• No additional heuristics or thresholds are applied to $r_{ij}$: low (near-zero) reliability naturally down-weights poor-quality views in the affinity sum.
• Instance merging is governed by label propagation on the affinity graph, obviating the need for hand-tuned cutoffs.

A plausible implication is that this framework provides robustness across crop types and field conditions, as evidenced by consistent counting accuracy irrespective of crop morphology.

7. Significance in Agricultural Computer Vision

The crop visibility and mask consistency scores represent an occlusion-aware, label-ambiguity-resilient method for weighing multi-view mask evidence in 3D segmentation. Their combination yields more precise 3D instance segmentation in clustered and occluded field settings, resulting in improved end-to-end crop counting performance over raw mask agreement alone. Integration with semantic NeRF eliminates the need for crop-specific tuning, highlighting the utility and generalizability of this approach for automated agricultural monitoring (Muzaddid et al., 1 Jan 2026).