CropNeRF: A Neural Radiance Field-Based Framework for Crop Counting

Abstract: Rigorous crop counting is crucial for effective agricultural management and informed intervention strategies. However, in outdoor field environments, partial occlusions combined with inherent ambiguity in distinguishing clustered crops from individual viewpoints poses an immense challenge for image-based segmentation methods. To address these problems, we introduce a novel crop counting framework designed for exact enumeration via 3D instance segmentation. Our approach utilizes 2D images captured from multiple viewpoints and associates independent instance masks for neural radiance field (NeRF) view synthesis. We introduce crop visibility and mask consistency scores, which are incorporated alongside 3D information from a NeRF model. This results in an effective segmentation of crop instances in 3D and highly-accurate crop counts. Furthermore, our method eliminates the dependence on crop-specific parameter tuning. We validate our framework on three agricultural datasets consisting of cotton bolls, apples, and pears, and demonstrate consistent counting performance despite major variations in crop color, shape, and size. A comparative analysis against the state of the art highlights superior performance on crop counting tasks. Lastly, we contribute a cotton plant dataset to advance further research on this topic.

• The paper introduces CropNeRF, a parameter-insensitive framework leveraging semantic neural radiance fields for precise crop instance segmentation and counting.
• It integrates multi-view RGB imagery with reliability-weighted graph-based clustering to accurately resolve crop instances, even under occlusion and mis-segmentation.
• Empirical results on diverse datasets show CropNeRF achieving low error rates (as low as 2.4% MAPE) and strong generalization across crop types.

CropNeRF: A Neural Radiance Field-Based Framework for Crop Counting

Problem Motivation and Related Work

Precision agriculture demands robust, high-fidelity methods for accurate crop instance enumeration, especially under challenging real-world conditions, such as occlusions, variations in crop morphology, and outdoor imaging constraints. Traditional 2D and 3D-based computer vision techniques, although effective in constrained environments, have significant shortcomings: limited depth cues in monocular images, calibration requirements for SfM-based reconstructions, and the reliance on hand-tuned parameters or fixed-size template fitting in clustering-based approaches. Recent NeRF and 3D Gaussian Splatting methods offer volumetric scene understanding, but often remain sensitive to initial segmentation errors, parameter choices, and fail with irregular, cluster-prone crops.

CropNeRF positions itself as a domain-agnostic, parameter-insensitive solution for crop counting by leveraging semantic neural radiance fields and robust graph-based segmentation for 3D instance counting. Unlike prior clustering-based approaches such as FruitNeRF and volume renderers that require template matching or indoor-controlled datasets (as in Cotton3DGaussians), CropNeRF generalizes across crop types (cotton, apples, pears) and outdoor environments, establishing strong practical value for phenotyping and yield estimation tasks.

CropNeRF Pipeline

The CropNeRF pipeline ingests multi-view RGB images with associated 2D instance masks and camera poses. The framework proceeds through the following stages:

1. NeRF-based Semantic Reconstruction: The captured images and segmentation masks are used to train a volumetric NeRF encoder augmented with a semantic field, allowing for explicit modeling of both general scene density and crop presence at every spatial location.
2. Semantic Point Cloud Generation: Semantic field activation is used to extract target crop point clouds from the 3D volume, refined by filtering on density to obtain high spatial fidelity.
3. Hierarchical Clustering: First, DBSCAN segments the point cloud into spatially disjoint "superclusters" to address computational tractability and isolate independent crop clusters. Each supercluster is further partitioned into $K$ "subclusters" via k-means (with $K >$ the maximal expected crop instances per region), intentionally generating fine-grained oversegmentation.

   Figure 2: The subcluster merging process. Top left: supercluster divided into subclusters; graph affinities reflect visibility/mask consistency; label propagation yields final merged instances.

4. Occlusion- and Consistency-Aware Affinity Modeling: For each subcluster-camera pair, the following are computed: a subcluster visibility score (based on camera pose and occlusion-aware projection), a mask consistency score (quantifying overlap quality with segmented 2D instance regions), and a joint mask reliability score as their product. These quantities modulate pairwise subcluster affinities in a complete graph built over all subclusters in a supercluster.
5. Subcluster Merging via Graph Partitioning: Affinity scores are aggregated across views. Edges penalize merging if instance labels differ, and are weighted by the aggregated reliability scores. Label Propagation partitions the graph, merging consistent subclusters into final 3D crop instances and thus yielding the exact count.

This pipeline is robust to input mask inconsistencies, partial occlusions, and variable instance morphologies, due to the multi-view, reliability-weighted merging procedure.

Empirical Evaluation

CropNeRF is validated on three diverse datasets: an in-field cotton plant set (with multi-view iPhone imagery, high shape variation, and challenging occlusions), and two FruitNeRF benchmarks (apples and synthetic pears). Notably, the cotton dataset, collected outdoors with strong annotation error via 2D SAM-based instance masks, stresses generalization beyond controlled lab settings.

Accuracy

On the cotton dataset, CropNeRF achieves a mean absolute percentage error (MAPE) of 4.9%, outperforming FruitNeRF (18.1%) and COLMAP-based baselines. For apples the error is 4.7% (FruitNeRF: 4.5%), and for pears, 2.4% (FruitNeRF: 3.6%). In all scenarios, CropNeRF's parameter setting for $K$ and clustering remains fixed irrespective of crop, while prior methods require dataset or instance-specific tuning for template size. The method is also shown to tolerate high levels of mask ambiguity and annotation error, maintaining superior counting accuracy in cases of mis-segmentation and missed detections.

(Figure 2)

Figure 3: Visual overview of cluster partitioning and graph-based merging for subcluster instance resolution.

Mask and Clustering Parameter Sensitivity

Only 15–30 instance masks per crop are generally sufficient for accurate counting—substantial annotation efficiency compared to typical video-based or single-image tracking approaches that scale linearly with number of viewpoints. The method is empirically robust for moderate to large $K$ values, as long as $K$ upper-bounds the number of spatially-confined crop instances.

Figure 4: Varying numbers of subclusters $K$ have minimal impact on final crop counts once $K$ exceeds the maximum cluster occupancy.

Ablation

Progressive integration of visibility scores, mask consistency, and label propagation each incrementally improve performance. Ablation demonstrates that visibility-awareness is particularly critical for disambiguating overlapping instances and suppressing dataset-specific segmentation artifacts.

Theoretical and Practical Implications

CropNeRF demonstrates that integrating volumetric scene priors with multi-view, mask-reliability-weighted graph partitioning yields parameter-insensitive, generalizable solutions for dense instance segmentation in unstructured agricultural environments. By avoiding fixed templates and explicit cross-view instance correspondence, this method is naturally extendable to other domains in robotics and biological imaging where instance separation under occlusion is a critical bottleneck.

Practically, the core insight—decoupling initial mask quality from segmentation output by reliability-weighted, multi-view integration—promises to relax annotation requirements for future datasets. The approach can leverage noisy or partial instance masks, reducing the cost of annotation and enabling deployment across variable species or growth stages without retraining or parameter search.

Future Directions

While CropNeRF's 3D NeRF backbone poses substantial computational expense, future research may leverage lighter-weight volume representations or hybridize with fast 3DGS techniques. An important open direction is integrating multi-spectral and thermal modalities to counter the limitations of RGB for severe occlusion or specular crop surfaces. Active sensing (e.g., LiDAR fusion) and self-supervised, in-situ segmentation refinement using NeRF uncertainty feedback loops may further advance robustness.

Conclusion

CropNeRF establishes a new framework for robust crop instance enumeration under challenging field conditions, delivering strong, crop-agnostic segmentation and counting reliability. Its volumetric, multi-view, reliability-based methodology generalizes beyond prior template-dependent approaches and demonstrates potent practical and theoretical contributions toward scalable, high-quality agricultural automation.