---
title: 'PlantSAM: Plant-Adaptive Segmentation Model'
url: https://www.emergentmind.com/topics/plantsam
type: topic
---

# PlantSAM: Plant-Adaptive Segmentation Model

PlantSAM denotes a plant-adapted use of the Segment Anything Model family in which generic promptable segmentation is constrained by plant-specific priors, auxiliary detectors, or domain-specific representations. In the narrow sense, it is the name of an object-detection–driven segmentation pipeline for digitized herbarium specimens that integrates YOLOv10 with SAM1 or SAM2, with the best-performing variant, PlantSAM2, combining YOLOv10 and SAM2 for automated plant–background segmentation in herbarium sheets [2507.16506]. In a broader research sense, the label has also been used to describe SAM-centered systems for zero-shot leaf segmentation, crop–weed panoptic segmentation, field delineation, tree-crown extraction, orchard counting, and NeRF-based 3D phenotyping, all of which couple SAM-like segmentation with plant-specific prompting, post-processing, or multimodal cues [2305.09418], [2309.13578], [2506.16318], [2503.20199], [2508.20954], [2506.04970], [2505.13633]. A plausible implication is that PlantSAM now refers both to a specific herbarium pipeline and to a methodological family of plant-aware SAM adaptations.

## 1. PlantSAM as a herbarium-specimen segmentation pipeline

In the specific sense established by "PlantSAM: An Object Detection-Driven Segmentation Pipeline for Herbarium Specimens" [2507.16506], PlantSAM addresses a central problem in herbarium image analysis: the foreground specimen typically occupies a small fraction of the sheet, while the remainder contains paper, labels, pins, barcodes, mounting strips, and other artifacts. The paper reports that many taxa have less than 15–20% plant coverage, with examples including Magnolia at 19.68% plant and 80.32% background, and Convolvulaceae at 7.12% plant and 92.88% background. The stated motivation is that heterogeneous background can mislead deep learning classifiers and reduce trait-classification accuracy.

The architecture is organized into four phases: patching, plant region detection with YOLOv10, segmentation with SAM1 or SAM2, and unpatching. Herbarium sheets of approximately 4000×6000 pixels are preprocessed with erosion and dilation, then partitioned into patches whose size depends on image width: 1024, 512, or 256 pixels. Each patch is normalized and processed by a herbarium-specific YOLOv10 detector that predicts bounding boxes for plant regions. These boxes are then used as prompts for SAM. Two prompting strategies are described: a single-box strategy and a multi-region strategy. The latter became the default because a controlled experiment on 19 images increased the average plant-area / bounding-box-area ratio from 42.01% to 51.96%, thereby reducing background within prompts [2507.16506].

PlantSAM distinguishes two model variants. PlantSAM1 combines YOLOv10 with SAM1, whereas PlantSAM2 combines YOLOv10 with SAM2. Unless otherwise specified, PlantSAM refers to the general YOLOv10 + SAM design. Both SAM variants are fine-tuned on herbarium masks, and the final masks from multiple prompts within a patch are merged by logical OR or union. After reconstruction at full resolution, the output is a binary plant mask covering leaves, stems, fruits, and flowers while excluding the sheet background. The authors also describe a semi-automatic annotation tool in which PlantSAM2 can be refined with point prompts, turning unusable masks into usable ones and enlarging training data [2507.16506].

## 2. Training data, optimization, and segmentation performance

The herbarium PlantSAM study uses three distinct data resources. For segmentation, 1,476 images with high-quality masks were selected from previously published herbarium segmentation datasets; these images span 9 genera and 2 families, including Magnolia, Castanea, Rubus, Laurus, Litsea, Eugenia, Ulmus, Monimiaceae, Amborella, Desmodium, and Convolvulaceae. Evaluation uses a separate curated test set of 333 images from the Digitised Herbarium Image Segmentation Dataset. For plant-region detection, YOLOv10 is trained on 19,078 annotated patches, split into 14,307 training, 3,815 validation, and 956 test patches. For out-of-distribution analysis, the paper also uses 171 digitized herbarium images with challenging backgrounds such as blue, grey, yellow, brown, orange, pink paper, newspaper, long thin stems, long leaves, and pins [2507.16506].

The segmentation models are trained with Dice loss,
$$
D(P,G) = \frac{2 |P \cap G|}{|P| + |G|},
\qquad
L_{\text{Dice}} = 1 - D(P,G),
$$
using Adam with initial learning rate $10^{-5}$ and cosine annealing. SAM1 and SAM2 are trained for up to 80 epochs with batch size 1. A UNet with ResNet101 backbone is trained for up to 100 epochs with early stopping. YOLOv10 is trained for 250 epochs on 640×640 inputs and achieves mAP@0.5 = 0.959 on the plant-region detection test split [2507.16506].

On the curated herbarium test set, PlantSAM2 achieves mean IoU = 0.94 and Dice = 0.97, outperforming both UNet and PlantSAM1. The paper reports taxon-level IoU gains over UNet, including Convolvulaceae from 0.8222 to 0.8789, Desmodium from 0.8337 to 0.9018, Ulmus from 0.8995 to 0.9386, Rubus from 0.9185 to 0.9504, and Magnolia from 0.9497 to 0.9656. Dice improvements are smaller but remain positive for many taxa, including Amborella from 0.9619 to 0.9750, Desmodium from 0.9410 to 0.9528, Rubus from 0.9714 to 0.9798, and Ulmus from 0.9654 to 0.9729 [2507.16506].

The same paper links segmentation quality to downstream trait recognition. ResNet101 classifiers trained on segmented and segmented-cropped images outperform classifiers trained on raw sheets across five botanical traits: armatures, fruits, leaves with acuminate tips, infructescence, and leaves with an acute base. The largest reported gains for segmented-cropped versus unsegmented inputs are +4.36 accuracy for armatures and +4.15 F1 for infructescence. This suggests that, within herbarium analysis, PlantSAM is not only a segmentation module but also a preprocessing stage that reallocates image resolution from background to diagnostically relevant plant structure [2507.16506].

## 3. Zero-shot and prompt-driven predecessors

Several earlier SAM-based plant pipelines established the design patterns later associated with PlantSAM. "Leaf Only SAM: A Segment Anything Pipeline for Zero-Shot Automated Leaf Segmentation" [2305.09418] uses SAM in automatic mask generation mode with a 32×32 grid of points and multi-level crops, then applies four post-processing stages: a green-pixel filter in HSV space with $35 \leq \bar{H} \leq 75$ and $\bar{S} > 35$, removal of whole-plant masks via $\text{IoU}(M_j,U)>0.9$, a shape filter based on $r_j = A_j / A^\text{circ}_j$ with threshold 0.1, and overlap-based pruning of multi-leaf composites using $\bar{S}_j > 1.5$ and 90% containment rules. On a potato leaf dataset, Leaf Only SAM attains AR$_{0.75}$ = 63.2, AP$_{0.75}$ = 60.3, AR = 58.4, AP = 56.4, and DSC = 0.700, while a fine-tuned Mask R-CNN reaches AR$_{0.75}$ = 78.7, AP$_{0.75}$ = 74.7, AR = 69.8, AP = 65.4, and DSC = 0.871. The paper explicitly presents the SAM-based system as zero-shot with respect to the target potato dataset.

A later development is "Unlocking Zero-Shot Plant Segmentation with Pl@ntNet Intelligence" [2510.12579], which combines PlantNet, its DinoV2 backbone, and SAM for agricultural imagery. The abstract states that specialized plant representations are used to identify plant regions and produce coarse segmentation masks, which are then refined by SAM into detailed segmentations. The method is evaluated on four publicly available datasets and reports consistent performance gains when using Plantnet-fine-tuned DinoV2 over base DinoV2, measured by the Jaccard Index (IoU). Because the provided details note that the actual paper text was unavailable, no architecture, dataset names, or numerical IoU values beyond the abstract can be reconstructed faithfully. Even so, the abstract clearly situates PlantSAM-like work at the intersection of foundation segmentation and plant-specialized representation learning [2510.12579].

Together, these studies show two recurrent mechanisms. One is prompt-free or weakly prompted zero-shot segmentation followed by plant-specific filtering, as in Leaf Only SAM. The other is plant-aware localization followed by SAM refinement, as in the Pl@ntNet-based zero-shot approach. This suggests that PlantSAM, in the broad methodological sense, is less a single architecture than a prompt-and-prior paradigm for converting generic segmentation models into plant-centric systems.

## 4. Expansion to agriculture, remote sensing, and 3D phenotyping

The broader literature repeatedly reuses the PlantSAM pattern in different domains. In "A SAM-based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition" [2309.13578], HQ-SAM is paired with DINO and YOLOv8 detectors to generate box prompts for crop, weed, and leaf masks on PhenoBench. The best system, using HQ-SAM ViT-H with LoRA and DINO-enhanced detection, reaches PQ+ = 81.33, with IoU(soil) = 99.18, IoU(weed) = 70.66, PQ(leaf) = 73.81, PQ(crop) = 81.66, and overall PQ = 77.73. The design is explicitly detector-guided and hierarchical: semantic segmentation, plant instance segmentation, and leaf instance segmentation are all generated from prompt-conditioned SAM outputs.

In remote sensing, "Segment Anything for Satellite Imagery: A Strong Baseline and a Regional Dataset for Automatic Field Delineation" [2506.16318] adapts SAM to field-boundary extraction using LoRA on the image encoder and optionally the decoder. The best configuration applies LoRA to both encoder and decoder with rank $r=32$ and single-point prompts, achieving mAP50 = 27.0 and mAR150 = 43.3 on AI4Boundaries. The same work introduces the ERAS dataset for Emilia-Romagna and reports that direct zero-shot SAM performs poorly on field delineation, while fine-tuned SAM+LoRA outperforms Mask R-CNN across AI4B, ERAS S2, and ERAS HR. In this setting, PlantSAM refers not to whole-plant objects but to an agricultural SAM specialized for parcel delineation.

Tree-level forestry work extends the same logic with elevation data. "Assessing SAM for Tree Crown Instance Segmentation from Drone Imagery" [2503.20199] shows that out-of-the-box SAM variants do not outperform a custom Mask R-CNN for young plantations, but RSPrompter and DSMPrompter, which keep SAM frozen while learning prompt generators and DSM encoders, surpass the baseline. The best DSMPrompter model reaches single-class mAP 65.03, mIoU 83.24, multi-class mAP 54.40, and weighted mAP 64.84. "Bringing SAM to new heights: Leveraging elevation data for tree crown segmentation from drone imagery" [2506.04970] generalizes this across boreal plantations, temperate forest, and tropical forest, and introduces BalSAM, which fuses RGB and DSM embeddings as
$$
E_{\text{fusion}} = E_{RGB} + E_{DSM}.
$$
On Quebec plantations, BalSAM achieves single-class mAP 65.03, mIoU 83.24, multi-class mAP 54.40, and weighted mAP 64.84, outperforming Mask R-CNN+DSM on multi-class metrics. The same paper emphasizes that zero-shot SAM remains inadequate for automatic crown mapping, even when DSM-based point prompts are used [2506.04970].

In orchards, "Olive Tree Satellite Image Segmentation Based On SAM and Multi-Phase Refinement" [2508.20954] uses SAM Automatic Mask Generator, row-and-column inference, SAM Predictor prompts at grid intersections, IQR-based area filtering, and cosine-similarity shape constraints. It reports an increase from 82.928% phase-1 count accuracy to 98.564% overall accuracy across five Tunisian olive plots. Here the central prior is agronomic regularity: rows and columns are inferred from initial SAM detections, and missing trees are then segmented with point or point+box prompts.

A 3D phenotyping extension appears in "IPENS: Interactive Unsupervised Framework for Rapid Plant Phenotyping Extraction via NeRF-SAM2 Fusion" [2505.13633]. IPENS combines SAM2 video segmentation with NeRF-based inverse rendering to obtain organ-level 3D masks and point clouds for rice and wheat. It reports grain-level segmentation mIoU of 63.72% on rice and 89.68% on wheat, while supporting downstream trait estimation such as rice grain volume with $R^2 = 0.7697$ and wheat spike volume with $R^2 = 0.9956$. The formulation introduces a 3D voxel mask field rendered along rays as
$$
M(r) = \int_{t_n}^{t_f} w(r(t))\, G(r(t))\, dt,
$$
followed by optimization against SAM2 masks. This is a different modality from 2D herbarium segmentation, but it preserves the same PlantSAM logic: promptable foundation segmentation coupled to plant-specific geometry [2505.13633].

## 5. Design principles across PlantSAM systems

Across these studies, several architectural principles recur. The first is detector- or representation-guided prompting. In herbarium PlantSAM, YOLOv10 provides SAM2 box prompts [2507.16506]. In crop–weed panoptic segmentation, DINO and YOLOv8 serve as box generators for HQ-SAM [2309.13578]. In tree-crown studies, RSPrompter, DSMPrompter, and BalSAM learn prompts in embedding space, often while SAM itself remains frozen [2503.20199], [2506.04970]. In olive orchards, grid intersections act as point prompts for the SAM Predictor [2508.20954]. In IPENS, prompts are manual or YOLOv11-derived, then propagated by SAM2 across a video sequence [2505.13633].

The second principle is multimodal or prior-driven refinement. PlantNet-fine-tuned DinoV2 provides plant-specialized representations before SAM refinement in zero-shot agricultural segmentation [2510.12579]. DSM or elevation data are incorporated either as point-prompt generators, fourth-channel inputs to R-CNN backbones, or dedicated encoders fused with SAM embeddings [2503.20199], [2506.04970]. In orchards, spatial alignment and shape consistency act as agronomic priors [2508.20954]. In Leaf Only SAM, color, extent, shape, and overlap filters replace learned priors with hand-crafted ones [2305.09418].

A third principle is modularity rather than end-to-end retraining. Herbarium PlantSAM fine-tunes YOLOv10 and SAM1/SAM2 directly [2507.16506], but much of the broader PlantSAM literature keeps SAM frozen and learns only LoRA modules, prompt generators, side encoders, or task-specific detection heads [2506.16318], [2503.20199], [2506.04970]. This suggests that the appeal of PlantSAM lies partly in parameter-efficient adaptation under scarce annotation.

The fourth principle is that segmentation quality is often judged not only by IoU or Dice but by downstream plant-science utility. Herbarium PlantSAM improves five trait-classification tasks [2507.16506]. IPENS turns SAM2-derived masks into surface area, volume, length, and width estimates [2505.13633]. Olive-tree segmentation is evaluated by count accuracy rather than pixel overlap because tree counting is the stated operational goal [2508.20954]. This suggests that PlantSAM should be understood as an intermediate representation layer in larger phenotyping, biodiversity, or agronomic pipelines.

## 6. Limitations, misconceptions, and future directions

A common misconception is that SAM can simply be deployed in automatic mode for plant imagery. Multiple studies contradict this. Leaf Only SAM remains below a fine-tuned Mask R-CNN on potato leaf phenotyping [2305.09418]. Out-of-the-box SAM for tree crowns yields single-class mAP values around 8–11 and does not outperform custom Mask R-CNN baselines [2503.20199], [2506.04970]. Field delineation papers likewise report poor zero-shot performance before LoRA adaptation [2506.16318]. The evidence therefore favors the narrower claim that SAM is a strong substrate for plant-specific adaptation, not a universally sufficient off-the-shelf plant segmenter.

Another misconception is that PlantSAM denotes a single universally accepted architecture. The literature instead shows at least three meanings. One is the named herbarium system YOLOv10 + SAM2 [2507.16506]. Another is a broad class of prompt-driven plant segmentation pipelines, often described explicitly in the supplementary details of other papers [2305.09418], [2309.13578], [2506.16318]. A third is a conceptual direction toward plant-specific foundation segmentation models, suggested by work on Pl@ntNet-specialized DinoV2, SAM fine-tuning, DSM fusion, and interactive 3D mask lifting [2510.12579], [2506.04970], [2505.13633].

The limitations are also domain-specific. Herbarium PlantSAM still encounters difficult out-of-distribution backgrounds, extremely fine structures, and imperfect ground-truth masks [2507.16506]. Leaf Only SAM depends on glasshouse imagery and hand-tuned HSV thresholds that may not transfer to field scenes [2305.09418]. Tree-crown and field-boundary systems require DSM quality, large memory budgets, and domain-specific tuning [2503.20199], [2506.16318], [2506.04970]. Olive-tree refinement assumes approximately regular orchard grids [2508.20954]. IPENS currently depends on controlled, wind-free acquisition for NeRF reconstruction [2505.13633].

The current research trajectory points in several directions. One is foundation-model specialization through plant-centric pretraining or fine-tuning, as exemplified by Pl@ntNet-based DinoV2 representations [2510.12579]. Another is multimodal PlantSAM, where RGB is augmented with DSM, radiance fields, or possibly other sensing modalities, though only DSM and NeRF-based representations are concretely documented in the provided sources [2506.04970], [2505.13633]. A third is prompt automation through detectors, learned prompters, and grid or geometric priors [2507.16506], [2503.20199], [2508.20954]. A plausible implication is that future PlantSAM systems will be less about replacing plant-specific modeling with a single general foundation model and more about integrating foundation segmentation with plant-specific structure, sensing, and task objectives.

Source: https://www.emergentmind.com/topics/plantsam