SilvaScenes: Under-canopy Tree Segmentation

• SilvaScenes is a computer vision benchmark for instance segmentation and species classification, capturing the challenges of under-canopy views in natural forests.
• It comprises 172 in-situ annotated RGB images from five bioclimatic domains in Quebec, with 1476 tree instances spanning 24 species.
• The dataset addresses issues like heavy occlusion, variable lighting, and intra-species variation, and evaluates both CNN and transformer-based models.

Searching arXiv for the specified paper and closely related forestry perception datasets to ground the article. arxiv_search(query="(Duclos et al., 10 Oct 2025) OR SilvaScenes", max_results=5) SilvaScenes is a computer-vision dataset and benchmark for tree instance segmentation and fine-grained species classification from under-canopy images in natural forests. It was introduced to address a specific perception regime that is central to forest robotics but poorly represented in earlier datasets: ground-level observation of trees in dense, unmanaged stands, where individual trunks must be detected and assigned species labels despite heavy occlusion, variable illumination, cluttered understory vegetation, and substantial intra-species appearance variation (Duclos et al., 10 Oct 2025). The dataset was collected across five bioclimatic domains in Quebec, Canada, and contains 172 images with 1476 unique annotated trees spanning 24 species, with annotations produced using in-situ forestry expertise rather than image-only labeling (Duclos et al., 10 Oct 2025).

1. Problem formulation and research context

SilvaScenes targets the joint problem of detecting individual trees in natural forest scenes and assigning each detected trunk a species label. The paper formulates this as instance segmentation with class labels: each visible tree trunk is treated as an object instance, represented by a segmentation mask and one species class (Duclos et al., 10 Oct 2025). This framing is directly aligned with robotics use cases in which semantic tree instances can support biodiversity monitoring, precision forestry, autonomous harvesting, and semantic data association in SLAM.

The dataset’s central premise is that under-canopy natural forests constitute a missing regime in the existing literature. Many forestry operations, including forest inventory, harvesting, and felling, occur in situ from beneath the canopy rather than from UAV viewpoints. In that setting, perceptual difficulty is driven by trunk-to-trunk occlusion, vegetation obstruction, large lighting contrasts between bright canopy gaps and deep shadows, white-balance shifts under dense canopy, and the need to distinguish species using subtle bark and contextual cues rather than crown-level appearance (Duclos et al., 10 Oct 2025).

The paper explicitly positions SilvaScenes against several inadequacies in prior data resources. Existing datasets were described as focusing on urban trees, snowy or otherwise simplified environments, close-up bark classification that avoids detection, or a very limited number of species. SilvaScenes was created to fill that gap by combining natural under-canopy imagery, instance-level trunk segmentation, and fine-grained species labels in a single benchmark (Duclos et al., 10 Oct 2025). This suggests that the dataset is intended not merely as a classification corpus, but as a full-scene perception benchmark for embodied systems operating in complex forests.

2. Geographic coverage and image acquisition

The dataset contains 172 under-canopy RGB images acquired in June and July 2025 across Quebec, Canada, and includes 1476 unique annotated trees from 24 species (Duclos et al., 10 Oct 2025). A key design choice was the avoidance of duplicated trees across images: the collection protocol aimed not to capture the same individual tree more than once, whether labeled or unlabeled, in order to prevent data leakage and artificially inflated benchmark performance.

SilvaScenes spans five bioclimatic domains of Quebec: Sugar Maple–Bitternut Hickory, Sugar Maple–Basswood, Sugar Maple–Yellow Birch, Balsam Fir–Yellow Birch, and Balsam Fir–White Birch (Duclos et al., 10 Oct 2025). These domains range from southern temperate deciduous-rich forests to southern boreal conifer-dominated forests. Data were collected at 11 sites, with deliberate balancing of inter-domain and intra-domain diversity. Most species appear in multiple sites and multiple bioclimatic domains, increasing both environmental diversity and intra-species appearance diversity.

Image acquisition was handheld and mainly off-trail. The paper explains this choice in two ways: autonomous off-trail navigation remains difficult, and off-trail acquisition better reflects the natural complexity that field robots must ultimately handle (Duclos et al., 10 Oct 2025). The camera was a Fujifilm GFX 100S with a 43.8 mm × 32.9 mm sensor and a native resolution of 102 MP. The lens was a Fujifilm GF23mmF4 R LM WR with a 99.9° diagonal field of view, selected to balance wide spatial coverage with low radial distortion. Typical capture settings were around $f/6.4$ aperture and $1/50$ s shutter speed, intended to provide extended depth of field, low blur and noise, and adequate exposure.

Although the source imagery was very high resolution, images were downsampled to 1.6 MP for modeling because current deep networks scale poorly with image resolution (Duclos et al., 10 Oct 2025). The paper later shows that resolution materially affects performance, which is notable because species recognition depends heavily on fine bark texture and subtle appearance cues that are partially suppressed by downsampling.

The visual statistics reported in the paper further characterize the dataset’s scene complexity. The number of trees per image follows an approximately Gaussian distribution with a median of 8 trees per image, and the number of distinct species per image also follows a roughly Gaussian distribution with a median of 4 species per image (Duclos et al., 10 Oct 2025). Tree widths vary strongly and follow a distribution close to inverse exponential when measured as the median trunk width across image height. The annotation threshold required a median width of at least 16 px in the released images.

3. Taxonomy and annotation protocol

SilvaScenes adopts a fine-grained taxonomy with 24 species labels, divided into deciduous and coniferous groups, plus an Unknown label for trees that cannot be reliably identified because of heavy damage, disease, or death (Duclos et al., 10 Oct 2025). In the benchmark protocol, four species with fewer than ten instances were merged with Unknown into an “Other” class, reducing the effective benchmark taxonomy to 21 classes.

The species listed in the paper are as follows.

Group	Species
Deciduous	Betula alleghaniensis — Yellow Birch (BBA); Betula papyrifera — White Birch (BBP); Ostrya virginiana — Ironwood (BOV); Fagus grandifolia — American Beech (FFG); Quercus rubra — Red Oak (FOR); Carya cordiformis — Bitternut Hickory (JCC); Tilia americana — Basswood (MTA); Fraxinus americana — White Ash (OFA); Prunus grandidentata — Largetooth Aspen (SPG); Populus tremuloides — Trembling Aspen (SPT); Acer pensylvanicum — Striped Maple (SAP); Acer rubrum — Red Maple (SAR); Acer saccharum — Sugar Maple (SAC); Ulmus americana — White Elm (UUA)
Coniferous	Thuja occidentalis — Eastern White-Cedar (CTO); Abies balsamea — Balsam Fir (PAB); Larix laricina — Tamarack (PLL); Picea glauca — White Spruce (PPG); Picea mariana — Black Spruce (PPM); Picea rubens — Red Spruce (PPR); Pinus strobus — Eastern White Pine (PPS); Tsuga canadensis — Eastern Hemlock (PTC)

Annotation quality is a defining property of SilvaScenes. The paper states that “ground truth for most of the data was obtained in situ by a forestry expert,” who could use bark, leaves, shoots, cones, shape, and environmental cues while physically present in the forest (Duclos et al., 10 Oct 2025). This is materially different from post hoc labeling from image crops alone and is especially important because reliable species identification in natural forests is difficult even for human observers.

The annotations include per-instance class labels and segmentation masks for individual trees, with masks restricted to trunks rather than branches or foliage (Duclos et al., 10 Oct 2025). The paper justifies this restriction on the basis that branches and canopy are harder to annotate and less necessary for operations such as felling and harvesting. The annotation rules are explicit:

1. Species labels are assigned with in-situ forestry expertise.
2. Only trunks are segmented.
3. Trees are labeled if most of the trunk is visible.
4. Occluded trunk segments may still be labeled if their shape can be reasonably inferred, for example behind small branches or light foliage, but not when the trunk is overlapped by another trunk.
5. If a trunk forks below breast height ($1.3$ m), each section is treated as a separate tree, consistent with Canadian Forest Service guidelines.
6. A tree is annotated only if its median width across height is at least 16 px in the released images.
7. Trees that cannot be identified reliably are assigned to Unknown.

The paper does not specify the serialized annotation file format, but it notes that the task permits non-contiguous masks and that the Ultralytics YOLO implementation was modified to support non-contiguous segmentation masks (Duclos et al., 10 Oct 2025). This strongly indicates that a single trunk instance may be represented by disconnected visible regions when partial occlusion is present.

4. Tasks, evaluation protocol, and benchmarked models

SilvaScenes supports two principal tasks: tree instance segmentation, where the objective is to segment each trunk instance, and joint tree instance segmentation with species classification, where each predicted instance must also be assigned the correct species label (Duclos et al., 10 Oct 2025). The latter is the primary benchmark task and the more difficult of the two.

For ablation, the paper defines a class-agnostic tree instance segmentation setting denoted “Binary” and a full species-aware instance segmentation setting denoted “Multi” (Duclos et al., 10 Oct 2025). In the notation used for the resolution study, “$A \rightarrow B$” means training on task $A$ and evaluating on task $B$. Thus, Binary $\rightarrow$ Binary corresponds to tree-only training and tree-only evaluation; Multi $\rightarrow$ Multi corresponds to species-aware training and species-aware evaluation; and Multi $\rightarrow$ Binary corresponds to training on the multi-class task while collapsing predictions to tree-versus-background at evaluation time.

Because the dataset is relatively small and class-imbalanced, all experiments use stratified five-fold cross-validation (Duclos et al., 10 Oct 2025). Images are automatically partitioned into five folds while attempting to ensure that each fold contains about 20% of each species’ trees. To make such stratification feasible, species with fewer than ten trees are merged into Other. Metrics are reported as macro average percentages across classes, a deliberate choice intended to avoid domination by common species and to reflect performance on fine-grained species recognition under imbalance.

The benchmark includes modern instance segmentation architectures from both CNN-based and transformer-based families (Duclos et al., 10 Oct 2025):

Model	Variant(s)
YOLOv11	Small, X-Large
YOLOv12	Small, X-Large
Mask2Former	Swin-Small, Swin-Large

All models were implemented in PyTorch and initialized from COCO pretraining for instance segmentation (Duclos et al., 10 Oct 2025). YOLO models used official Ultralytics implementations modified to handle non-contiguous masks, whereas Mask2Former used the HuggingFace implementation. Each architecture used its native augmentation pipeline. Hyperparameters were tuned separately for each experiment through Bayesian hyperparameter search. To mitigate class imbalance, the authors replaced Mask2Former’s standard cross-entropy classification loss with focal loss, matching YOLO’s use of focal loss. The paper does not provide the exact loss hyperparameters, learning rates, or augmentation coefficients.

Evaluation uses AP and AR for instance segmentation, with AP summarized as $\mathrm{mAP}_{50:95}$ over IoU thresholds $1/50$0, together with $1/50$1, $1/50$2, and $1/50$3 (Duclos et al., 10 Oct 2025). Classification is evaluated using accuracy and F1-score at IoU threshold $1/50$4. The paper provides the standard reconstruction of these metrics:

$1/50$5

$1/50$6

$1/50$7

$1/50$8

$1/50$9

These definitions reflect standard COCO-style evaluation, but the paper’s emphasis is on macro averaging across species rather than instance-frequency-weighted summaries.

5. Quantitative performance and empirical difficulty

The benchmark results show a pronounced gap between tree detection and species discrimination. The paper’s headline observation is that tree segmentation is relatively easy, whereas species classification remains difficult (Duclos et al., 10 Oct 2025). The abstract reports a top mean average precision of $1.3$0 for segmentation and only $1.3$1 for species classification. The latter value corresponds exactly to the best multi-class $1.3$2 obtained in the main benchmark.

The best overall model is Mask2Former with a Swin-Large backbone. Its reported performance is $1.3$3, $1.3$4, $1.3$5, $1.3$6, accuracy $1.3$7, and F1-score $1.3$8 (Duclos et al., 10 Oct 2025). The model has 216.0 M parameters, 868.0 B FLOPs, and runs at 4.7 FPS on an NVIDIA RTX 4090 with BF16 mixed precision, including pre- and post-processing.

The remaining models establish the accuracy-efficiency trade-off explored by the benchmark.

Model	$1.3$9	Accuracy	FPS
Mask2Former Swin-Small	$A \rightarrow B$0	$A \rightarrow B$1	7.0
YOLOv11 Small	$A \rightarrow B$2	$A \rightarrow B$3	57.7
YOLOv11 X-Large	$A \rightarrow B$4	$A \rightarrow B$5	33.0
YOLOv12 Small	$A \rightarrow B$6	$A \rightarrow B$7	51.8
YOLOv12 X-Large	$A \rightarrow B$8	$A \rightarrow B$9	20.6
Mask2Former Swin-Large	$A$0	$A$1	4.7

Two conclusions are stated explicitly in the paper. First, transformer-based Mask2Former provides the best joint segmentation-and-classification performance, which the authors interpret as evidence that global context and stronger mask modeling help in cluttered forest scenes (Duclos et al., 10 Oct 2025). Second, YOLOv12 consistently outperforms YOLOv11 on AP metrics, and the authors interpret this as evidence that attention mechanisms are beneficial in forestry contexts. At the same time, the YOLO models are substantially faster and more parameter-efficient, making them attractive for real-time robotics.

The resolution ablation further clarifies the difficulty structure of the problem. Starting from 1.6 MP images, the authors downsample by factors of two to 0.1 MP and evaluate Mask2Former Swin-Large on Binary and Multi settings (Duclos et al., 10 Oct 2025). The paper does not list the full numeric curve, but it reports several qualitative trends: performance increases consistently with image resolution; in the Multi $A$2 Multi setting, $A$3 follows an approximate power law, increasing by about 6 percentage points each time resolution is doubled; Binary $A$4 Binary achieves higher $A$5 than Multi $A$6 Multi; and Multi $A$7 Binary performs worse than Binary $A$8 Binary, though it scales better with increased resolution. A plausible implication is that class-aware representations may become more useful for generic tree segmentation when sufficient spatial detail is preserved.

6. Error structure, relation to earlier datasets, and limitations

The paper’s confusion analysis indicates that broad deciduous-versus-coniferous discrimination is not the central bottleneck. Using Mask2Former Swin-Large, confusion between deciduous and coniferous species accounts for only 8% of errors (Duclos et al., 10 Oct 2025). Instead, errors concentrate within visually similar groups. Many deciduous species are misclassified as red maple (Acer rubrum, SAR) or sugar maple (Acer saccharum, SAC), which the paper attributes to both prevalence and bark variability across age and environment. Balsam fir (Abies balsamea, PAB), the most abundant species, is also over-predicted, again plausibly reflecting class imbalance. The spruce species—white spruce, black spruce, and red spruce—show high intra-genus confusion, which is unsurprising in lower-resolution under-canopy scenes where bark-only distinctions are subtle. The Other class is especially difficult because it aggregates rare species and unknowns, behaving partly like an open-set recognition problem.

Qualitative analysis supports these observations. The model often handles heavy occlusion reasonably well and can detect partially hidden trunks (Duclos et al., 10 Oct 2025). It also sometimes predicts trees that were not included in the ground-truth annotations, which may occur when the model detects trunks excluded by the visibility, width, or identifiability rules. Conversely, some clearly visible trees are missed, possibly because of unusual stand structure or atypical trunk arrangements. The paper also notes white-balance and color-bleeding effects under dense canopy.

The comparison to prior datasets is a central part of SilvaScenes’ contribution. Against urban-tree datasets such as TDoUS and Auto Arborist, SilvaScenes provides natural-forest imagery rather than street-view or maintained environments, where visibility is typically higher and competition is lower (Duclos et al., 10 Oct 2025). Against close-up bark datasets such as BarkNet and CentralBark, it adds the detection and localization problem, which bark-only crops bypass. Against natural-forest segmentation datasets such as FinnWoodlands, it provides fine-grained species labels under visually complex summer conditions rather than broad genera in snowy trail scenes. The paper further states that the closest prior species-aware under-canopy dataset had only four visually distinct species, whereas SilvaScenes includes 24 species, roughly six times more than previous comparable under-canopy work.

The limitations are presented candidly. SilvaScenes remains modest in scale, with 172 images and 1476 trees, despite its annotation quality and taxonomic diversity (Duclos et al., 10 Oct 2025). Rare species are underrepresented, which forces category merging during benchmarking. Downsampling from 102 MP to 1.6 MP likely suppresses fine-grained bark information. Even the best model reaches only $A$9 $B$0 for species-aware instance segmentation, indicating that the benchmark remains open. The authors suggest future work in very-high-resolution modeling capable of exploiting 100 MP imagery or more, and in enriching labels with attributes such as tree age, size, or physiological state to reduce confusion. They also propose studying whether semantic information from datasets like SilvaScenes improves semantic data association in forest SLAM under dense natural conditions (Duclos et al., 10 Oct 2025).

For practical research use, the paper states that the dataset and source code will be available at the NORLAB-Ulaval GitHub repository, but it does not specify an exact software or data license in the provided text (Duclos et al., 10 Oct 2025). Overall, SilvaScenes is best understood as a benchmark designed to expose a specific gap in forest perception: trunk detection under natural clutter is already tractable, but fine-grained species attribution from under-canopy imagery remains a difficult and unresolved problem.