From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts

Abstract: Hyperspectral tree species classification is challenging due to limited and imbalanced class labels, spectral mixing (overlapping light signatures from multiple species), and ecological heterogeneity (variability among ecological systems). Addressing these challenges requires methods that integrate biological and structural characteristics of vegetation, such as canopy architecture and interspecific interactions, rather than relying solely on spectral signatures. This paper presents a biologically informed, semi-supervised deep learning method that integrates multi-sensor Earth observation data, specifically hyperspectral imaging (HSI) and airborne laser scanning (ALS), with expert, ecological knowledge. The approach relies on biologically inspired pseudo-labelling over a precomputed canopy graph, yielding accurate classification at low training cost. In addition, ecological priors on species cohabitation are automatically derived from reliable sources using LLMs and encoded as a cohabitation matrix with likelihoods of species occurring together. These priors are incorporated into the pseudo-labelling strategy, effectively introducing expert knowledge into the model. Experiments on a real-world forest dataset demonstrate 5.6% improvement over the best reference method. Expert evaluation of cohabitation priors reveals high accuracy with differences no larger than 15%.

• The paper presents a hybrid deep learning approach that integrates LLM-extracted ecological priors with a dual-stream network for tree species classification.
• The methodology employs a two-pass pseudo-labelling scheme on a canopy graph, combining classifier confidence, spatial proximity, and ecological regularization.
• Experimental results reveal a significant macro F1 improvement (+5.6%) over traditional methods, validating the integration of domain-specific ecological knowledge.

Knowledge-Driven Pseudo-Labelling for Tree Species Classification: Integrating LLM-Derived Ecological Priors

Introduction and Problem Context

The challenge of tree species classification from hyperspectral imaging (HSI) and airborne LiDAR (ALS) data in complex, mixed forests is exacerbated by sparse and imbalanced labels, spectral mixing at crown edges, and ecological heterogeneity. Traditional machine learning solutions for remote sensing tasks often optimize solely on spectral and structural features, failing to exploit the rich contextual biological knowledge available in scientific literature. The paper "From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts" (2604.16115) addresses the underutilization of expert ecological knowledge by proposing a hybrid deep learning and knowledge-driven approach for semi-supervised tree species classification, employing a LLM as a virtual ecological expert to derive cohabitation priors.

Dataset and Preprocessing

The study uses a unique, co-registered HSI and ALS dataset covering 158.3 km² of Wigierski National Park. Reference data comprises 942 field-surveyed trees across 16 taxa and a constructed “other” class, with spatial splits to generate training, validation, and test sets. Preprocessing workflows standardize ALS and HSI data, ensuring geometric and radiometric consistency.

Figure 1: Spatial layout of Wigierski National Park with field-sampled trees subdivided into training, validation, and test set partitions.

Methodology

Dual-Stream Neural Network Architecture

Classification is realized via a dual-stream MLP with modality-specific (HSI, ALS) encoders projecting to a joint latent space, then decoded via a fusion MLP for pixel-wise labeling. The model is trained end-to-end with cross-entropy loss and Adam optimizer. Architectural design prioritizes resilience to hyperparameter selection, facilitating robust application with limited supervision.

Semi-Supervised, Ecology-Informed Pseudo-Labelling

The central algorithmic innovation is a biologically motivated, two-pass pseudo-labelling scheme over a canopy graph. After training an initial model on limited labels, candidate pixels (tree-tops detected via CHM maxima) are assigned pseudo-labels using a fusion of classifier confidence, spatial proximity, and a context-aware cohabitation matrix that encodes prior knowledge of taxa co-occurrence. The canopy graph enables expansion from annotated trees to high-likelihood neighbors, systematically augmenting the training dataset while mitigating propagation of implausible pseudo-labels through contextual ecological regularization.

This cohabitation matrix is generated automatically, not via labor-intensive field surveys or manually encoded rules, but by mining literature and databases (notably EUNIS) with an LLM-based information extraction pipeline. The prompt instructs the LLM to return a symmetric pairwise likelihood matrix over taxa, referencing only verifiable and region-specific sources.

Figure 2: LLM-derived cohabitation matrix encoding pairwise species adjacency priors for the study area, validated through expert review.

Integration of Ecological Priors

The derived cohabitation matrix is applied in pseudo-label selection, influencing region-growing in the canopy graph conditional on both spatial proximity and the ecological plausibility of co-occurrence. This approach directly injects structured botanical knowledge into the training signal for the deep neural classifier, enabling the model to better leverage unlabelled data under strong class imbalance and presence of rare or niche taxa.

Experimental Results

The proposed dual-stream DNN with cohabitation-aware pseudo-labelling (DSNN+P) exhibits substantial performance gains over traditional pipelines and even over feature-engineered multi-modal classical models (e.g., CatBoost using MNF-IND-ALS features).

• Macro F1: $79.38\%$ (DSNN+P) vs. $73.78\%$ (best classical; improvement: $+5.6\%$)
• Accuracy: $90.09\%$
• Balanced accuracy (mean recall): $81.87\%$

The advantage is even more pronounced against raw HSI ($+37.89\%$ macro F1) and classical models without ALS. Pseudo-labelling on the canopy graph yields a further $1.96\%$ improvement over the base DSNN alone.

Per-class F1 analysis reveals high recall across 12 of 16 taxa, though certain classes (e.g., Carpinus betulus, Acer platanoides) are subject to high confusion, particularly with rare or morphologically similar species. Case studies demonstrate that DSNN+P provides site-specific maps more consistent with field observations and expert knowledge than purely spectral models.

Figure 3: Tree taxa classification maps for the full study region contrasting MNF-IND-ALS and DSNN+P outputs, highlighting divergence in complex stands.

Figure 4: Detailed comparison of classification results and ground validation in areas of maximal disagreement, with field photograph reference points.

Figure 5: Distribution of area by taxa for maps derived using MNF-IND-ALS and DSNN+P, indicating close matching with notable differences in specific classes.

Figure 6: Average confusion matrix for DSNN+P over test splits, illustrating class-specific precision and confusion patterns.

Evaluation of LLM-Generated Ecological Priors

Domain experts reviewed the LLM-generated cohabitation matrix, finding maximal corrections of only ±0.15 in affinity scores, and confirming that key habitat compatibilities were captured. Most adjustments targeted genus-level classes or circumstances requiring finer habitat stratification. Crucially, the LLM-based system produced plausibility-consistent priors without requiring field data, demonstrating high utility for deployment in sparsely sampled regions.

Implications, Limitations, and Future Prospects

The research substantiates the strong claim that automated extraction of ecological priors from the scientific corpus via LLMs can supplant manual expert rule construction and expensive field campaigns for context-aware semi-supervised learning—without notable sacrifice in ecological fidelity. The integration of these priors with deep classifiers advances not only pixel-level accuracy but also interpretability, especially under label-limited, class-imbalanced scenarios.

This paradigm is generalizable to other remote sensing applications where domain knowledge exists but is not codified for ML pipelines, and where unlabelled data vastly outnumbers labelled exemplars.

There are, however, residual challenges: class aggregation at genus or functional-group levels can blur cohabitation relationships, and the translation of literature-derived probabilities to spatial priors remains sensitive to region and habitat definition granularity. For broader AI, the methodological precedent is clear: LLMs can efficiently operationalize domain knowledge for weak supervision and context modelling, offering scalable mechanisms for knowledge injection into deep learning frameworks.

Conclusion

The integration of LLM-derived ecological priors with dual-stream DNNs and structured pseudo-labelling yields material gains in remote tree species classification, particularly in challenging, label-scarce, ecologically heterogeneous environments. This advances the automation, scalability, and contextual interpretability of remote sensing classification systems, suggesting new research avenues for coupling structured knowledge extraction with semi-supervised representation learning in environmental informatics and beyond.