---
title: 'LeafTrackNet: Leaf-Level Tracking Framework'
url: https://www.emergentmind.com/topics/leaftracknet
type: topic
---

# LeafTrackNet: Leaf-Level Tracking Framework

Searching arXiv for LeafTrackNet and closely related plant leaf tracking papers to ground the article in the relevant literature.
LeafTrackNet is a leaf-level multi-object tracking framework for top-down RGB plant phenotyping, introduced together with the CanolaTrack benchmark for canola (*Brassica napus*) [2512.13130]. It is designed to maintain the identities of individual leaves across long time intervals in realistic growth sequences, where leaves emerge, grow, overlap, disappear, reappear, and undergo large orientation changes caused by pot rotation. The framework combines a YOLOv10-based leaf detector, a MobileNetV3-based embedding network, and an embedding-based memory association strategy, thereby separating localization from identity matching and avoiding reliance on motion prediction [2512.13130].

## 1. Problem setting and scope

LeafTrackNet addresses a regime in which standard MOT assumptions are systematically violated. The target scene is a dynamic biological system rather than a conventional video sequence with dense temporal continuity. The reported difficulties include non-rigid leaf growth, self-occlusion, the emergence of new leaves, the senescence of older leaves, strong visual similarity among leaves, pot rotation, and temporal gaps of about 24 hours between observations [2512.13130].

The framework is motivated by two limitations identified in prior work. Existing plant-specific tracking methods are described as typically limited to small-scale species or constrained imaging conditions, while generic MOT methods such as ByteTrack, BoT-SORT, and MOTRv2 are not designed for dynamic biological scenes and assume more stable motion and geometry [2512.13130]. A plausible implication is that identity maintenance in plant phenotyping requires appearance- and structure-based association mechanisms that are tolerant to growth and reconfiguration rather than extrapolative motion models.

The task focus also differs from earlier plant video analysis formulations. An earlier fluorescence-video framework posed a joint multi-leaf segmentation, alignment, and tracking problem for rosette plants such as *Arabidopsis thaliana*, with explicit template transformation and optimization over leaf structure and identity [1505.00353]. LeafTrackNet instead targets top-down RGB canola sequences and formulates tracking through detection, embedding, and memory-based association [2512.13130].

## 2. CanolaTrack benchmark

LeafTrackNet is introduced with CanolaTrack, a benchmark comprising 5,704 RGB images with 31,840 annotated leaf instances spanning the early growth stages of 184 canola plants [2512.13130]. The images are top-down RGB acquisitions at 1200 × 1200 resolution, collected over 31 consecutive days with about 24 hours between successive images per plant. The train/test split is 147 plants, 4,557 images, and 25,485 leaves for training, and 37 unseen plants, 1,147 images, and 6,355 leaves for testing [2512.13130].

The dataset is described as the largest dataset for leaf tracking in agriculture crops and the largest high-resolution dataset for leaf tracking in crops to date [2512.13130]. Its stated significance is not only scale but realism: CanolaTrack contains leaf birth, leaf death, occlusion, reoccurrence, non-uniform growth, and pot rotation. The inclusion of rotation is emphasized as uncommon in previous datasets [2512.13130].

The benchmark is positioned against smaller or more specialized predecessors. The paper contrasts it with LeTra, KOMATSUNA, MSU-PID, and PhenoTrack3D, and earlier fluorescence-video work operated on 41 Arabidopsis videos over a 5-day growth period with 1,807 labeled leaves in total [1505.00353]. This suggests that CanolaTrack shifts the evaluation regime toward longer-term, RGB-only, top-down crop phenotyping under realistic acquisition conditions [2512.13130].

## 3. Model architecture

LeafTrackNet is an efficient two-stage tracking framework. First, a fine-tuned YOLOv10 detector localizes leaves in each image. Second, a MobileNetV3-based embedding network maps each detected leaf crop into a learned identity space. During inference, a memory bank stores track prototypes and performs association using cosine similarity and Hungarian assignment [2512.13130].

For frame \(t\), the detector outputs
\[
\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},
\]
with each detection parameterized as
\[
\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).
\]
Detections with confidence below \(0.5\) are discarded [2512.13130].

Each detected or ground-truth leaf crop is resized to \(224 \times 224\) and embedded as
\[
\mathcal{E}(x) = \mathcal{F}_\phi(\mathcal{N}_\theta(x)),
\]
where \(\mathcal{N}_\theta\) is a MobileNetV3 backbone pretrained on ImageNet and truncated before classification, and \(\mathcal{F}_\phi\) is a linear projection layer. The output \(\mathcal{E}(x) \in \mathbb{R}^D\) is the leaf embedding. The paper reports that MobileNetV3 provides the best tradeoff between accuracy and efficiency among the evaluated backbones [2512.13130].

At time \(t\), the active tracks are represented as
\[
\mathcal{T}^t = \{(p_\ell^t, a_\ell^t)\}_{\ell=1}^{N_t},
\]
where \(p_\ell^t\) is the prototype embedding for track \(\ell\) and \(a_\ell^t\) is an age counter recording how many consecutive frames the track has been absent [2512.13130]. This architecture makes identity a property of an evolving prototype rather than of instantaneous bounding-box overlap.

## 4. Training objective and inference procedure

Training is based on crop extraction and triplet supervision. Given image \(I_k^t\) and leaf box \(b_{k,i}^t\), the crop-and-resize operator \(\psi\) produces
\[
x_{k,i}^t = \psi(I_k^t, b_{k,i}^t) \in \mathbb{R}^{3 \times W \times H}.
\]
Triplets are sampled as an anchor \(x_a\), a positive \(x_p\) corresponding to the same leaf at a different time, and a negative \(x_n\) corresponding to a different leaf from the same plant:
\[
(x_a, x_p, x_n) = (x_{k,i}^{t_a}, x_{k,i}^{t_p}, x_{k,j}^{t_n}), \quad j \neq i,\; t_p \neq t_a.
\]
The embedding model is optimized with triplet margin loss,
\[
\mathcal{L}(x_a, x_p, x_n) = \max\left\{0,\; \lVert e_a - e_p \rVert_2^2 - \lVert e_a - e_n \rVert_2^2 + m \right\},
\]
where \(m = 0.3\) and \(e_a, e_p, e_n\) are the corresponding embeddings [2512.13130].

The reported training details are: Adam optimizer, learning rate \(10^{-4}\), weight decay \(10^{-5}\), crop size \(224 \times 224\), batch size 48, maximum 80 epochs, 4 NVIDIA Tesla V100S GPUs, and early stopping [2512.13130]. In ablations, the paper evaluates cross-plant flexible, intra-plant full-cycle, and intra-plant temporal window triplet sampling strategies.

Inference does not use motion prediction. At \(t=1\), every detection initializes a new track:
\[
\mathcal{T}^{1} = \{(p_j^1 = e_j^1,\; a_j^1=0)\}_{j=1}^{N_1}.
\]
For later times, each detection embedding \(e_j^t\) is compared with the previous prototypes \(p_\ell^{t-1}\) using cosine similarity,
\[
S_{\ell j} = (p_\ell^{t-1})^\top e_j^t,
\]
yielding a similarity matrix \(S \in \mathbb{R}^{N_{t-1}\times N_t}\) and cost matrix \(C = 1-S\). The Hungarian algorithm solves
\[
\pi^* = \arg\min_{\pi}\sum_{\ell=1}^{N_{t-1}}\left(1 - (p_\ell^{t-1})^\top e_{\pi(\ell)}^t\right).
\]
A match is accepted only when \(S_{\ell,\pi^*(\ell)} \ge \tau_s\) with \(\tau_s = 0.4\) [2512.13130].

When a match is accepted, the prototype is updated by exponential moving average,
\[
p_\ell^t = \alpha p_\ell^{t-1} + (1-\alpha)e_j^t,\quad a_\ell^t = 0,
\]
with \(\alpha = 0.5\). Unmatched detections initialize new tracks, unmatched tracks retain their previous prototype and increment age, and tracks are removed when \(a_\ell^t > \tau_a\) with \(\tau_a = 5\) [2512.13130]. The stated rationale is that embeddings capture leaf appearance and structural cues, memory preserves identity through temporary occlusion, and the absence of rigid motion assumptions improves robustness to rotation and growth.

## 5. Evaluation and empirical performance

Evaluation uses an 80/20 train/test split by plants, TrackEval, and the standard MOT metrics HOTA, DetA, AssA, MOTA, and IDF1 [2512.13130]. The baselines are BoT-SORT, ByteTrack, MOTRv2, LeTra, and Plant-Doctor.

On CanolaTrack, the reported results are: BoT-SORT at HOTA 33.32, DetA 91.30, AssA 12.18, MOTA 40.35, and IDF1 26.13; ByteTrack at 33.58, 91.94, 12.29, 41.88, and 26.20; MOTRv2 at 78.30, 77.33, 79.36, 79.68, and 83.78; LeTra at 67.02, 82.03, 54.98, 82.09, and 69.06; Plant-Doctor at 59.74, 74.42, 48.20, 79.71, and 69.56; and LeafTrackNet at 88.03, 92.25, 84.07, 93.64, and 92.90, respectively [2512.13130]. The improvement over the strongest competing method is reported as +9.73 HOTA, +0.31 DetA, +4.71 AssA, +11.55 MOTA, and +9.12 IDF1, with the 9% HOTA improvement highlighted in the abstract [2512.13130].

Ablation studies attribute several design choices to these results. In backbone comparisons among MobileNetV3, ResNet18, ResNet34, ResNet50, ResNet101, and ViT-B/16, MobileNetV3 performs best overall for identity-related metrics while using 2.97M parameters and 0.23 GMACs [2512.13130]. In triplet sampling, cross-plant flexible yields the best HOTA at 88.30, intra-plant full-cycle yields 88.03, and temporal-window sampling degrades markedly for small or very large windows. For inference hyperparameters, HOTA is stable for \(\tau_s \in [0.4, 0.6]\), too low \(\tau_s\) produces noisy associations, too high \(\tau_s\) over-prunes tracks, and moderate smoothing with \(\alpha = 0.5\) is effective [2512.13130].

The qualitative analysis emphasizes two difficult scenarios: heavy occlusion and pot rotation of about \(90^\circ\). In both settings, the paper reports that LeafTrackNet preserves identities more reliably than the baselines, and accompanying t-SNE plots and heatmaps indicate that same-leaf embeddings cluster tightly over 31 days while different leaves remain separated [2512.13130].

## 6. Position within the literature, misconceptions, and limitations

LeafTrackNet occupies a specific position between plant-specific tracking and generic MOT. The paper argues that plant-specific trackers such as LeTra and Plant-Doctor still struggle on complex crops such as canola, while generic methods like BoT-SORT, ByteTrack, and MOTRv2 rely on assumptions of smooth motion, rigid trajectories, or strong spatial continuity that fail under growth, overlap, and rotation [2512.13130]. In contrast, LeafTrackNet uses leaf-centric embeddings, cosine-similarity association, and a memory bank updated by EMA.

Its relation to earlier plant tracking work is complementary rather than identical. The fluorescence-video framework of 2015 formulated leaf analysis as joint multi-leaf segmentation, alignment, and tracking using Chamfer matching, pseudo-Boolean local search, gradient-descent tracking, and candidate birth/death management [1505.00353]. LeafTrackNet does not adopt that template-driven optimization; instead, it separates detection from identity matching and applies a deep embedding model to top-down RGB canola imagery [2512.13130].

A recurrent misconception is to treat LeafTrackNet as interchangeable with lightweight object-tracking systems for generic visual tracking. It should not be confused with LightTrack, which uses one-shot NAS to design lightweight Siamese object trackers for resource-constrained deployment [2104.14545]. LeafTrackNet denotes the plant-phenotyping framework introduced with CanolaTrack [2512.13130].

The paper does not present an extensive limitations section, but it suggests future directions including cross-species transferability, field-level deployment, and real-time agricultural decision-making [2512.13130]. Implicit limitations listed in the source are the focus on top-down RGB imaging only, evaluation on a single crop species, the need to tune thresholds and update rules in the memory-based association scheme, and the absence of explicit motion modeling, which may matter in some settings [2512.13130].

Source: https://www.emergentmind.com/topics/leaftracknet