LeafTrackNet: Leaf-Level Tracking Framework

• LeafTrackNet is a leaf-level multi-object tracking framework that uses deep embeddings and memory-based association to track individual leaves over long-term growth sequences.
• It is validated on the CanolaTrack benchmark, the largest high-resolution dataset capturing realistic canola leaf dynamics including occlusions and pot rotation.
• The architecture combines a YOLOv10 detector and a MobileNetV3 embedding network with EMA updates to robustly manage non-rigid growth and complex leaf reconfigurations.

Searching arXiv for LeafTrackNet and closely related plant leaf tracking papers to ground the article in the relevant literature. LeafTrackNet is a leaf-level multi-object tracking framework for top-down RGB plant phenotyping, introduced together with the CanolaTrack benchmark for canola (Brassica napus) (Liu et al., 15 Dec 2025). It is designed to maintain the identities of individual leaves across long time intervals in realistic growth sequences, where leaves emerge, grow, overlap, disappear, reappear, and undergo large orientation changes caused by pot rotation. The framework combines a YOLOv10-based leaf detector, a MobileNetV3-based embedding network, and an embedding-based memory association strategy, thereby separating localization from identity matching and avoiding reliance on motion prediction (Liu et al., 15 Dec 2025).

1. Problem setting and scope

LeafTrackNet addresses a regime in which standard MOT assumptions are systematically violated. The target scene is a dynamic biological system rather than a conventional video sequence with dense temporal continuity. The reported difficulties include non-rigid leaf growth, self-occlusion, the emergence of new leaves, the senescence of older leaves, strong visual similarity among leaves, pot rotation, and temporal gaps of about 24 hours between observations (Liu et al., 15 Dec 2025).

The framework is motivated by two limitations identified in prior work. Existing plant-specific tracking methods are described as typically limited to small-scale species or constrained imaging conditions, while generic MOT methods such as ByteTrack, BoT-SORT, and MOTRv2 are not designed for dynamic biological scenes and assume more stable motion and geometry (Liu et al., 15 Dec 2025). A plausible implication is that identity maintenance in plant phenotyping requires appearance- and structure-based association mechanisms that are tolerant to growth and reconfiguration rather than extrapolative motion models.

The task focus also differs from earlier plant video analysis formulations. An earlier fluorescence-video framework posed a joint multi-leaf segmentation, alignment, and tracking problem for rosette plants such as Arabidopsis thaliana, with explicit template transformation and optimization over leaf structure and identity (Yin et al., 2015). LeafTrackNet instead targets top-down RGB canola sequences and formulates tracking through detection, embedding, and memory-based association (Liu et al., 15 Dec 2025).

2. CanolaTrack benchmark

LeafTrackNet is introduced with CanolaTrack, a benchmark comprising 5,704 RGB images with 31,840 annotated leaf instances spanning the early growth stages of 184 canola plants (Liu et al., 15 Dec 2025). The images are top-down RGB acquisitions at 1200 × 1200 resolution, collected over 31 consecutive days with about 24 hours between successive images per plant. The train/test split is 147 plants, 4,557 images, and 25,485 leaves for training, and 37 unseen plants, 1,147 images, and 6,355 leaves for testing (Liu et al., 15 Dec 2025).

The dataset is described as the largest dataset for leaf tracking in agriculture crops and the largest high-resolution dataset for leaf tracking in crops to date (Liu et al., 15 Dec 2025). Its stated significance is not only scale but realism: CanolaTrack contains leaf birth, leaf death, occlusion, reoccurrence, non-uniform growth, and pot rotation. The inclusion of rotation is emphasized as uncommon in previous datasets (Liu et al., 15 Dec 2025).

The benchmark is positioned against smaller or more specialized predecessors. The paper contrasts it with LeTra, KOMATSUNA, MSU-PID, and PhenoTrack3D, and earlier fluorescence-video work operated on 41 Arabidopsis videos over a 5-day growth period with 1,807 labeled leaves in total (Yin et al., 2015). This suggests that CanolaTrack shifts the evaluation regime toward longer-term, RGB-only, top-down crop phenotyping under realistic acquisition conditions (Liu et al., 15 Dec 2025).

3. Model architecture

LeafTrackNet is an efficient two-stage tracking framework. First, a fine-tuned YOLOv10 detector localizes leaves in each image. Second, a MobileNetV3-based embedding network maps each detected leaf crop into a learned identity space. During inference, a memory bank stores track prototypes and performs association using cosine similarity and Hungarian assignment (Liu et al., 15 Dec 2025).

For frame $t$, the detector outputs

$\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$

with each detection parameterized as

$$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$

Detections with confidence below $0.5$ are discarded (Liu et al., 15 Dec 2025).

Each detected or ground-truth leaf crop is resized to $224 \times 224$ and embedded as

$$\mathcal{E}(x) = \mathcal{F}_\phi(\mathcal{N}_\theta(x)),$$

where $\mathcal{N}_\theta$ is a MobileNetV3 backbone pretrained on ImageNet and truncated before classification, and $\mathcal{F}_\phi$ is a linear projection layer. The output $\mathcal{E}(x) \in \mathbb{R}^D$ is the leaf embedding. The paper reports that MobileNetV3 provides the best tradeoff between accuracy and efficiency among the evaluated backbones (Liu et al., 15 Dec 2025).

At time $t$, the active tracks are represented as

$\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$0

where $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$1 is the prototype embedding for track $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$2 and $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$3 is an age counter recording how many consecutive frames the track has been absent (Liu et al., 15 Dec 2025). This architecture makes identity a property of an evolving prototype rather than of instantaneous bounding-box overlap.

4. Training objective and inference procedure

Training is based on crop extraction and triplet supervision. Given image $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$4 and leaf box $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$5, the crop-and-resize operator $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$6 produces

$\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$7

Triplets are sampled as an anchor $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$8, a positive $\mathcal{D}^t = \{\hat b_i^t\}_{i=1}^{N_t},$9 corresponding to the same leaf at a different time, and a negative $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$0 corresponding to a different leaf from the same plant: $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$1 The embedding model is optimized with triplet margin loss,

$$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$2

where $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$3 and $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$4 are the corresponding embeddings (Liu et al., 15 Dec 2025).

The reported training details are: Adam optimizer, learning rate $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$5, weight decay $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$6, crop size $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$7, batch size 48, maximum 80 epochs, 4 NVIDIA Tesla V100S GPUs, and early stopping (Liu et al., 15 Dec 2025). In ablations, the paper evaluates cross-plant flexible, intra-plant full-cycle, and intra-plant temporal window triplet sampling strategies.

Inference does not use motion prediction. At $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$8, every detection initializes a new track: $$\hat b_i^t = (\hat u_i^t, \hat v_i^t, \hat w_i^t, \hat h_i^t).$$9 For later times, each detection embedding $0.5$0 is compared with the previous prototypes $0.5$1 using cosine similarity,

$0.5$2

yielding a similarity matrix $0.5$3 and cost matrix $0.5$4. The Hungarian algorithm solves

$0.5$5

A match is accepted only when $0.5$6 with $0.5$7 (Liu et al., 15 Dec 2025).

When a match is accepted, the prototype is updated by exponential moving average,

$0.5$8

with $0.5$9. Unmatched detections initialize new tracks, unmatched tracks retain their previous prototype and increment age, and tracks are removed when $224 \times 224$0 with $224 \times 224$1 (Liu et al., 15 Dec 2025). The stated rationale is that embeddings capture leaf appearance and structural cues, memory preserves identity through temporary occlusion, and the absence of rigid motion assumptions improves robustness to rotation and growth.

5. Evaluation and empirical performance

Evaluation uses an 80/20 train/test split by plants, TrackEval, and the standard MOT metrics HOTA, DetA, AssA, MOTA, and IDF1 (Liu et al., 15 Dec 2025). The baselines are BoT-SORT, ByteTrack, MOTRv2, LeTra, and Plant-Doctor.

On CanolaTrack, the reported results are: BoT-SORT at HOTA 33.32, DetA 91.30, AssA 12.18, MOTA 40.35, and IDF1 26.13; ByteTrack at 33.58, 91.94, 12.29, 41.88, and 26.20; MOTRv2 at 78.30, 77.33, 79.36, 79.68, and 83.78; LeTra at 67.02, 82.03, 54.98, 82.09, and 69.06; Plant-Doctor at 59.74, 74.42, 48.20, 79.71, and 69.56; and LeafTrackNet at 88.03, 92.25, 84.07, 93.64, and 92.90, respectively (Liu et al., 15 Dec 2025). The improvement over the strongest competing method is reported as +9.73 HOTA, +0.31 DetA, +4.71 AssA, +11.55 MOTA, and +9.12 IDF1, with the 9% HOTA improvement highlighted in the abstract (Liu et al., 15 Dec 2025).

Ablation studies attribute several design choices to these results. In backbone comparisons among MobileNetV3, ResNet18, ResNet34, ResNet50, ResNet101, and ViT-B/16, MobileNetV3 performs best overall for identity-related metrics while using 2.97M parameters and 0.23 GMACs (Liu et al., 15 Dec 2025). In triplet sampling, cross-plant flexible yields the best HOTA at 88.30, intra-plant full-cycle yields 88.03, and temporal-window sampling degrades markedly for small or very large windows. For inference hyperparameters, HOTA is stable for $224 \times 224$2, too low $224 \times 224$3 produces noisy associations, too high $224 \times 224$4 over-prunes tracks, and moderate smoothing with $224 \times 224$5 is effective (Liu et al., 15 Dec 2025).

The qualitative analysis emphasizes two difficult scenarios: heavy occlusion and pot rotation of about $224 \times 224$6. In both settings, the paper reports that LeafTrackNet preserves identities more reliably than the baselines, and accompanying t-SNE plots and heatmaps indicate that same-leaf embeddings cluster tightly over 31 days while different leaves remain separated (Liu et al., 15 Dec 2025).

6. Position within the literature, misconceptions, and limitations

LeafTrackNet occupies a specific position between plant-specific tracking and generic MOT. The paper argues that plant-specific trackers such as LeTra and Plant-Doctor still struggle on complex crops such as canola, while generic methods like BoT-SORT, ByteTrack, and MOTRv2 rely on assumptions of smooth motion, rigid trajectories, or strong spatial continuity that fail under growth, overlap, and rotation (Liu et al., 15 Dec 2025). In contrast, LeafTrackNet uses leaf-centric embeddings, cosine-similarity association, and a memory bank updated by EMA.

Its relation to earlier plant tracking work is complementary rather than identical. The fluorescence-video framework of 2015 formulated leaf analysis as joint multi-leaf segmentation, alignment, and tracking using Chamfer matching, pseudo-Boolean local search, gradient-descent tracking, and candidate birth/death management (Yin et al., 2015). LeafTrackNet does not adopt that template-driven optimization; instead, it separates detection from identity matching and applies a deep embedding model to top-down RGB canola imagery (Liu et al., 15 Dec 2025).

A recurrent misconception is to treat LeafTrackNet as interchangeable with lightweight object-tracking systems for generic visual tracking. It should not be confused with LightTrack, which uses one-shot NAS to design lightweight Siamese object trackers for resource-constrained deployment (Yan et al., 2021). LeafTrackNet denotes the plant-phenotyping framework introduced with CanolaTrack (Liu et al., 15 Dec 2025).

The paper does not present an extensive limitations section, but it suggests future directions including cross-species transferability, field-level deployment, and real-time agricultural decision-making (Liu et al., 15 Dec 2025). Implicit limitations listed in the source are the focus on top-down RGB imaging only, evaluation on a single crop species, the need to tune thresholds and update rules in the memory-based association scheme, and the absence of explicit motion modeling, which may matter in some settings (Liu et al., 15 Dec 2025).