PlantPose: Dual Agricultural Vision Systems

• PlantPose is a dual-purpose framework that uses computer vision to perform both category-level 6D object pose estimation with deformable base meshes and single-image plant skeleton extraction via tree-structured graphs.
• The 6D pose estimation component employs a small ViT encoder and cubic B-spline deformation to adapt a single base mesh to diverse agricultural produce, achieving improved accuracy over RGB-only baselines.
• The plant skeleton estimation module leverages a transformer-based architecture with MST-constrained graph generation and selective feature suppression to ensure connected, acyclic tree graphs for complex plant structures.

PlantPose is the name used for two distinct computer-vision systems in recent agricultural and plant-science literature. One work, written as PLANTPose, addresses category-level 6D object pose estimation in agricultural settings from RGB images only by jointly estimating rigid pose and mesh deformation relative to a single category-level base mesh (Glytsos et al., 30 May 2025). The other, written as PlantPose, addresses single-image plant skeleton estimation by generating an arbitrary tree graph under explicit botanical topology constraints through the integration of a graph generator, a minimum spanning tree (MST) solver, and a differentiable reparameterization layer (Liu et al., 18 May 2026). The shared name can therefore denote either a pose-estimation framework for deformable produce instances or a universal plant-structure estimator for branching skeletons; disambiguation depends on whether the target output is a 6D rigid pose with deformation parameters or a 2D tree graph.

1. Dual usage of the name

In the 2025 pose-estimation work, PLANTPose is introduced as a framework for category-level 6D object pose estimation in agriculture, motivated by the observation that fruits and vegetables exhibit high intra-class variability in shape, size, and texture, while many existing methods rely on instance-specific CAD models or depth sensors (Glytsos et al., 30 May 2025). Its central representation is a rigid transform in $SE(3)$ together with a learned deformation of a category-level base mesh.

In the 2026 skeleton-estimation work, PlantPose is introduced as a universal plant skeleton estimator for single images, motivated by the fact that plant skeleton estimation differs from human pose estimation because plant topology is not fixed and must instead be inferred as an arbitrary tree graph (Liu et al., 18 May 2026). Its output is a graph $G=(V,E)$, where $V$ denotes 2D node coordinates and $E$ denotes undirected edges, under the hard tree constraints of connectivity, acyclicity, and $|E|=|V|-1$.

A concise comparison is useful because the two systems address different problem classes.

System	Core task	Output
PLANTPose (Glytsos et al., 30 May 2025)	Category-level 6D object pose estimation in agriculture from RGB images only	Rotation $R$, translation $t$, and deformation parameters relative to a base mesh
PlantPose (Liu et al., 18 May 2026)	Single-image plant skeleton estimation via tree-constrained graph generation	Tree graph $G=(V,E)$ with 2D nodes and undirected edges

This suggests that “PlantPose” functions less as a single method family than as a coincident label for two separate lines of agricultural vision research: one geometric and instance-adaptive, the other structural and topological.

2. PLANTPose as category-level 6D object pose estimation

PLANTPose formulates the agricultural pose-estimation problem as RGB-only category-level 6D object pose estimation without instance-specific CAD models (Glytsos et al., 30 May 2025). The stated goal is to use a single base mesh per category and adapt it to unseen instances via learned deformations, thereby enabling pose estimation across large intra-class variation in shape, size, and texture.

The method defines 6D pose as an element of $SE(3)$ parameterized by rotation and translation:

• $R \in SO(3)$, a $G=(V,E)$0 orthonormal rotation matrix,
• $G=(V,E)$1, a translation vector in the camera frame.

PLANTPose jointly estimates $G=(V,E)$2 and per-instance mesh deformation parameters relative to a category base mesh. The deformation model is a cubic B-spline on a $G=(V,E)$3 control lattice aligned with the base mesh bounding box, with the network predicting a 24D vector corresponding to 8 corners $G=(V,E)$4 3 coordinates (Glytsos et al., 30 May 2025). This deformation space is intended to adapt a single category-level CAD model to novel instances such as bananas with diverse curvature and thickness.

The architecture comprises a detection-and-cropping stage, a backbone, and three heads. During training, object crops are defined by 2D projections of ground-truth vertices; during inference, a YOLOv11 detector finds the object ROI, which is then cropped and resized. The backbone is a small ViT encoder (30.9M parameters, 32×32 input patch size), pretrained on ImageNet-21k. The heads predict:

• a 6D rotation representation later orthonormalized into a valid $G=(V,E)$5,
• a 3D translation vector in the cropped image frame,
• the 24D lattice corner offsets for deformation (Glytsos et al., 30 May 2025).

PLANTPose is trained with three mean-squared-error terms: a rotation loss, a deformation loss, and a 2D projection loss based on the perspective projection of deformed, rotated, and translated vertices into the cropped image frame. The overall objective is

$G=(V,E)$6

with empirically set weights $G=(V,E)$7 (Glytsos et al., 30 May 2025).

3. Deformation model, disentanglement, and synthetic-data pipeline in PLANTPose

The deformation model begins from a category-level reference mesh and places a tight lattice bounding box around it. Let

• $G=(V,E)$8,
• $G=(V,E)$9 denote the bounding-box corners of the base mesh. For a vertex $V$0, normalized coordinates $V$1 are computed as fractional distances along the $V$2, $V$3, and $V$4 axes within the box (Glytsos et al., 30 May 2025).

The lattice has eight corner offsets $V$5 for $V$6, which are clamped or extended to a $V$7 grid to ensure correct boundary behavior with cubic B-splines. The 1D cubic B-spline basis functions are

$V$8

$V$9

The vertex displacements are then

$E$0

$E$1

$E$2

with displacement vector $E$3 and deformed vertex

$E$4

The paper states that deformation magnitude is constrained to an empirical upper bound to avoid extreme or unhelpful shapes, and that B-splines provide $E$5 continuity, yielding smooth deformations appropriate for organic produce (Glytsos et al., 30 May 2025).

A specific issue addressed by the method is pose–deformation disentanglement. Random lattice perturbations can introduce unintended global rotation or translation when cubic B-splines are applied. To prevent deformation from leaking rigid motion, the method estimates a similarity alignment between the original and deformed meshes using Umeyama,

$E$6

with $E$7 fixed to $E$8, and then corrects the lattice so that shape change remains independent of pose (Glytsos et al., 30 May 2025).

The training data pipeline is synthetic-first. A BlenderProc-based renderer generates scenes with procedural rooms, textured surfaces, distractor objects, varied ambient and directional lighting, physically simulated placement, and multiple camera views with fixed intrinsics meant to emulate a real camera. For each rendered image, the annotations include ground-truth $E$9 and projected vertices (Glytsos et al., 30 May 2025).

To narrow the synthetic-to-real gap, the method applies Stable Diffusion inpainting with ControlNet depth conditioning only to the object of interest while preserving the background. Curated prompts are used for unripe, ripe, rotten, and natural color/texture variations so as to mimic agricultural appearance changes due to lifecycle and environment (Glytsos et al., 30 May 2025). The paper states that this post-render augmentation improves realism while preserving pose.

4. Empirical evaluation, ablations, and limitations of PLANTPose

The reported training set contains 4,000 synthetic scenes, each with a random lattice deformation and 3 images per scene from varied viewpoints. The real benchmark is an in-house banana dataset with 6 distinct bananas covering shapes and ripeness, with 100 RGB images captured using an Intel RealSense D435. Each banana was 3D scanned using an iPhone 14 Pro LiDAR to obtain real meshes; ground-truth alignment to the base mesh is performed via ICP, and deformation error is computed after removing pose (Glytsos et al., 30 May 2025).

Evaluation uses:

• Chamfer distance (mm) between scanned meshes and predicted deformed meshes,
• rotation error (degrees) as mean and median,
• translation error (mm) as mean and median,
• deformation error (mm) after removing estimated rotation and translation via ICP (Glytsos et al., 30 May 2025).

On the 100-image banana benchmark, the RGB-only baseline MegaPose is evaluated using the same single banana mesh for all test images. The reported results are:

Method	Chamfer	Rotation error	Translation error
MegaPose	90.1 mm	mean 52.4°, median 43.6°	mean 59.9 mm, median 45.6 mm
PLANTPose	59.8 mm	mean 32.6°, median 23.6°	mean 42.5 mm, median 37.9 mm

The corresponding deformation error is 29.1 mm for MegaPose and 12.1 mm for PLANTPose (Glytsos et al., 30 May 2025). The paper attributes these improvements to the ability to adapt the mesh shape via lattice deformation while jointly estimating pose.

Ablation results isolate the effects of Stable Diffusion augmentation and Umeyama correction. The full model reports Chamfer 59.8 mm, rotation mean/median 32.6°/23.6°, translation mean/median 42.5/37.9 mm, and deformation 12.1 mm. Without Stable Diffusion, results become 71.5 mm, 37.7°/23.0°, 48.9/48.7 mm, and 13.9 mm. Without Umeyama, results become 73.1 mm, 36.5°/26.2°, 50.3/45.8 mm, and 12.7 mm. Removing both yields 89.7 mm, 36.1°/26.7°, 61.1/61.9 mm, and 26.5 mm (Glytsos et al., 30 May 2025). The observations reported in the paper are that removing Stable Diffusion increases Chamfer and translation errors, while removing Umeyama notably degrades translation, confirming that cubic B-spline perturbations can leak pose.

The paper also states several limitations. Performance depends on base mesh representativeness; extremely atypical instances may remain difficult. Stable Diffusion improves realism but is not perfect, and in rare cases inpainting may slightly alter appearance. Real agricultural settings may involve severe occlusions and foliage, and RGB-only methods are susceptible to poor visibility. In addition, the current formulation uses a separate model per category, with multi-category conditioning left as future work (Glytsos et al., 30 May 2025).

5. PlantPose as universal plant skeleton estimation

The 2026 PlantPose work addresses a different problem: estimating a plant’s skeleton from a single image as a tree graph rather than as a fixed-topology keypoint set (Liu et al., 18 May 2026). The formal task is: given an image $|E|=|V|-1$0 of a plant, estimate a graph $|E|=|V|-1$1 where $|E|=|V|-1$2 is the set of 2D node coordinates, $|E|=|V|-1$3 is the set of undirected edges connecting pairs of nodes, and the resulting graph is connected and acyclic with $|E|=|V|-1$4.

The objective is expressed as

$|E|=|V|-1$5

where $|E|=|V|-1$6 is the learned image-to-graph predictor and $|E|=|V|-1$7 denotes edge sets forming a tree over nodes $|E|=|V|-1$8 (Liu et al., 18 May 2026).

For each ordered node pair $|E|=|V|-1$9, the network predicts a binary edge probability vector

$R$0

with $R$1, derived by softmax from edge logits

$R$2

An unconstrained edge set is formed as

$R$3

A non-differentiable projection $R$4 then enforces the tree constraint, for example by computing the MST of the complete graph defined on $R$5 with edge costs

$R$6

This yields the constrained graph

$R$7

The work emphasizes that plant skeleton estimation requires handling arbitrary tree graphs, which distinguishes it from human-pose settings with fixed topology (Liu et al., 18 May 2026).

Architecturally, PlantPose builds on RelationFormer, described as a state-of-the-art non-autoregressive image-to-graph model. It uses a deformable DETR-style transformer encoder-decoder over ResNet-50 features, with decoder queries consisting of [obj]-tokens for node hypotheses and a single [rtn]-token for global relational context. A node detection head predicts node positions from the [obj]-tokens, and an MLP with layer normalization predicts pairwise edge logits (Liu et al., 18 May 2026).

6. Tree-constrained graph generation, SFS, data curation, and evaluation in PlantPose

The key technical feature of PlantPose is the incorporation of Kruskal’s MST into the training loop together with the Selective Feature Suppression (SFS) layer (Liu et al., 18 May 2026). After the unconstrained prediction stage, the model builds the complete graph on the predicted nodes using edge costs $R$8, then runs Kruskal’s MST with union-find cycle prevention to obtain constrained edges $R$9.

The method defines:

• $t$0, the edges added by MST,
• $t$1, the edges removed by MST.

Because MST is non-differentiable, SFS modifies logits so that the forward pass agrees with the MST decision while gradients still flow. The rules are:

• if $t$2, set $t$3,
• if $t$4, set $t$5,
• otherwise leave $t$6 unchanged, with $t$7 large (Liu et al., 18 May 2026).

After softmax on the modified logits, the probabilities become approximately

• $t$8 for $t$9,
• $G=(V,E)$0 for $G=(V,E)$1, where

$G=(V,E)$2

The paper explicitly states that no Gumbel-Softmax or relaxation is needed, because SFS is a deterministic feature-level reparameterization (Liu et al., 18 May 2026).

The edge loss uses cross-entropy on both unconstrained and constrained probabilities:

$G=(V,E)$3

$G=(V,E)$4

and the total loss is

$G=(V,E)$5

The paper’s interpretation is that hard constraints are enforced by MST, while SFS makes the forward pass differentiably consistent with MST so that learning uses the tree-constrained outputs (Liu et al., 18 May 2026).

Training uses the official RelationFormer code in PyTorch, with $G=(V,E)$6 during training, early stopping after 30 epochs without improvement, and training times on 8× NVIDIA RTX A100 GPUs of approximately 141 h (synthetic), 10 h (root), and 98 h (grapevine) (Liu et al., 18 May 2026). The method also notes a complexity of $G=(V,E)$7 for pairwise edge scoring and $G=(V,E)$8 for Kruskal MST with $G=(V,E)$9.

The data curation is unusually broad. Domain-specific datasets include:

• L-system synthetic: 100,000 train / 1,000 val / 20,000 test,
• Root (Arabidopsis early roots): 625 train / 78 val / 78 test, augmented to 62,500 train,
• Grapevine (3D2cut Single Guyot): 1,185 train / 63 val / 255 test, augmented to 118,500 train (Liu et al., 18 May 2026).

Generalized training uses six datasets: grapevine; LVD2021 leaf image; LVD2021 vein mask; MIPDB; self-captured; web-sourced, standardized to 256×256, with nodes sampled uniformly at 13 pixels and a maximum-node cap of 384. Four out-of-domain test datasets are then used: thickened L-system, LVD with background, tree synthetic, and root (Liu et al., 18 May 2026).

Evaluation uses three principal metrics:

• Street Mover’s Distance (SMD) for geometric fidelity of skeletons,
• TOPO score for topological node precision, recall, and F1,
• Tree rate for the percentage of outputs that satisfy the tree constraint (Liu et al., 18 May 2026).

7. Reported performance, generalization, and conceptual significance

Under domain-specific training, PlantPose reports:

• on synthetic L-system, SMD ≈ 4.78×10^−6, TOPO F1 ≈ 0.977, Tree rate 100%;
• on root, SMD ≈ 8.82×10^−5, TOPO F1 ≈ 0.833, Tree rate 100%;
• on grapevine, SMD ≈ 1.03×10^−4, TOPO F1 ≈ 0.870, Tree rate 100% (Liu et al., 18 May 2026).

The same section reports that the unconstrained RelationFormer has much lower tree validity—approximately 36% tree rate on synthetic L-system and approximately 0% on grapevine—while test-time MST improves performance but remains behind full PlantPose on the reported tasks (Liu et al., 18 May 2026).

Under generalized training across six datasets, PlantPose is reported to maintain Tree rate 100% and balanced TOPO and SMD across in-domain benchmarks. The listed typical outcomes are:

• Grapevine: F1 ≈ 0.839, SMD ≈ 1.57×10^−4,
• LVD leaf image: F1 ≈ 0.817, SMD ≈ 5.41×10^−4,
• LVD vein mask: F1 ≈ 0.888, SMD ≈ 3.75×10^−4,
• MIPDB: F1 ≈ 0.686, SMD ≈ 1.61×10^−3,
• Self-captured: F1 ≈ 0.691, SMD ≈ 1.92×10^−3,
• Web-sourced: F1 ≈ 0.594, SMD ≈ 8.26×10^−3 (Liu et al., 18 May 2026).

On the out-of-domain test sets, the reported results are:

• Thickened L-system: F1 ≈ 0.841, SMD ≈ 5.59×10^−5, Tree rate 100%,
• LVD with background: F1 ≈ 0.594, SMD ≈ 2.22×10^−2, Tree rate ≈ 99.8%,
• Tree synthetic: F1 ≈ 0.483, SMD ≈ 7.34×10^−3, Tree rate 100%,
• Root: F1 ≈ 0.555, SMD ≈ 2.70×10^−2, Tree rate ≈ 98.2% (Liu et al., 18 May 2026).

The paper attributes this robustness to a combination of diverse training data and constraint-aware learning. It further reports qualitative generalization to unlabeled web images, sketches, and even retinal vessel images after simple preprocessing, though this is described qualitatively rather than through benchmark metrics (Liu et al., 18 May 2026).

The work also lists limitations. Training incurs computational overhead because MST is computed at every iteration. Performance depends on accurate node detection; missed nodes lead to broken topology. Very thin or heavily occluded structures, dense foliage, cluttered backgrounds, and overlapping branches remain challenging because wrong node or edge cues can induce incorrect connections (Liu et al., 18 May 2026).

Taken together, the two PlantPose systems illustrate two distinct ways of imposing structure in agricultural vision. PLANTPose (Glytsos et al., 30 May 2025) imposes geometric structure through a low-parameter cubic B-spline deformation around a category-level base mesh, together with pose–deformation disentanglement. PlantPose (Liu et al., 18 May 2026) imposes topological structure through MST-constrained graph generation and SFS-based differentiable consistency. A plausible implication is that the shared name reflects a broader methodological tendency in plant and agricultural vision: embedding explicit structural priors—whether geometric or graph-theoretic—into learning pipelines in order to improve robustness under heterogeneity, domain shift, and sparse supervision.