VFTrack: Real-Time CNT Feature Tracking

• VFTrack is a specialized, real-time visual feature tracking system that automates the detection and kinematic analysis of CNT particles in SEM images.
• It integrates advanced keypoint detectors like ALIKED and efficient matchers such as LightGlue to achieve high precision and rapid processing.
• Its modular design supports dynamic shape estimation and motion decomposition, enabling real-time feedback and precise morphology reconstruction.

Visual Feature Tracking (VFTrack) is a specialized in situ real-time particle tracking framework designed for the automated detection, association, tracking, and kinematic analysis of carbon nanotube (CNT) particles as imaged by scanning electron microscopy (SEM). VFTrack addresses the limitations of prior approaches—in particular, the inability to continuously and automatically decompose and track thousands of individual nanoscale particle trajectories—by integrating advanced feature detection and robust matching within a modular and extensible pipeline. This enables rich kinematic analyses, real-time feedback, and dynamic shape estimation of growing nanostructures, bridging the gap between experimental observation and physics-based models for CNT synthesis (Safavigerdini et al., 26 Aug 2025).

1. Framework Architecture and Modular Components

VFTrack comprises a sequential pipeline involving four principal modules: feature detection, matching, track management (pruning and association), and kinematic/shape analysis.

Detection employs leading keypoint detectors (e.g., SIFT, DISK, ALIKED, SuperPoint), with ALIKED identified as the optimal choice. Keypoints are extracted from each SEM frame, targeting distinctive local structures within the growing CNT ensembles.

Matching leverages the LightGlue matcher, which associates keypoints between consecutive frames. Its precision and runtime efficiency enable near-real-time processing even on voluminous trajectories (mean frame processing time ≈ 0.31 s).

Track Formation and Pruning is achieved by appending valid, displacement-constrained matches to existing tracks and initializing new trajectories for unmatched detections. Outliers—such as those with excessive frame-to-frame jumps (e.g., >40 pixels)—are pruned to ensure track continuity and displacement realism.

Kinematic Analysis and Shape Estimation utilizes the resulting trajectories to perform motion decomposition and progressive 2D pillar reconstruction. The modular framework allows for the direct integration of alternative detectors and matchers, and its design is informed by benchmarking over 13,540 manually annotated trajectories (Safavigerdini et al., 26 Aug 2025).

2. Feature Detection and Matching Algorithms

A systematic evaluation identified the detector–matcher pair ALIKED + LightGlue as providing the optimal balance of recall, precision, and computational overhead.

ALIKED is a learned keypoint detector adept at localizing repeatable and distinct features in SEM images of CNTs, delivering high recall and precision even in complex or low-contrast environments.

LightGlue is a neural matcher designed for rapid feature pairing using context-sensitive attention mechanisms, minimizing both false positives and missed associations.

The resultant feature-tracking pipeline (as formalized in provided pseudocode) proceeds as follows per frame:

• Detect keypoints in frame t and t+1.
• Match keypoints with LightGlue.
• Prune matches exceeding a displacement threshold.
• Extend existing tracks with matched points; initiate new tracks for unmatched detections; terminate tracks without continuations.

A motion decomposition is performed on each match: $l = \sqrt{(\Delta X)^2 + (\Delta Y)^2}$

$\theta = \mathrm{atan2}(\Delta X, \Delta Y)$

where $l$ represents the displacement per frame and $\theta$ encodes deviation from the vertical axis.

3. Motion Vector Decomposition and Morphology Reconstruction

VFTrack delivers fine-grained, per-particle kinematics by decomposing displacement vectors into physically interpretable growth modalities:

• Axial Growth ($\Delta Y$) corresponds to the primary vertical advancement of the CNT pillar.
• Lateral Drift and Oscillations ($\Delta X$) quantify sidewise shifts and dynamic local oscillations within the growing forest.

These temporal vector components provide quantitative assessments of heterogeneous local growth rates and oscillatory dynamics.

Pillar Shape Estimation is realized by recursively aggregating mean displacements at each growth front and superimposing newly detected tracks near the base. The layered reconstruction algorithm accumulates both vertical (growth) and horizontal (drift/oscillation) deviations, thereby capturing the evolving morphology of CNT micropillars through the SEM sequence.

4. Performance Evaluation and Quantitative Metrics

The VFTrack framework is benchmarked extensively using manually annotated datasets and several quantitative performance measures:

• F1-Score: Assesses balanced tracking quality, defined as the harmonic mean of precision and recall. VFTrack achieves a peak F1-score of 0.78 with the ALIKED+LightGlue setting.
• $\alpha$-Score: A Jaccard-inspired similarity, $\alpha(X, Y) = 1 - \frac{d(X,Y)}{d(X, \emptyset)}$, where $d(X, Y)$ represents the aggregate error between estimated and ground-truth trajectories and $d(X, \emptyset)$ is the total ground-truth displacement. VFTrack attains $\alpha = 0.89$.
• Runtime: Processing time per frame is benchmarked at 0.31 s with the optimal configuration, meeting the requirements for quasi-real-time feedback in typical SEM imaging contexts.

Table: Detector and Matcher Benchmarking Results (Extracted Summary)

Detector	Matcher	F1-Score	α-Score	Time/frame (s)
ALIKED	LightGlue	0.78	0.89	0.31
SIFT	LightGlue	...	...	...
DISK	LightGlue	...	...	...

(Only the optimal combination is shown; further details are in the original paper.)

5. Applications in Nanomaterials Science

VFTrack is engineered for in situ characterization of CNT-based nano-structures, offering:

• Real-time kinematic analysis: Direct quantification of axial/lateral growth rates, local oscillation dynamics, and detection of pillar drift.
• Automated morphology reconstruction: Layer-by-layer estimation of changing pillar geometry, enabling concurrent observation and physical modeling.
• Bridging experimental and theoretical domains: The quantitative, particle-resolved data facilitates direct comparison with physics-based models, allowing for validation, calibration, and potentially closed-loop adaptive control of the CNT synthesis process.

The system is suited for applications where live feedback on nano-material growth is critical, such as adaptive process optimization in CVD reactors or the paper of structure–property relationships in flexible electronics, sensors, and composite materials.

6. Broader Implications and Future Directions

VFTrack's robust architecture and validated tracking algorithms suggest several expansion possibilities:

• Real-time SEM deployment: On-instrument integration is anticipated to provide immediate analytics for experimenters and enable adaptive process interventions.
• Generalization to other imaging modalities: While developed for CNTs, the underlying pipeline and algorithms are translatable to other domains demanding reliable tracking in noisy, low-contrast, or otherwise challenging imaging settings.
• Algorithmic adaptation to external variables: Future work may target compensation for environmental influences (e.g., temperature, humidity, gas flow) on feature stability, potentially by incorporating adaptive matching thresholds or detector retraining.
• 3D Reconstruction: Extension of the framework to multi-view or tomographic modalities would enable 3D morphology tracking, providing even richer physical insight into material growth dynamics.

7. Significance for the Field of Visual Feature Tracking

VFTrack advances the state of the art by:

• Demonstrating the feasibility of fully automated, real-time particle tracking in highly challenging nanoscale imaging,
• Establishing objective benchmarking for detector–matcher performance in such environments,
• Enabling per-particle kinematic decomposition and morphology tracking that directly inform both theoretical modeling and experimental strategy.

The approach thereby represents a model of practical, extensible, and quantitatively rigorous visual feature tracking for dynamic nanomaterial applications (Safavigerdini et al., 26 Aug 2025).