- The paper introduces a modular autonomous visual intelligence system that integrates real-time facial tracking, environmental scanning, and trajectory following on low-cost UAV platforms.
- It employs advanced AI models like YuNet, FaceNet, and DepthAnything V2 to achieve over 92% tracking and 97.4% recognition accuracy, validated under complex indoor conditions.
- The study demonstrates practical scalability using open-source tools and efficient algorithms on resource-constrained hardware, achieving sub-150ms response times.
Modular Real-Time Visual Intelligence on Low-Cost UAVs
System Overview and Architectural Contributions
This work introduces a modular autonomous visual intelligence system for low-cost UAVs, integrating advanced AI-powered perceptual capabilities with off-the-shelf drone hardware. Centered on the DJI Tello platform, the system employs a Python-based backend supported by a web interface, enabling robust control and telemetry in real time. Its primary innovation is the fusion of facial detection, facial recognition, and monocular depth estimation into a single deployable architecture, emphasizing accessibility by leveraging open-source tools and commodity embedded hardware. Differentiating itself from commercial and prior academic solutions, this approach achieves state-of-the-art accuracy and responsiveness on resource-constrained platforms, validating the practical viability of sophisticated AI perception for low-cost UAV operations.
Core Modules and Technical Approach
FollowMe: Autonomous Person Tracking
The FollowMe module combines the YuNet facial detector with closed-loop flight control based on PID regulation. YuNet is selected for its efficient inference and compact memory footprint, critical for real-time performance on embedded hardware. The detected facial centroid and bounding box area serve as proxies for lateral and depth positioning, respectively. The control stack translates these signals into UAV movement directives, with temporal validation to mitigate false positives from transient detection gaps. During evaluations, the FollowMe module achieved >92% tracking accuracy and sub-300ms correction latency, with rapid reacquisition following occlusions. The system demonstrates robust dynamic response suitable for indoor navigation and person-following tasks.
SkyScan: Autonomous Scanning and Recognition
SkyScan extends the UAV’s perceptual abilities with autonomous environmental scanning coupled with facial recognition against a cloud-based identity database. For depth perception, the system incorporates the DepthAnything V2 ViT-based model, enabling room-scale geometry estimation using a monocular RGB feed. This choice optimizes the trade-off between computational efficiency and depth estimation fidelity, with demonstrated coverage of up to 20 meters and robust operation under varied lighting.
Upon detecting human presence, the module invokes FaceNet to extract 128-dimensional embeddings, supporting high-precision identification even under challenging image conditions (e.g., partial occlusions, variable illumination). Reference testing corroborates a recognition accuracy of 97.4%, aligning with leading literature benchmarks and demonstrating resilience to real-world degradations. Temporal filtering ensures identification robustness. This fully-integrated scanning/recognition loop enables applications in surveillance or search and rescue within GPS-denied environments.
LineTracker: Visual Trajectory Following
The LineTracker module addresses navigation along visually marked trajectories via monocular vision. Overcoming the Tello’s camera constraints, a custom 45° 3D-printed mirror fixture redirects the field of view to the floor, thus enabling ground-based line detection. A color-thresholded HSV mask in OpenCV isolates guide paths, segmented into a virtual sensor matrix for robust lane detection and direction inference. The system employs proportional logic for forward velocity adaptation, decelerating in high-curvature segments to preserve stability.
Extensive trajectory testing with complex path features yielded centerline deviations under 15 pixels, zero navigation faults, and seamless manual/autonomous transition. The solution achieves fully software-driven navigation without resorting to hardware line sensors, confirming the feasibility of portable, cost-minimal UAV guidance in structured environments.
Experiments across all modules in controlled, complex indoor conditions underscore the system’s reliability. Notably:
- FollowMe: Maintained >92% accuracy, <2s reacquisition after occlusion in 89% of cases.
- SkyScan: YuNet detected faces in >95% of scenarios; FaceNet maintained 97.4% recognition accuracy with zero false positives on evaluation sets >20 individuals.
- LineTracker: Completed all predefined paths, with tight adherence to guide lines, no observed drift or off-path events.
Integrated operation imposed an average CPU load below 65% (Ryzen 7), with all perception-control tasks achieving <150ms end-to-end response time. System modularity allowed concurrent multi-threaded operation without observable cross-interference or latency penalties.
Implications and Future Directions
This study empirically confirms that real-time AI-driven perceptual tasks—typically associated with higher-cost UAV platforms—are achievable using open-source ecosystems and sub-$100 drones by judicious architectural and algorithmic choices. The emphasis on modularity, platform independence, and minimal hardware extensions (e.g., mirror-based visual augmentation) yields a deployable template for scaling autonomous aerial assistants across numerous constrained application domains, including low-resource defense, automated inspection, or indoor logistics.
Key limitations include reliance on offboard processing; migrating inference to onboard edge AI accelerators (e.g., Jetson Nano, Coral Edge TPU) is a clear extension to enable field autonomy. Augmentation with multi-view or time-of-flight sensors, multi-face tracking, larger recognition corpora, and robust sensor fusion/SLAM techniques would address environmental and operational uncertainties in future iterations. Integration with cloud/IOT platforms for fleet coordination and large-scale data management also represents a promising vector for operationalizing such systems in real-world deployments.
Conclusion
This modular intelligent UAV platform demonstrates that open-source, low-cost hardware can be transformed into highly capable autonomous systems via appropriate AI integration and lightweight software design. The approach advances the state of the art in cost-effective drone autonomy, bridging theoretical advances in deep computer vision and real-world, resource-limited deployment scenarios. Future work will focus on further miniaturization, enriched perception, and fully onboard AI-driven autonomy to expand applicability in both civil and tactical environments.