AI-Drone Assisted Human Rescue

• AI-based drone-assisted human rescue is a multidisciplinary approach that combines UAVs, machine learning, and sensor fusion to detect and locate victims in emergencies.
• It leverages real-time visual and acoustic processing with CNNs and multimodal sensors, achieving high detection rates and rapid localization in challenging scenarios.
• The system integrates autonomous mission planning, swarm coordination, and human–robot interaction to enhance search and rescue efficiency, reliability, and safety.

AI-based drone-assisted human rescue comprises an interdisciplinary set of systems, algorithms, and operational workflows that leverage unmanned aerial vehicles (UAVs) equipped with artificial intelligence for the detection, localization, and support of humans in disaster and emergency scenarios. Modern research demonstrates the integration of machine learning, real-time perception, robust mission autonomy, and complex human–machine interaction in both single-drone and multi-UAV swarms, across terrestrial, maritime, and urban-fire contexts, to increase the speed, accuracy, and scalability of search and rescue (SAR) operations.

1. Core Architectures and Modalities

Recent systems for drone-assisted human rescue are architected for end-to-end autonomy, real-time perception, and on-site decision making. Architectures typically involve:

• Airborne Platforms: Commercial-grade UAVs or custom quadrotors capable of >15 min flight, equipped with high-resolution RGB cameras, thermal infrared (FLIR) sensors, microphone arrays, and onboard compute (Jetson Nano/Xavier, Intel NUC) (Surmann et al., 2023, Mnaouer et al., 2021, Papyan et al., 2024).
• Onboard AI: Lightweight CNN-based object detectors (YOLOv3, TOOD, RetinaNet, Deformable DETR variants) are deployed for high-throughput human, fire, and vehicle detection; onboard inferencing supports inference rates of 10–20 fps (Surmann et al., 2023, Pyrrö et al., 2021).
• Multimodal Sensing: Acoustic sensors (MEMS microphone arrays) facilitate scream/cry localization via time-difference-of-arrival (TDOA) and CNN-based spectrogram classification; radar and LiDAR contribute to operation in occlusion or low-visibility (Papyan et al., 2024).
• Communications: Wi-Fi, LTE/5G, and LoRa links connect UAVs to ground stations, with bandwidth-aware data offloading, metadata streaming, and multi-hop protocols for swarm distribution (Queralta et al., 2020, Mnaouer et al., 2021).
• Ground Component: Operator dashboards aggregate geotagged victim detections, telemetry, and alert feeds for actionable response planning (Agrawal et al., 2020, Surmann et al., 2023).

Table: High-Level Modalities in Modern Rescue UAV Systems

Sensing Modality	Purpose	Example Onboard Processing
RGB/IR/Thermal Imaging	Person/fire detection, mapping	Onboard CNN (YOLOv3, RetinaNet, U-Net)
Microphone Array	Scream localization	TDOA analysis, audio CNN for distress recognition
LiDAR/Radar	Obstacle/victim through debris	Occupancy grid, sensor-fusion (e.g., EKF, SLAM)

These system-level ingredients underpin both individual rescuer-UAVs and distributed, self-organizing drone fleets for large-scale, rapid-response missions.

2. AI Perception and Localization Algorithms

Visual and acoustic-based human localization is critical for effective drone-assisted SAR. Algorithmic approaches include:

• Visual Person Detection: Deep CNNs (TOOD, YOLOX, AIR (Pyrrö et al., 2021)), leveraging transfer learning from COCO/ImageNet and data augmentation, achieve real-time detection with mean average precision (mAP) up to ~0.39–0.92 depending on operating conditions (Surmann et al., 2023, Pyrrö et al., 2021, Schedl et al., 2021). Image tiling and robust postprocessing (MOB: Merging Overlapping Bounding boxes) optimize both localization recall and SAR operating precision requirements.
• Thermal and Acoustic Fusion: U-Net or similar backbones produce per-pixel thermal probability maps, which are fused with acoustic CNN outputs—using feature or decision-level fusion (e.g., $$p_{\rm fused} = \alpha P_{\rm ac} + (1–\alpha) P_{\rm th}$$)—to boost true positives and suppress false alarms, particularly for occluded or night operations (Papyan et al., 2024).
• Pseudo-Trilateration: For mobile phone-based victim localization, SARDO implements a pseudo-trilateration architecture using onboard SDR/microcell emulation, ToF-based ranging estimates, and a 2D-CNN that regresses over N circular-patrol ToF samples for trajectory reconstruction, achieving median localization error of ~30 m in 3 minutes (Albanese et al., 2020).

Notably, field-proven hybrid pipelines incorporate additional steps: motor noise cancellation, histogram equalization, online data augmentation, and filtering/merging of bounding boxes using non-maximal suppression and IoU-based cluster heuristics.

3. Mission Autonomy, Multi-Agent Planning, and Human–Robot Teaming

AI-enabled mission autonomy is realized using a combination of:

• Finite-State Flight Controllers: UAVs operate under layered autonomy architectures; high-level mission logic (search, track, return-to-launch) is orchestrated by finite-state machines, while low-level autopilots handle obstacle avoidance, geofencing, and battery/network safety (Agrawal et al., 2020).
• Adaptive and Human-Aware RL: Deep reinforcement learning planners, optionally incorporating Analytic Hierarchy Process (AHP) for multi-objective reward shaping (speed, energy, safety, human-comfort), deliver context-sensitive trajectory generation. Methods such as TD3+AHP with similarity-based experience replay optimize both operational efficiency and survivor comfort (e.g., capped approach speed, proximity constraints) (Ramezani et al., 2024).
• Multi-Agent and Swarm Coordination: Formation control and connectivity maintenance balance area coverage and communication constraints in maritime and terrestrial swarms (Queralta et al., 2020). Centralized PPO-based learning frameworks coordinate heterogeneous UAV agents under partial observability, encoding roles such as high-cover rescuer and close-range guide, leveraging panic-aware agent-based human behavior modeling (Mendoza et al., 27 Oct 2025). Anticipatory planning integrates probabilistic lost-person models, human searcher trajectory priors, and Gaussian process sensor models to compute risk-minimizing cooperative UAV paths (Cangan et al., 2020).
• Human–AI Synergy: Participatory design (SA-cards, scenario-based workshops), mixed-initiative control (voice, gesture, and touch modalities (Cacace et al., 2016)), and advising agents for operator support (imitation learning from small demos, future-utility prediction) (Barr et al., 25 Feb 2025) are extensively validated for situational awareness, decision support, and cognitive load management (Agrawal et al., 2020).

4. Field Deployment, Evaluation Metrics, and Case Studies

Robust field validation is fundamental for AI-based rescue systems. Key quantitative metrics include:

• Detection/Localization Performance: Precision, recall, mAP at varied IoU thresholds (e.g., AIR+MOB: 94.9% precision, 92.9% recall, AP=91.7% under a SAR-specific "zone" criterion) (Pyrrö et al., 2021); median ToF-based phone localization error of ~30 m (Albanese et al., 2020); thermal+audio multimodal fusion achieves 88% true-positive rate with ~1.2 s end-to-end latency (Papyan et al., 2024).
• Mission Throughput and Efficiency: UAVs such as SARDO localize one UE per 3 min (~5% battery overhead), with spatial coverage determined by patrol radius and mission time; swarm configurations parallelize area scanning for coverage maximization (Albanese et al., 2020, Queralta et al., 2020).
• Human-System Metrics: NASA-TLX-based cognitive load, operator effort, detection rate, and area coverage (e.g., advising agent improves victim detection by 7 pp and reduces mission time by up to 2.7 min) (Barr et al., 25 Feb 2025); human-centric RL planners maintain low survivor "flee" rates via approach speed and gendered/anthropomorphic cues (Ramezani et al., 2024).
• Autonomous/Operator Mode Distribution: Mixed modality use (e.g., 60% autonomous, 35% mixed, 5% teleop; 96% command recognition accuracy with multimodal fusion) (Cacace et al., 2016).
• Robustness and Adaptability: Algorithms accommodate environmental variation—clutter, smoke, low-light, occlusion—with sensor/algorithm fusion (RGB/thermal, radar, LiDAR), domain adaptation, and active online learning (Surmann et al., 2023, Schedl et al., 2021).

Table: Comparison of Selected AI-Drone Rescue Systems

System/Paper	Main Modality	Key Metric (Human Det.)	Field-Reported Time/Energy
SARDO (Albanese et al., 2020)	Phone ToF+ML	~30m median error	3 min/UE, +5% battery
AIR+MOB (Pyrrö et al., 2021)	Visual CNN (RetinaNet)	91.7% AP (SAR-APD zone)	~1 s/4K image
Multimodal (Papyan et al., 2024)	Thermal+acoustic fusion	88% TPR, -35% FPR (fusion)	1.2 s end-to-end alert
GNN (1D SA) (Schedl et al., 2021)	Thermal+DNN, low link	AP=86–93% (1D/2D sampling)	6–20 min/mission

5. Human Factors, Co-Design, and Interaction

Effective rescue UAVs must align not only with technical but also with human-centric requirements:

• Interface Design: Co-designed UIs (damage map, camera, alarms) with participatory input from first responders; adaptive role assignment; safety and autonomy countermeasures (geofencing, full-screen focus, pre-flight checklists) (Agrawal et al., 2020).
• Human Trust and Acceptability: Human-centric trajectory planners (e.g., (Ramezani et al., 2024)) balance mission speed against proximity-induced survivor discomfort. Surveyed users prefer masculine cues for urgency and feminine/anthropomorphic cues for calming, and trust increases with clear voice feedback.
• Operator Workflow and Mixed-Initiative: Multimodal interaction (gesture, speech, touch) enables high-value, low-rate corrections embedded in mostly autonomous missions (Cacace et al., 2016). Advising agents leverage imitation learning to propose contextual action recommendations, statistically improving detection rates and coverage without increasing cognitive load (Barr et al., 25 Feb 2025).
• Multi-Actor Integration: Architectures such as AutoSOS (Queralta et al., 2020) and SHERPA (Cacace et al., 2016) seamlessly integrate drones, ground vehicles, and human searchers into a collaborative team, with workflows that exploit both autonomous and human-override states (e.g., rescue kit dropping, window breaking in FireFly (Mnaouer et al., 2021)).

6. Limitations, Challenges, and Future Directions

Despite operational viability, multiple open challenges remain:

• Coverage and Throughput: Single-UAV systems are area and time limited; swarm deployment, fast trajectories, and hybrid ground-air teams are essential for large/distributed search (Albanese et al., 2020, Schedl et al., 2021).
• Perception Robustness: Performance degrades with smoke, rain, debris, and NLoS. Fusing thermal, radar, and audio increases resilience; sensor calibration and online model updates are needed for new environments (Papyan et al., 2024, Surmann et al., 2023).
• Resource Constraints: Real-time embedded AI requires model quantization, pruning, and adaptive inference to run on low-power SoC platforms; task offloading and scheduling balance bandwidth, latency, and compute (Queralta et al., 2020).
• Human Interaction: Effective mixed-initiative and trust calibration (e.g., context-aware anthropomorphism) are critical for safe and effective victim engagement and operator control (Ramezani et al., 2024).
• Scalability and Regulation: Scaling to large areas, non-line-of-sight, and multiple victims exposes challenges in communication, coordination (e.g. multi-UAV consensus), and regulatory compliance (BVLOS, airspace integration) (Cangan et al., 2020).
• Algorithmic Directions: Reinforcement learning with dynamic reward shaping (AHP), coordinated multi-agent RL (PPO-R), and federated or unsupervised adaptation across heterogeneous fleets are leading directions; improvements in data-efficient domain adaptation and explainable AI for operator advisement remain open (Mendoza et al., 27 Oct 2025).

Significant advances in multimodal sensing, model-based planning, and human–AI teaming continue to drive progress toward scalable, robust AI-based drone rescue solutions supporting first responders in increasingly challenging environments.