Agentic UAVs Framework

• Agentic UAVs Framework is a multi-layered architecture that blends sensor fusion with LLM-based reasoning to enable context-rich, autonomous drone operations.
• It employs a five-layer structure—from perception to learning—that integrates advanced tool-calling and multi-agent ecosystem connectivity for dynamic decision support.
• Performance metrics demonstrate significant gains in detection confidence, action recommendation, and contextual analysis compared to traditional rule-based UAV systems.

Agentic UAVs frameworks refer to layered architectures that endow unmanned aerial vehicles with cognitive, adaptive, and integrated decision-making capabilities far beyond traditional rule-based autonomy. These frameworks center on the incorporation of LLMs, advanced perception modules, contextual reasoning engines, structured tool-calling, and ecosystem connectivity, enabling UAVs to transform diverse, uncertain sensor input into context-aware, explainable actions, with feedback-driven learning and continuous improvement. The following sections comprehensively detail the components, workflow, methodologies, evaluation, and implications of the Agentic UAVs framework as established in (Koubaa et al., 14 Sep 2025).

1. Five-Layer Agentic UAVs Architecture

The Agentic UAVs framework is structured as a five-layer stack, engineered to bridge low-level sensor data with high-level cognitive reasoning and systems integration:

1. Perception Layer
   • Transforms high-dimensional, uncertain sensor data (e.g., RGB/thermal/LiDAR, IMU) into a probabilistic world model.
   • Employs multi-modal foundation models (e.g., YOLOv11 for detection, semantic segmentation), data fusion (EKF, factor graphs), and outputs a structured dynamic 3D scene graph enriched with confidence/covariance metadata.
2. Reasoning Layer
   • Serves as the cognitive core, employing a ReAct (Reasoning + Acting) paradigm.
   • Leverages LLMs (e.g., GPT-4, local Gemma-3) for goal decomposition, plan generation, and situation-specific tool-calling (e.g., querying APIs, sandboxed code evaluation).
   • Outputs machine-readable policy graphs (typically JSON), containing plan dependencies, preconditions, corrective/replanning triggers, and error-handling instructions.
3. Action Layer
   • Translates high-level reasoning outputs into executable flight trajectories and digital-system commands.
   • Uses motion planners (e.g., MPC, RRT* for flight) and supports API-based ecosystem actions (e.g., alert emails, external database updates).
   • Reports structured responses enabling plan verification or dynamic replanning.
4. Integration Layer
   • Implements secure, modular ecosystem connectivity.
   • Adopts standardized protocols (Model Context Protocol [MCP], Agent Communication Protocol [ACP], Agent-to-Agent [A2A]) for inter-UAV and system-to-UAV communication.
   • Enables real-time, multi-agent coordination and external service integration (e.g., weather queries, NOTAM ingestion).
5. Learning Layer
   • Responsible for feedback-driven continuous improvement.
   • Supports RL, RLHF, fleet-level data aggregation, Retrieval-Augmented Generation (RAG), and operational “memory” storage for transfer learning and model refinement.

2. Cognitive Reasoning and Tool-Calling

A pivotal innovation is the embedding of LLM-driven cognitive workflows equipped with robust tool-calling:

• ReAct Workflow: The LLM decomposes abstract objectives into conditional, context-aware plans, then actuates external tool-calls (e.g., weather APIs, sensor queries) to validate preconditions or obtain real-time data.
• Plan Representation: Plans are serialized as JSON policy graphs:

  { "plan_id": "P-789", "goal": "Inspect perimeter anomaly.", "steps": [ { "step_id": 1, "action": "call_tool", "tool_name": "api.weather.get_forecast", ... }, { "step_id": 2, "action": "fly_to", "args": {"target_id": "Anomaly-01"}, "preconditions": ["step_1.wind_speed < 15"], "on_fail": "trigger_reflection" } ], "dependencies": {"2": ["1"]} }

• Model Deployment: To balance latency and performance, the architecture supports both local LLMs (e.g., Gemma-3, achieving 1.48 s/decision) and cloud LLMs (e.g., GPT-4, achieving 4.95 s/decision) with identical interface hooks for tool integration.
• Self-Reflection: When an action diverges from the expected outcome, the LLM triggers reflection, querying additional data sources and replanning.

3. Prototype Realization

The framework is instantiated in a ROS2 and Gazebo Harmonic simulation, emulating realistic search-and-rescue scenarios:

• Sensor Integration: RealSense D455 visual stream, YOLOv11 for real-time object detection (30 Hz), and extended Kalman filter fusion for probabilistic 3D scene modeling.
• Cognitive Module: GPT-4 or local Gemma-3 LLMs operate within a LangGraph-based computational graph for multistep plan generation and tool-calling.
• Actuator Loop: Action recommendations—physical (flight control) or digital (e.g., dispatching email alerts with GPS/imagery)—are fed back into the simulation and logged for subsequent plan verification or adaptation.

4. Performance Metrics and Empirical Results

The system demonstrates marked improvements over rule-based UAV frameworks in the following domains:

Metric	Rule-based (YOLO)	Agentic (Gemma-3)	Agentic (GPT-4)
Detection confidence	0.716	0.760	0.790
Person detection rate (%)	75	84	91
Action recommendation rate (ARR, %)	0	79	92
Contextual analysis rate (CAR, %)	0	88	94
Processing time (s)	0.00003	1.48	4.95

• Action Recommendation Rate (ARR):

$$\text{ARR} = \frac{\text{Number of instances with recommended actions} \neq \emptyset}{\text{Total detections}} \times 100$$

• Contextual Analysis Rate (CAR):

$$\text{CAR} = \frac{\text{Number of detections with meaningful (>|50| char) scene context}}{\text{Total detections}} \times 100$$

Statistical validation (ANOVA, effect sizes) confirms these increases are significant. Although agentic processing incurs orders-of-magnitude higher latency than YOLO-only detection, it enables qualitatively new cognitive and contextual capacities.

5. Computational Overhead and Deployment Strategies

The addition of LLM-driven reasoning and tool-calling introduces non-negligible processing delay (from microseconds to 1–5 seconds per decision), but this cost is justified for missions requiring:

• High-confidence, context-rich decision-making
• Autonomous action recommendations
• Flexible integration with digital ecosystems (alerts, knowledge bases)

To achieve practical deployment envelopes, the paper proposes a hybrid selection strategy: frequent tasks are handled by fast rule-based modules, while agentic UAV workflows are selectively activated for critical or novel events.

6. Ecosystem Integration and Multi-Agent Coordination

The Integration Layer, via MCP/ACP/A2A protocols, enables:

• Secure data and command exchange with cloud systems, weather services, or mission databases
• Multi-UAV swarm coordination for distributed perception and action
• Real-time participation in broader cyber-physical networks (e.g., emergency infrastructure, distributed robotics teams)

This integration supports collaborative decision-making and system-wide knowledge sharing, critical in mission-critical scenarios such as disaster response or large-scale surveillance.

7. Implications and Future Directions

The Agentic UAVs framework represents a paradigm shift, blurring the line between perception/execution platforms and autonomous reasoning agents:

• Toward SAE Level 4–5 autonomy: Beyond narrow rule-based control, agentic UAVs can interpret ambiguous goals, adapt to environmental uncertainty, and execute multi-step plans.
• Qualitative advancement: By integrating scene semantics, external tool invocation, and learning layers, these systems provide capabilities (e.g., explanation, mission replanning, context-aware alerting) previously inaccessible to legacy UAV control stacks.
• Scalability and Adaptability: The modular architecture supports deployment in fleet-wide contexts, enabling fleetwise transfer learning, memory-driven adaptation, and distributed agency for large-scale aerial missions.

Further research is expected to focus on optimization of cognitive module deployment, further reduction in agentic processing latency, and broadening of ecosystem connectivity—paving the way for general-purpose, reasoning-enabled aerial agents deployable in complex, high-stakes environments (Koubaa et al., 14 Sep 2025).