Lightweight Safe Reinforcement Learning for End-to-End UAV Navigation

Abstract: With the rapid development of autonomous aerial systems, Unmanned Aerial Vehicles (UAVs) are increasingly deployed in applications such as inspection, environmental monitoring, and rescue, creating growing demand for reliable autonomous navigation. However, autonomous UAV navigation in dense environments remains challenging under sparse perception and dynamic constraints. Most reinforcement learning (RL) methods lack explicit safety mechanisms, leading to unsafe exploration, unstable training, and risky behaviors, especially during high-speed flight. Even in safe RL approaches, safety is often enforced by projecting policy outputs onto a safe action set, which may introduce instability. Meanwhile, many learning-based methods rely on dense inputs or large networks, increasing computational burden and limiting lightweight onboard deployment. Facing the above challenges, we propose a safety-constrained perception-control integrated framework for UAV navigation. A lightweight network encodes sparse observations into collision-risk-aware features using asymmetric and depthwise separable convolutions. We formulate the task as a constrained Markov decision process within a hierarchical control architecture and solve it using a Lagrangian-based safe PPO algorithm. Curriculum learning further improves training stability. Experiments with varying obstacle densities and flight speeds demonstrate higher success rates, improved safety, and better efficiency than existing reinforcement learning baselines.

• The paper introduces a lightweight LiDAR feature extractor and safety-aware RL framework that integrates sparse perception with explicit collision avoidance.
• The methodology employs Lagrangian Safe PPO and curriculum learning to balance reward maximization and safety constraints in complex, high-density environments.
• Experimental results demonstrate high success rates (above 0.94) and significant model efficiency, making the approach suitable for SWaP-limited UAVs.

Safety-Constrained Lightweight RL for UAV Navigation under Sparse Perception

Introduction

This paper addresses the challenge of safe UAV navigation in complex, cluttered environments under the constraints of sparse perception and computationally limited onboard platforms. Conventional navigation strategies often rely on classical planners or high-capacity RL models leveraging dense LiDAR or image sensors, which are inadequate for deployment on lightweight aerial vehicles experiencing strict computational and sensing bottlenecks. Moreover, standard RL approaches frequently overlook explicit safety modeling, thereby suffering from unsafe exploration and poor reliability in dense environments. The paper proposes a perception-control integrated framework utilizing a lightweight convolutional network and a Lagrangian-based Safe PPO, combined with curriculum learning, to enable robust and efficient UAV navigation with explicit safety guarantees.

Figure 1: Overview of the proposed safety-aware PPO framework integrating sparse LiDAR features for reward and cost estimation in a CMDP setting.

Lightweight Perception Architecture

Sparse multi-beam LiDAR is prevalent in resource-limited UAVs, but its restricted vertical resolution complicates reliable perception of obstacles, especially continuous boundaries and narrow gaps. The paper introduces a compact convolutional encoder that leverages asymmetric and depthwise-separable convolutions to efficiently process this sparse input and extract collision-risk-aware features.

Figure 2: Architecture of the LiDAR Feature Extraction Network, showing the sequence of Conv and DSConv layers for efficient geometric encoding.

Depthwise-separable convolutions significantly curtail parameter count and computational demand relative to conventional convolutions while preserving the network's capacity to model local and global geometric features.

Figure 3: Schematic illustration of Depthwise-Separable Convolution, highlighting the separation of spatial and channel-wise filtering.

This design is particularly suited for real-time operation on low-latency and low-power onboard processors, eliminating the need for oversized models without sacrificing detection of subtle structural features in the input.

Safe RL and Policy Optimization

The UAV navigation task is formalized as a Constrained Markov Decision Process (CMDP) in which the agent must maximize task efficiency (e.g., velocity along goal direction, altitude regulation, action smoothness) subject to explicit safety constraints (e.g., collision avoidance, proximity margins, boundary compliance). The Safe PPO algorithm, augmented by Lagrangian relaxation, integrates a dual-objective optimization, where a learnable Lagrange multiplier balances the policy's reward maximization and constraint satisfaction.

The policy is realized as a hierarchical control stack: the RL agent outputs body-frame velocity commands, sampled from a Beta-distributed bounded policy, that are subsequently tracked by a geometric controller. The network employs a shared feature encoder, supplying latent representations to the policy (actor), value (critic), and cost critic branches, facilitating efficient representation learning while minimizing model redundancy.

Cost modeling combines hard signals (collision indicator) and soft penalties (proximity margin) to ensure informative gradient signals for pre-collision avoidance rather than post hoc penalization. Policy updates are performed using advantage estimates that combine reward and cost terms, with the Lagrange multiplier dynamically tuned by dual gradient ascent to enforce constraints throughout training.

Curriculum Learning for Stability

Safe RL under high environmental complexity and sparse perception is susceptible to unstable convergence and suboptimal exploration. The curriculum learning strategy progressively stages training from simpler (lower-density) to harder (higher-density) environments, reusing and transferring encoder weights between stages. Actor output layers are re-initialized during transfer, ensuring continued exploration and preventing overfitting to prior scenarios. This approach expedites convergence and confers policy robustness as task complexity escalates.

Experimental Results

Task and Evaluation Setup

Experiments are conducted in photo-realistic 3D forest environments with UAVs equipped with 1×36×4 sparse LiDAR—a highly challenging perception regime. Obstacles range from 100 to 300 in number, generating both sparse and extremely cluttered environments.

Figure 4: Simulation environments illustrating increasing obstacle density: from left (100) to right (300) obstacles.

Performance is assessed primarily by Success Rate (SR), Collision Rate (CR), Average Return (AR), and Average Episode Length (AEL).

Ablation and Module Analysis

Systematic module-wise ablation across all obstacle densities highlights distinct contributions:

• Depthwise-separable convolutions enhance feature discrimination and elevate success rate, particularly in high-density scenes.
• Safe RL with explicit cost modeling markedly improves navigation robustness and suppresses collision rate compared to standard PPO.
• Curriculum learning yields further gains in SR as environmental complexity rises, corroborating that progressive task difficulty enables superior policy generalization.

Comparison to RL Baselines

The proposed method is benchmarked against PPO, PPO+GRU, PPO+LSTM, SAC, and TD3, both in terms of navigation performance and model compactness. Across all densities, the proposed model consistently registers the highest SR (e.g., 0.945 in 300-obstacle environments), with recurrent variants offering only marginal benefit at a steep computational and sample efficiency cost.

Figure 5: Quantitative comparison with RL baselines under differing obstacle densities, showing superior navigation success for the proposed method.

Additionally, the model exhibits an order-of-magnitude lower parameter count compared with these baselines, underscoring its suitability for real-time onboard deployment.

Robustness to Velocity and Dynamic Constraints

Systematic evaluation under varying maximum flight velocities and obstacle densities confirms reliable performance. The navigation SR is sustained above 0.94 up to moderate speeds (6–8 m/s), with graceful degradation observed at higher velocities and extreme clutter, reflecting the practical trade-off between dynamic agility and perception-limited reaction time.

Figure 6: Navigation success rate as a function of maximum velocity, delineating the robust operating range under different obstacle densities.

Overall, the framework demonstrates robust policy stability even as both dynamics and environmental complexity escalate.

Implications and Future Prospects

This research demonstrates that explicit safety constraints, lightweight perception modeling, and staged curriculum training can jointly enable sample-efficient, deployable, and robust end-to-end RL-based UAV navigation under sparse sensing—conditions emulating real-world SWaP-limited robotic platforms. The strong empirical results, especially SR near or above 0.94 even in the most difficult scenarios, challenge the notion that high capacity or dense sensors are prerequisites for safe RL-based navigation. The results also directly contradict the common perception that strong model compression necessarily degrades RL policy performance under complex, high-risk tasks.

Practically, these advances provide a viable pathway for deploying RL navigation stacks on commodity UAV hardware with modest sensing and computation, scalable to diverse and previously intractable environments, particularly where global mapping and dense reconstruction are infeasible.

Theoretically, the integration of Lagrangian-based constrained RL with perceptual feature transfer and staged task escalation could be explored for broader classes of safety-critical embodied agents. Future directions include extending to dynamic obstacles, multi-agent interaction, and the integration of uncertainty-aware state estimation to further increase real-world robustness under sensing noise, localization drift, or partial observability.

Conclusion

The paper delivers a comprehensive solution to end-to-end UAV navigation with explicit safety guarantees under severe perceptual and computational constraints. The combination of a lightweight LiDAR encoder, Lagrangian Safe PPO, and curriculum-based staged training yields strong empirical results, with high navigation success rates maintained in increasingly complex environments and under various flight regimes. The framework lays the groundwork for scalable deployment of RL agents in real-world, resource-constrained robotic applications where safety is paramount.