FH-DRL: Exponential-Hyperbolic Frontier Heuristics with DRL for accelerated Exploration in Unknown Environments

Abstract: Autonomous robot exploration in large-scale or cluttered environments remains a central challenge in intelligent vehicle applications, where partial or absent prior maps constrain reliable navigation. This paper introduces FH-DRL, a novel framework that integrates a customizable heuristic function for frontier detection with a Twin Delayed DDPG (TD3) agent for continuous, high-speed local navigation. The proposed heuristic relies on an exponential-hyperbolic distance score, which balances immediate proximity against long-range exploration gains, and an occupancy-based stochastic measure, accounting for environmental openness and obstacle densities in real time. By ranking frontiers using these adaptive metrics, FH-DRL targets highly informative yet tractable waypoints, thereby minimizing redundant paths and total exploration time. We thoroughly evaluate FH-DRL across multiple simulated and real-world scenarios, demonstrating clear improvements in travel distance and completion time over frontier-only or purely DRL-based exploration. In structured corridor layouts and maze-like topologies, our architecture consistently outperforms standard methods such as Nearest Frontier, Cognet Frontier Exploration, and Goal Driven Autonomous Exploration. Real-world tests with a Turtlebot3 platform further confirm robust adaptation to previously unseen or cluttered indoor spaces. The results highlight FH-DRL as an efficient and generalizable approach for frontier-based exploration in large or partially known environments, offering a promising direction for various autonomous driving, industrial, and service robotics tasks.

• The paper introduces SHANGUS, a framework that combines DRL with heuristic frontier selection to optimize autonomous vehicle exploration.
• It employs the TD3 algorithm and a novel occupancy scoring method to enhance path planning and dynamic obstacle avoidance.
• Experimental evaluations show SHANGUS outperforms traditional methods in completion times and exploration rates across diverse scenarios.

SHANGUS: Integrating Deep Reinforcement Learning with Heuristic Optimization for Autonomous Exploration

The paper presents SHANGUS, a novel framework that integrates Deep Reinforcement Learning (DRL) with heuristic optimization techniques to advance autonomous vehicle exploration in unknown environments. Specifically, it capitalizes on the adaptability of DRL in dynamic obstacle avoidance, paired with heuristic-driven frontier exploration, to improve operational efficiency in intelligent autonomous vehicles engaged in tasks such as search and rescue, industrial automation, and space exploration.

Framework and Methodology

The central innovation of SHANGUS lies in its dual-component system: a frontier selection node and a DRL navigation node. The frontier selection node identifies unexplored regions using a novel algorithm, while the navigation node employs the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. This combination enables robust path planning and dynamic obstacle avoidance, significantly addressing existing inefficiencies in traditional frontier-based exploration strategies.

The framework prioritizes frontier points using a heuristic function that assigns normalized scores based on distance and occupancy values. This prioritization helps in choosing valuable exploration points, thereby optimizing the path for efficient navigation. The introduction of the occupancy stochastic score serves as a key determinant in evaluating exploration worth by analyzing map regions' occupancy values.

Empirical Evaluation

Rigorous experimental simulations compare SHANGUS against the Nearest Frontier (NF), Novel Frontier-Based Exploration (CFE), and Goal-Driven Autonomous Exploration (GDAE) algorithms across environments of varying complexities. Metrics used in the experiments include average travel distance, completion time, and exploration rate. Notably, SHANGUS, especially with DRL integration, consistently excels, achieving the shortest completion times and highest exploration rates across tested scenarios.

Implications and Future Prospects

The strong numerical results underscore SHANGUS's capability to elevate autonomous navigation operations. Its robustness and scalability underscore its practicality across a broad spectrum of applications, from autonomous driving to household robotics. The integration of heuristic optimization with DRL yields a synergistic effect, streamlining exploration and decision-making processes. The framework's potential contribution to real-time autonomous exploration fosters advancements in robotics and AI, particularly in environments characterized by uncertainty and complexity.

Future research prospects involve enhancing SHANGUS's efficiency further by incorporating additional sensory inputs and refining heuristic functions. These improvements would aim to bolster its applicability in even more challenging dynamic environments.

In conclusion, SHANGUS exemplifies an advanced strategy in autonomous exploration, providing a framework that robustly combines heuristic methods with DRL to optimize frontier-based navigation. Its ability to outperform traditional methods positions it as a promising direction for future intelligent robotic systems.