Decentralized Ergodic Coverage Control in Unknown Time-Varying Environments

Abstract: A key challenge in disaster response is maintaining situational awareness of an evolving landscape, which requires balancing exploration of unobserved regions with sustained monitoring of changing Regions of Interest (ROIs). Unmanned Aerial Vehicles (UAVs) have emerged as an effective response tool, particularly in applications like environmental monitoring and search-and-rescue, due to their ability to provide aerial coverage, withstand hazardous conditions, and navigate quickly and flexibly. However, efficient and adaptable multi-robot coverage with limited sensing in disaster settings and evolving time-varying information maps remains a significant challenge, necessitating better methods for UAVs to continuously adapt their trajectories in response to changes. In this paper, we propose a decentralized multi-agent coverage framework that serves as a high-level planning strategy for adaptive coverage in unknown, time-varying environments under partial observability. Each agent computes an adaptive ergodic policy, implemented via a Markov-chain transition model, that tracks a continuously updated belief over the underlying importance map. Gaussian Processes are used to perform those online belief updates. The resulting policy drives agents to spend time in ROIs proportional to their estimated importance, while preserving sufficient exploration to detect and adapt to time-varying environmental changes. Unlike existing approaches that assume known importance maps, require centralized coordination, or assume a static environment, our framework addresses the combined challenges of unknown, time-varying distributions in a more realistic decentralized and partially observable setting. We compare against alternative coverage strategies and analyze our method's response to simulated disaster evolution, highlighting its improved adaptability and transient performance in dynamic scenarios.

• The paper introduces a decentralized framework that integrates ergodic Markov chain policies with GP-based belief updates for adaptive multi-agent coverage.
• It demonstrates that localized, rapid communications enable agents to effectively track dynamic regions of interest while minimizing empirical visitation regret.
• Experimental results reveal that the proposed method outperforms greedy baselines in scalability, adaptability, and robust coverage performance.

Decentralized Ergodic Coverage Control in Unknown Time-Varying Environments

Problem Formulation and Motivation

This paper presents a formalism and solution for decentralized multi-agent coverage control in environments exhibiting unknown, dynamic, and partially observable regions of interest (ROIs). The real-world motivation is disaster response, where UAVs are increasingly deployed for persistent situational awareness, but must operate in settings that are characterized by evolving hazards, communication constraints, and incomplete knowledge of the environment. Unlike approaches relying on static importance maps or centralized coordination, the proposed work addresses the need for decentralized, adaptive policies robust to both unknown map evolution and local sensing constraints.

The coverage objective is expressed through empirical visitation regret: minimizing the cumulative discrepancy between the space-time visitation distribution achieved by a team of UAVs and an evolving, unobserved spatial importance map encoding priority ROI locations. To achieve this, the authors integrate multi-agent ergodic control, probabilistic map inference, and graph-based rapid mixing policies—targeting scalability and adaptivity to environmental dynamics.

System Model and Framework

The workspace is discretized as a graph $\mathcal{G} = (\mathcal{R}, \mathcal{E})$, with regions $\mathcal{R}$ connected via edges $\mathcal{E}$, capturing constraints such as no-fly zones and obstacles. Each UAV agent operates as a node, collects localized noisy observations, and communicates only within a finite radius.

The true ROI distribution is dynamic and unknown, with abrupt changes represented stochastically at discrete time points. Each agent computes, from its local and neighbor-shared observations, a belief map via a Gaussian Process (GP) regression. These belief maps are then exploited via Upper Confidence Bounds (UCBs) to support both information-driven exploitation and spatial exploration.

The central control mechanism is the use of ergodic Markov chain policies. Each agent constructs a rapidly mixing transition kernel over $\mathcal{G}$ using the REMC method, ensuring, under static assumptions, that visitation frequencies converge to the inferred target distribution. In the dynamic setting, agents recompute policies and beliefs online, yielding a procedure for continual adaptation to environmental changes.

Figure 1: Map exploration performance as a function of communication radius and team size, evaluating time to first reach an ROI and to achieve full map coverage for varying numbers of UAVs.

Policy Synthesis and Online Adaptation

Each agent executes the following iterative loop:

1. Local observation acquisition: Sensing within a prescribed radius.
2. Neighborhood data sharing: Exchanging local observations with communicating neighbors.
3. Belief map update: Fitting a GP model to the aggregate dataset and constructing a UCB-based conservative information estimate.
4. Policy generation: Solving for a new stochastic transition matrix using REMC, with the updated belief map as the ergodicity objective.
5. Decentralized action selection: Transitioning within the induced Markov chain to a neighboring region.

This process is asynchronous and robust to partial information. Theoretical guarantees exist under static maps, but the empirical regret is shown to be controlled provided the environment does not change faster than the belief/policy update rates.

Experimental Evaluation

Performance is extensively validated in realistic simulations, varying map structures, ROI distributions, team sizes, and communication radii. Key experimental findings include:

• Policy update rates: Maintaining a policy/belief update period that is a small fraction of the environment change timescale is critical for keeping regret low. Update rates exceeding the rate of environmental dynamics are necessary for robust adaptation, but excessively frequent updates yield diminishing returns relative to computation.
• Communication constraints: Local communication (one or a few hop radius) is generally sufficient for rapid discovery of ROIs and map coverage, especially as team size increases. Belief convergence (quantified via KL divergence with respect to the network-wise mean) is only marginally improved with global communication for large teams.
• Adaptation to dynamic events: The regimen effectively tracks shifting ROIs or spreading hazards, maintaining balanced exploration and exploitation through ergodic objectives rather than myopic reward maximization.
• Comparison to MAC-DT: The proposed framework significantly outperforms the Multi-Agent Coverage with Doubling Trick (MAC-DT) baseline, especially for long-horizon performance and when the ROI distribution is dynamic. MAC-DT's greedy GP-directed strategy exhibits increased regret and coverage inefficiency, with only sporadic ROI discovery and poor adaptation to environmental shifts.

(Figure 2)

Figure 2: GP-based policy and belief update timing impact on regret and coverage error metrics under different environment evolution schedules.

(Figure 3)

Figure 3: KL-divergence between each UAV’s local belief and the global mean for various communication radii and team sizes, demonstrating that moderate communication yields rapid consensus.

The framework enables robust decentralized multi-agent coordination, as quantified by both coverage time and belief convergence metrics.

Implications and Theoretical Perspectives

This framework embodies a principled integration of ergodic control and online Bayesian inference in multi-agent systems. The Markov chain construction via REMC is critical—it ensures that agent trajectories mix rapidly to estimated belief distributions, providing strong guarantees on long-term visitation statistics. GP-based UCB modeling suits nonparametric, partially observed, and time-varying spatial maps—directly addressing unknown and dynamic settings in which classical coverage, assignment, or information-maximization techniques either fail or exhibit brittle behavior.

The method’s reliance on only local communication enhances its scalability and resilience to real-world constraints (e.g., communication dropouts, heterogeneous agent capabilities). The regret analysis formalizes desirable properties in evolving disaster scenarios: the team tracks spatial priorities as long as environmental volatility does not vastly outpace policy updates.

Beyond disaster response, the generality of this ergodic Bayesian framework makes it suitable for problems in environmental monitoring, exploration in unknown or hazardous domains, and other persistent information-gathering tasks where operational objectives require both adaptive exploration and exploitation of time-variant priorities.

Future Directions

Areas for extension include analysis of scalability in very large graphs and further understanding of transient performance under rapid or adversarial environmental changes. Adaptive selection of policy and GP update rates (e.g., event-triggered updates upon detecting distribution shift) could reduce computational load without harming performance. Practical deployment would benefit from further integration with path planning under kinematic and energy constraints as investigated in energy-aware ergodic search literature. Bridging the remaining gap between theoretical regret guarantees in fully dynamic maps and real-world stochasticity is also an open question.

Conclusion

This paper advances decentralized active coverage in unknown, dynamic environments by combining ergodic Markov policies and Gaussian Process-based belief adaptation under partial observability and communication constraints. The approach is validated across diverse scenarios, demonstrating robustness, improved coverage, and adaptability compared to greedy alternatives. These results strengthen the theoretical and practical case for ergodicity-driven decentralized multi-agent planning as a foundation for scalable, resilient autonomous systems.