Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learning

Abstract: Monitoring human activity in indoor environments is important for applications such as facility management, safety assessment, and space utilization analysis. While mobile robot teams offer the potential to actively improve observation quality, existing multi-robot monitoring and active perception approaches typically rely on coverage or visitation based objectives that are weakly aligned with the accuracy requirements of human-centric monitoring tasks. In this work, we formulate cooperative active observation as a decentralized control problem in which multiple robots adjust their motion to directly optimize monitoring accuracy under partial observability. We propose a learning-based framework for cooperative policies from decentralized observations using multi-agent reinforcement learning (MARL), supported by an architecture that handles variable numbers of humans and temporal dependencies. Simulation results across diverse indoor environments and monitoring tasks show that the proposed approach consistently outperforms classical coverage, persistent monitoring, and learning-free multi-robot baselines, while remaining robust to changes in the number of observed humans.

• The paper proposes a decentralized MARL framework that reframes active sensing as a Dec-POMDP to enhance human state estimation in dynamic indoor spaces.
• It employs set-based observation encoding and dual-stage recurrent memory to enable effective multi-robot coordination under occlusions and partial observability.
• Empirical evaluations show significant improvements in tracking accuracy and scalability, with hybrid mobile-fixed strategies further optimizing sensing performance.

Cooperative Informative Sensing in Dynamic Indoor Environments via Multi-Agent Reinforcement Learning

Problem Statement and Motivation

The paper "Cooperative Informative Sensing for Monitoring Dynamic Indoor Environments via Multi-Agent Reinforcement Learning" (2604.23179) fundamentally advances multi-robot observation strategies for monitoring human activities in complex indoor environments. Traditional methods—such as coverage, persistent monitoring, and visitation-based strategies—fail to directly optimize for the accuracy of human-centric tasks under severe occlusion, mobile occluders (humans), and complex topologies. This work reframes the sensing objective as a Dec-POMDP to learn decentralized, task-specific active sensing policies. The emphasis moves from spatial coverage towards maximizing the informativeness of observations conditional on latent human states, enabling flexible adaptation across tasks and improved scalability.

Figure 1: Cooperative active observation scenario showing mobile robots controlling their motion to maximize the FoV overlap with humans, supporting persistent monitoring under partial observability and occlusions.

Methodology

Dec-POMDP Formulation

The environment consists of $n$ mobile robots and $m$ dynamic human agents in a 2D indoor layout with occlusions represented by walls and rooms. Robots sense via LiDAR for obstacle detection and cameras for human observation, with limited FoV. Each robot receives partial, noisy observations, with human visibility determined by a function incorporating spatial and sensor constraints.

The team objective is to minimize the estimation error of human state variables (e.g., location, zone occupancy, flow between zones), rather than maximizing visitation frequency. The reward is defined as the $\ell_1$-norm of the improvement in belief estimates, directly reflecting the informativeness of new observations for downstream monitoring tasks.

Policy Architecture

A significant contribution is the network design enabling robust decentralized processing and coordination:

• Set-based Observation Encoding: Human observations per robot are modeled as unordered sets and encoded via permutation-invariant attention. This ensures architectural compatibility regardless of the number of humans present or the order in which they are observed.
• Dual-stage Recurrent Interaction Memory: The architecture preserves temporal memory through an ego-GRU for individual history, followed by inter-robot coordination via attention and a subsequent interaction GRU. This enables scalable agent interaction and decentralized policy execution without explicit peer tracking.
• Centralized Critic for Training: Value estimation aggregates information from all robots using attention pooling, but during execution, each robot acts solely on local inputs, ensuring decentralization.

Figure 2: Policy network schematic highlighting set-based feature extraction, dual-stage memory, and decentralized policy/value estimation.

Multi-Agent Reinforcement Learning and Training Protocol

Policy training leverages MAPPO with shared parameters across agents and centralized value estimation for improved credit assignment under partial observability. The environment simulator incorporates realistic human trajectories, map generation, and occlusion modeling. Zero-shot evaluation verifies both the scalability with varying numbers of agents and robustness to distributional shifts in human population and behavior.

Numerical Evaluation and Empirical Results

The MARL approach was benchmarked against classical fixed camera placement (FC), waypoint sampling (WS), coverage path planning (MCPP), and persistent monitoring (PM) baselines across multiple tasks: individual human tracking, zone occupancy estimation, and inter-zone human flow estimation.

Figure 3: Comparative performance—mean errors in tracking, zone occupancy, and flow decrease consistently with MARL as team size increases.

Major findings:

• Superior Performance across Tasks: MARL achieves the lowest mean tracking, occupancy, and flow errors, maintaining strong zero-shot generalization to smaller team sizes.
• Robustness in OOD Settings: Evaluation under population shifts and skewed dynamics demonstrates minimal degradation for MARL, with particularly good performance in scenarios where conventional fixed infrastructure loses coverage flexibility.

Figure 4: Tracking error demonstrates MARL's resilience to out-of-distribution scenarios (e.g., population size, trajectory shifts).

• Marginal Utility Saturation: Increasing robots reduces error, but with diminishing returns after a threshold (typically 5 robots in experiments). Fixed cameras excel only as numbers become large and optimal placement is feasible, but mobile teams achieve better marginal gains with smaller teams due to coverage adaptability.

Figure 5: Tracking error and diminishing marginal utility as robot team size increases, with MARL showing favorable scaling.

• Hybrid Mobile-Fixed Integration: Combining a limited number of fixed cameras with mobile MARL-controlled agents outperforms either modality alone, provided placements are judiciously chosen. The learned policy drives mobile robots to complement rather than overlap with static coverage.

Figure 6: Mean tracking errors under hybrid teams, demonstrating the performance advantage of mixed agent configurations.

Figure 7: Observed-region heatmaps highlight how MARL-driven robots adaptively avoid redundant coverage in the hybrid setup.

• Camera Placement Significance: Empirical studies reveal that optimal placement of fixed cameras targets peripheral zones, freeing robots for more complex, high-cost traversal areas. Central placements add little marginal coverage value.

Figure 8: Visualization of seven candidate fixed camera placements, with peripheral locations yielding greatest tracking error reduction.

• Reward-Performance Correlation: There exists a strong negative correlation between the learned reward and actual monitoring performance, confirming that MARL's reward design aligns with error minimization.

Figure 9: Strong negative linear correlation between mean episode reward and tracking error across differing behaviors and teams.

Implications and Future Directions

This research emphasizes the potential of MARL for cooperative active sensing in dynamic, occluded indoor settings. The use of belief-driven rewards and set-based policy architectures enables agent teams to attain flexible, adaptive coverage, with robust zero-shot transfer and out-of-distribution generalization. Hybrid strategies highlight the practical value of integrating mobile and fixed assets, with optimal fixed sensor placements allowing mobile robots to focus on high-cost or blind spots.

From a theoretical perspective, this work demonstrates the superiority of reward structures aligned with downstream information-theoretic goals, and establishes architectural standards for processing unordered entity observations and temporal-memory-based coordination. Practically, the findings indicate that mobile MARL-driven sensing can enhance legacy static monitoring systems, especially in environments with unpredictable occupancy dynamics.

Future directions include extension to high-fidelity 3D simulators and real-world robotics (processing raw sensor data and integrating visual-semantic pipelines), inclusion of richer semantic constraints (beyond position and zone transitions), and leveraging large-scale language/vision-LLMs to further improve coordination, explainability, and transferability across new environments and tasks.

Conclusion

The presented work delivers a rigorously benchmarked, architecturally novel MARL framework for informative multi-robot indoor sensing explicitly targeted to dynamic, human-centric monitoring objectives. Numerical evidence robustly supports the approach’s superiority over prior art in both conventional and distribution-shifted scenarios. The results motivate deeper inquiry into hybrid sensor integration strategies and simulations-to-real transfer, foreshadowing substantial gains in cost-effective, adaptable monitoring for built environments.