- The paper proposes a DRL framework that jointly optimizes beamforming and reconfigurable FIM shapes to minimize the DOA estimation CRB in ISAC systems.
- It employs a custom DDPG algorithm that effectively navigates a hybrid continuous/discrete action space while adhering to power and QoS constraints.
- Simulation results demonstrate significantly lower CRB and improved multiuser performance compared to conventional rigid array designs.
Introduction
The paper presents a deterministic deep reinforcement learning (DRL) framework for jointly optimizing beamforming and the three-dimensional (3D) surface morphing of Flexible Intelligent Metasurfaces (FIMs) in Integrated Sensing and Communication (ISAC) systems. The optimization aims to minimize the Cramér–Rao Bound (CRB) for direction-of-arrival (DOA) estimation, subject to transmit power and Quality of Service (QoS) constraints. By leveraging the unique morphing capabilities of FIMs, the approach introduces additional degrees of freedom over conventional rigid array (RA) ISAC implementations, optimizing both electromagnetic exposure and array geometry in a dynamic wireless setting.
Technical Contributions
The joint design problem addressed is fundamentally non-convex due to the mutual coupling between continuous-valued beamforming coefficients and discretized FIM surface deformations. This paper's main technical contributions include:
- Unified Formulation: The joint optimization over beamforming matrices and FIM element positions for both transmit and receive arrays, formulated to directly minimize the DOA estimation CRB while ensuring communication constraints for multiple users.
- Constraint-Aware DRL Framework: The use of a deep deterministic policy gradient (DDPG) algorithm with a custom reward structure that penalizes violations of power, QoS (SINR), and deformation constraints in the instantaneous system state. The agent operates in a high-dimensional, hybrid continuous/discrete action space, efficiently navigating the solution landscape via gradient-based actor-critic methods.
- Custom State/Action Design: Formal definitions of the system state and agent action parameters, incorporating not only the communication and sensing performance metrics but also explicit tracking of constraint violations. This supports consistent convergence of the DRL agent.
- Scalable Simulation Architecture: Realization of the above in a multi-user, multi-target ISAC setting, using high-resolution, high-capacity neural architectures and replay buffers for stable policy learning.
Key Results and Comparative Evaluation
Simulation studies robustly demonstrate the superiority of the proposed FIM-aided ISAC design over conventional alternatives in terms of parameter estimation accuracy as measured by the CRB.
- CRB Reduction: The DRL-based joint optimization achieves significantly lower CRB values compared to both the classical rigid array design and schemes optimizing transmit or receive FIM shapes only. The performance gap widens with increased numbers of users, where joint morphing and beamforming is critical for mitigating multiuser interference and maintaining Fisher information for DOA estimation.
- Convergence and Stability: The actor-critic DRL framework displays stable convergence with carefully selected learning rates. Overly high or low learning rates were shown to cause instability or slow convergence, aligning with best practices for deep RL in hybrid action spaces.
- Power and QoS Allocations: Increasing the transmit power improves the CRB as expected, but the FIM-based architecture consistently outperforms alternatives across all power regimes due to the morphable geometry's additional control. Higher QoS thresholds necessarily raise CRB values since resource allocation priorities shift from sensing to communication, confirming the reward function's correctness.
- Robustness Across Scenarios: The DRL-trained policy generalizes across variations in user count, transmit power budget, and QoS thresholds, demonstrating both adaptability and resilience of the joint design strategy.
Theoretical and Practical Implications
The integration of DRL for joint beamforming and array geometry adaptation marks a substantial advancement in realizing practical ISAC systems with dynamically reconfigurable metasurfaces. From a theoretical perspective, the approach validates the value of spatial degree-of-freedom enhancement, as morphing FIMs can directly modify the array steering vectors and hence the system Fisher information, which classical RAs and even discrete meta-atom RIS solutions cannot achieve.
Practically, this suggests that future 6G and beyond wireless networks should heavily invest in both the hardware deployment of flexible metasurface structures and intelligent learning-based optimization stacks that leverage these new reconfiguration capabilities. The DRL methodology, particularly with constraint-aware reward designs, offers a viable path to addressing the non-convex, multi-modal optimization space of real-world ISAC deployments. Moreover, the presented techniques lay groundwork for further research into wideband, distributed, and hardware-constrained ISAC systems, where adaptive morphing can be combined with other reconfigurable elements (e.g., STAR-RIS, SIM) for additional performance improvements.
Future Research Directions
Several key avenues emerge:
- Wideband ISAC Extensions: While the present work focuses on narrowband single-target estimation, the optimization problem becomes significantly more complex in distributed, wideband, or multi-target regimes. The DRL framework here could be extended to such settings, incorporating frequency-selective morphing and waveform adaptation.
- Hardware Constraints and Actuation Latency: Realizing practical FIMs will involve actuation power limits, finite morphing speed, and mechanical reliability issues. Integrating these hardware realities into the DRL framework (e.g., via action masking or penalization) remains an open challenge.
- Transfer and Multi-Agent RL: For networked ISAC with multiple flexible arrays, multi-agent RL methods—or meta-learning for adaptation across device types—may offer further improvements in scalability and performance.
Conclusion
This study establishes a blueprint for the application of DRL in the joint optimization of beamforming and array geometry for FIM-enabled ISAC systems. By formalizing a multi-constraint, high-dimensional control policy, the proposed approach uniquely exploits the full reconfiguration potential of FIMs, achieving superior DOA estimation performance under practical communication constraints. The results motivate further theoretical and practical innovations at the intersection of flexible hardware design and intelligent control for next-generation wireless ISAC applications.