Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization

Abstract: We present a Multi-task Soft Actor-Critic (SAC) Reinforcement Learning framework designed for open-system quantum control across diverse Hamiltonians, which learns optimal pulse sequences while simultaneously discovering problem-specific evolution time T and number of control pulse segments N. Experimental results across 51 Hamiltonian variations demonstrate that the multi-task SAC model is able to generate control pulses that can drive a system, under environment noise, from its initial state to its target state with high fidelities, establishing essential foundations for universal quantum control applicable to realistic noisy quantum devices. Through progressive expansion of the training Hamiltonian set, we investigate if a single multi-task model trained using a given number of sample Hamiltonians can successfully accomplish state-transfer tasks for Hamiltonians drawn from the same Hamiltonian space but not encountered during training. In addition, our Robustness Infidelity Measure (RIM) analysis reveals that SAC trained policies exhibit superior robustness to pulse amplitude perturbations and decoherence rate variations compared to GRAPE-optimized controls.

• The paper introduces a unified multi-task SAC framework that jointly optimizes evolution time, pulse discretization, and amplitudes for high-fidelity quantum state transfer.
• The approach achieves near-ideal fidelities (up to 0.999 in closed systems) and robust performance (median fidelity above 0.95) under open-system conditions.
• Adaptive temporal optimization minimizes decoherence and computational load, outperforming GRAPE by maintaining lower infidelity under perturbations.

Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization

Framework Overview and Methodological Foundations

The paper "Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization" (2605.26925) introduces a unified multi-task Soft Actor-Critic (SAC) RL architecture for quantum state transfer across diverse Hamiltonian classes under open-system Lindblad dynamics. Unlike conventional approaches—GRAPE or task-specific RL—that require per-model training and assume fixed control times and pulse segment counts, this framework integrates both adaptive temporal optimization and multi-task conditioning. The agent simultaneously learns problem-specific total evolution time ($T$), discretization steps ($N$), and optimal pulse amplitudes for systems with varying size, drift, and control structure.

The SAC policy is conditioned on a descriptor vector encoding Hamiltonian features, facilitating efficient transfer across single-, two-, and three-qubit systems. Training leverages Gymnasium-based quantum environments with simulated Lindblad evolution and fidelity-based rewards, achieving sample-efficient learning through replay buffers and stochastic exploration enforced by entropy maximization.

Closed-System Baseline Performance

To establish a fidelity baseline without decoherence, task-specific and multi-task SAC models were benchmarked across 51 Hamiltonian variations under closed-system dynamics. The results demonstrate parity between multi-task and task-specific performance, with fidelity discrepancies below $0.005$ in most cases and the multi-task agent obtaining near-ideal fidelities up to $0.999$ for select systems. This validates the unified policy's capacity for high-fidelity state preparation under unitary evolution.

Figure 1: Distribution of maximum state fidelities for closed-system dynamics showcases exceptional and uniform performance across diverse system sizes and Hamiltonians.

Open-System Control and Hamiltonian Generalization

Transitioning to open-system dynamics, the SAC agent displays robust performance across multiple quantum control problems. Box plots reveal median fidelities above 0.95 for virtually all single- and two-qubit configurations, with three-qubit systems achieving similar performance except for the more complex Dicke state transfer instance.

Figure 2: Fidelity distribution across open quantum systems highlights robust state transfer even with environmental noise and decoherence.

To examine policy generalization and transfer to unseen Hamiltonians, progressive expansion experiments were conducted. As the training pool grew from 5 to 51 Hamiltonians, inference success on both the full and held-out sets improved significantly—culminating in a 95.2% success rate for the unified model. Notably, single-qubit held-out systems are reliably controlled with fidelities exceeding 0.95, confirming that the SAC agent discovers transferable quantum control principles rather than memorizing task particulars.

Figure 3: Fidelity distribution on held-out Hamiltonians for model trained with 10 Hamiltonians demonstrates zero-shot generalization capacity.

Robustness Analysis: SAC Versus GRAPE

Robustness is quantitatively assessed via the Robustness Infidelity Measure (RIM), which computes mean infidelity under stochastic pulse amplitude and decoherence rate perturbations. SAC-trained pulse sequences maintain RIM scores as low as 0.045 under combined uncertainties, outperforming GRAPE controls where mean infidelity rises to 0.061 under identical conditions. The stochastic nature and maximum-entropy exploration in SAC confer increased resilience, while training under explicit open-system dynamics mitigates noise sensitivity.

Figure 4: RIM analysis reveals SAC's superior robustness to pulse pertubations and decoherence rate variations compared to GRAPE.

Adaptive Temporal Optimization

Unlike prior works that fix $T$ and $N$, the SAC agent concurrently optimizes these parameters along with control amplitudes. Adaptive selection of temporal parameters improves fidelity by minimizing unnecessary evolution, minimizing decoherence accumulation, and increasing computational efficiency. The learned distributions for $T$ and $N$ demonstrate convergence toward short control durations and moderate discretization.

Figure 5: Distribution of learned effective evolution time ($T$) and segment count ($N$) exhibits convergence to minimal, efficient control protocols.

Closed-System Versus Open-System Training

Applying closed-system-trained policies to open-system dynamics results in marked performance degradation, especially as decoherence rates rise. Retraining under Lindblad dynamics restores fidelity and success rates, with the SAC agent maintaining high performance even as environmental noise exceeds the training distribution. This outcome underscores the necessity of explicitly accounting for dissipation and environmental interactions during policy training to ensure robustness in practical implementations.

Implications, Limitations, and Future Directions

The integration of multi-task RL with adaptive temporal optimization presents a scalable pathway for robust control of quantum systems, offering practical advantages for NISQ devices and noisy intermediate-scale quantum hardware. The results demonstrate that a single SAC agent can accomplish high-fidelity, resilient control across a broad spectrum of Hamiltonian variations, adaptively tuning control time and discretization to mitigate decoherence and maximize computational efficiency.

This methodology provides foundational steps toward universal quantum control policies suitable for heterogeneous device platforms and gate sets. However, scalability is presently constrained by exponential Hilbert space growth and fixed initial-target state pairs. The framework does not yet address arbitrary state transfer nor guarantee reachability in higher-dimensional open systems.

Future research should focus on extending the architecture to variable initial-target pairs, exploring pulse parametrizations with reduced action space dimensionality (e.g., Fourier or sinc bases), and performing experimental validation on physical hardware. Rigorous controllability analysis and hardware-in-the-loop adaptation will be critical for establishing practical applicability in quantum technology domains.

Conclusion

This paper establishes a multi-task SAC RL framework for robust open-system quantum control, demonstrating high-fidelity state transfer, efficient generalization, and superior robustness to uncertainty versus classical control approaches. The simultaneous optimization of evolution time and pulse segment count reduces both decoherence impact and computational overhead. The results inform scalable strategies for universal quantum control in noisy environments, advancing automated control for quantum information processing and allied fields.