Papers
Topics
Authors
Recent
Search
2000 character limit reached

Applications of model-aware reinforcement learning in Bayesian quantum metrology

Published 8 Mar 2024 in quant-ph | (2403.05706v1)

Abstract: An important practical problem in the field of quantum metrology and sensors is to find the optimal sequences of controls for the quantum probe that realize optimal adaptive estimation. In Belliardo et al., arXiv:2312.16985 (2023), we solved this problem in general, by introducing a procedure capable of optimizing a wide range of tasks in quantum metrology and estimation by combining model-aware reinforcement learning with Bayesian inference. We take a model-based approach to the optimisation where the physics describing the system is explicitly taken into account in the training through automatic differentiation. In this follow-up paper we present some applications of the framework. The first family of examples concerns the estimation of magnetic fields, hyperfine interactions, and decoherence times for electronic spins in diamond. For these examples, we perform multiple Ramsey measurements on the spin. The second family of applications concerns the estimation of phases and coherent states on photonic circuits, without squeezing elements, where the bosonic lines are measured by photon counters. This exposition showcases the broad applicability of the method, which has been implemented in the qsensoropt library released on PyPI, which can be installed with pip.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (99)
  1. Flamini, F. et al. Photonic architecture for reinforcement learning. New Journal of Physics 22, 045002 (2020). URL https://iopscience.iop.org/article/10.1088/1367-2630/ab783c.
  2. Broughton, M. et al. TensorFlow Quantum: A Software Framework for Quantum Machine Learning (2021). URL http://arxiv.org/abs/2003.02989.
  3. Bergholm, V. et al. PennyLane: Automatic differentiation of hybrid quantum-classical computations (2022). URL http://arxiv.org/abs/1811.04968.
  4. Bukov, M. et al. Reinforcement Learning in Different Phases of Quantum Control. Physical Review X 8, 031086 (2018). URL https://link.aps.org/doi/10.1103/PhysRevX.8.031086.
  5. When does reinforcement learning stand out in quantum control? A comparative study on state preparation. npj Quantum Information 5, 85 (2019). URL http://www.nature.com/articles/s41534-019-0201-8.
  6. Universal quantum control through deep reinforcement learning. npj Quantum Information 5, 33 (2019). URL http://www.nature.com/articles/s41534-019-0141-3.
  7. Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements. Quantum 6, 747 (2022). URL https://quantum-journal.org/papers/q-2022-06-28-747/.
  8. Gradient-Ascent Pulse Engineering with Feedback. PRX Quantum 4, 030305 (2023). URL https://link.aps.org/doi/10.1103/PRXQuantum.4.030305.
  9. Reinforcement Learning with Neural Networks for Quantum Feedback. Physical Review X 8, 031084 (2018). URL https://link.aps.org/doi/10.1103/PhysRevX.8.031084.
  10. Cimini, V. et al. Calibration of Quantum Sensors by Neural Networks. Physical Review Letters 123, 230502 (2019). URL https://link.aps.org/doi/10.1103/PhysRevLett.123.230502.
  11. Neural-network-based parameter estimation for quantum detection. Quantum Science and Technology 6, 045012 (2021). URL https://iopscience.iop.org/article/10.1088/2058-9565/ac16ed.
  12. A machine learning approach to Bayesian parameter estimation. npj Quantum Information 7, 169 (2021). URL https://www.nature.com/articles/s41534-021-00497-w.
  13. Frequentist parameter estimation with supervised learning. AVS Quantum Science 3, 034401 (2021). URL https://avs.scitation.org/doi/10.1116/5.0058163.
  14. Nguyen, V. et al. Deep reinforcement learning for efficient measurement of quantum devices. npj Quantum Information 7, 100 (2021). URL http://www.nature.com/articles/s41534-021-00434-x.
  15. Palmieri, A. M. et al. Experimental neural network enhanced quantum tomography. npj Quantum Information 6, 20 (2020). URL http://www.nature.com/articles/s41534-020-0248-6.
  16. Adaptive quantum state tomography with neural networks. npj Quantum Information 7, 105 (2021). URL http://www.nature.com/articles/s41534-021-00436-9.
  17. Hsieh, H.-Y. et al. Direct Parameter Estimations from Machine Learning-Enhanced Quantum State Tomography. Symmetry 14, 874 (2022). URL https://www.mdpi.com/2073-8994/14/5/874.
  18. Marquardt, F. Machine learning and quantum devices. SciPost Physics Lecture Notes 29 (2021). URL https://scipost.org/10.21468/SciPostPhysLectNotes.29.
  19. Marquardt, F. Online Course: Advanced Machine Learning for Physics, Science, and Artificial Scientific Discovery (2021).
  20. Artificial intelligence and machine learning for quantum technologies. Physical Review A 107, 010101 (2023). URL https://link.aps.org/doi/10.1103/PhysRevA.107.010101.
  21. Fisher, R. A. The design of experiments. The design of experiments (Oliver & Boyd, Oxford, England, 1935).
  22. Foster, A. E. Variational, Monte Carlo and policy-based approaches to Bayesian experimental design. http://purl.org/dc/dcmitype/Text, University of Oxford (2021). URL https://ora.ox.ac.uk/objects/uuid:4a3e13ca-e6c6-4669-955e-f1a87e201228.
  23. Baydin, A. G. et al. Toward Machine Learning Optimization of Experimental Design. Nuclear Physics News 31, 25–28 (2021). URL https://www.tandfonline.com/doi/full/10.1080/10619127.2021.1881364.
  24. Machine learning and computation-enabled intelligent sensor design. Nature Machine Intelligence 3, 556–565 (2021). URL http://www.nature.com/articles/s42256-021-00360-9.
  25. Implicit Deep Adaptive Design: Policy-Based Experimental Design without Likelihoods. In Advances in Neural Information Processing Systems, vol. 34, 25785–25798 (Curran Associates, Inc., 2021). URL https://proceedings.neurips.cc/paper/2021/hash/d811406316b669ad3d370d78b51b1d2e-Abstract.html.
  26. Deep Adaptive Design: Amortizing Sequential Bayesian Experimental Design. In Proceedings of the 38th International Conference on Machine Learning, 3384–3395 (PMLR, 2021). URL https://proceedings.mlr.press/v139/foster21a.html.
  27. Neural-Network Heuristics for Adaptive Bayesian Quantum Estimation. PRX Quantum 2, 020303 (2021). URL https://link.aps.org/doi/10.1103/PRXQuantum.2.020303.
  28. Differentiable Particle Filtering without Modifying the Forward Pass (2021). URL http://arxiv.org/abs/2106.10314.
  29. qsensoropt: quantum sensor optimisation (2022). URL https://gitlab.com/federico.belliardo/qsensoropt.
  30. qsensoropt documentation (2022). URL https://qsensoropt-federico-belliardo-aafff0229087adae5a915fec60fdc5d37.gitlab.io/.
  31. An agnostic-Dolinar receiver for coherent states classification. Physical Review A 104, 042606 (2021). URL http://arxiv.org/abs/2106.11909.
  32. A variational toolbox for quantum multi-parameter estimation. npj Quantum Information 7, 1–5 (2021). URL https://www.nature.com/articles/s41534-021-00425-y.
  33. Zhang, M. et al. QuanEstimation: An open-source toolkit for quantum parameter estimation. Physical Review Research 4, 043057 (2022). URL https://link.aps.org/doi/10.1103/PhysRevResearch.4.043057.
  34. Granade, C. et al. QInfer: Statistical inference software for quantum applications. Quantum 1, 5 (2017). URL http://arxiv.org/abs/1610.00336.
  35. Optbayesexpt: Sequential Bayesian Experiment Design for Adaptive Measurements. Journal of Research of the National Institute of Standards and Technology 126, 126002 (2021). URL https://nvlpubs.nist.gov/nistpubs/jres/126/jres.126.002.pdf.
  36. Designing optimal protocols in Bayesian quantum parameter estimation with higher-order operations (2023). URL http://arxiv.org/abs/2311.01513.
  37. Quantum parameter estimation with optimal control. Physical Review A 96, 012117 (2017). URL http://link.aps.org/doi/10.1103/PhysRevA.96.012117.
  38. Xu, H. et al. Generalizable control for quantum parameter estimation through reinforcement learning. npj Quantum Information 5, 1–8 (2019). URL https://www.nature.com/articles/s41534-019-0198-z.
  39. Rembold, P. et al. Introduction to quantum optimal control for quantum sensing with nitrogen-vacancy centers in diamond. AVS Quantum Science 2, 024701 (2020). URL http://avs.scitation.org/doi/10.1116/5.0006785.
  40. Improving the dynamics of quantum sensors with reinforcement learning. New Journal of Physics 22, 035001 (2020). URL https://iopscience.iop.org/article/10.1088/1367-2630/ab6f1f.
  41. Generalizable control for multiparameter quantum metrology. Physical Review A 103, 042615 (2021). URL http://arxiv.org/abs/2012.13377.
  42. Optimal Scheme for Quantum Metrology. Advanced Quantum Technologies 5, 2100080 (2022). URL http://arxiv.org/abs/2111.12279.
  43. Parameter estimation in quantum sensing based on deep reinforcement learning. npj Quantum Information 8, 1–12 (2022). URL https://www.nature.com/articles/s41534-021-00513-z.
  44. Efficient and robust entanglement generation with deep reinforcement learning for quantum metrology. New Journal of Physics 24, 083011 (2022). URL https://dx.doi.org/10.1088/1367-2630/ac8285.
  45. Framework for Learning and Control in the Classical and Quantum Domains (2023). URL http://arxiv.org/abs/2307.04256.
  46. Gebhart, V. et al. Learning quantum systems. Nature Reviews Physics 5, 141–156 (2023). URL https://www.nature.com/articles/s42254-022-00552-1.
  47. Ma, Z. et al. Adaptive Circuit Learning for Quantum Metrology. In 2021 IEEE International Conference on Quantum Computing and Engineering (QCE), 419–430 (2021). URL http://arxiv.org/abs/2010.08702.
  48. Quantum Variational Optimization of Ramsey Interferometry and Atomic Clocks. Physical Review X 11, 041045 (2021). URL https://link.aps.org/doi/10.1103/PhysRevX.11.041045.
  49. Marciniak, C. D. et al. Optimal metrology with programmable quantum sensors. Nature 603, 604–609 (2022). URL https://www.nature.com/articles/s41586-022-04435-4.
  50. Optimal and Variational Multi-Parameter Quantum Metrology and Vector Field Sensing (2023). URL http://arxiv.org/abs/2302.07785.
  51. Superresolution imaging with multiparameter quantum metrology in passive remote sensing. Physical Review A 107, 032607 (2023). URL https://link.aps.org/doi/10.1103/PhysRevA.107.032607.
  52. Heras, A. M. d. l. et al. Photonic quantum metrology with variational quantum optical non-linearities (2023). URL http://arxiv.org/abs/2309.09841.
  53. Variational principle for optimal quantum controls in quantum metrology. Physical Review Letters 128, 160505 (2022). URL https://link.aps.org/doi/10.1103/PhysRevLett.128.160505.
  54. Model-aware reinforcement learning for high-performance bayesian experimental design in quantum metrology. URL http://arxiv.org/abs/2312.16985. eprint 2312.16985 [quant-ph].
  55. Application of machine learning to experimental design in quantum mechanics. International Journal of Quantum Information (2024). URL https://www.worldscientific.com/doi/abs/10.1142/S0219749924500023.
  56. Belliardo, F. et al. Optimizing quantum-enhanced Bayesian multiparameter estimation in noisy apparata (2022). URL http://arxiv.org/abs/2211.04747. ArXiv:2211.04747 [quant-ph].
  57. Adam: A Method for Stochastic Optimization. In Bengio, Y. & LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings (2015). URL http://arxiv.org/abs/1412.6980.
  58. Achieving Heisenberg scaling with maximally entangled states: An analytic upper bound for the attainable root-mean-square error. Physical Review A 102, 042613 (2020). URL https://link.aps.org/doi/10.1103/PhysRevA.102.042613.
  59. Quantum Computation and Quantum Information (Cambridge University Press, 2000).
  60. Holevo, A. S. Quantum Systems, Channels, Information: A Mathematical Introduction (De Gruyter, Berlin, Boston, 2019). URL https://doi.org/10.1515/9783110642490.
  61. Quantum science and technology based on color centers with accessible spin. Journal of Applied Physics 131, 010401 (2022). URL https://aip.scitation.org/doi/10.1063/5.0082219.
  62. Chen, M. et al. Quantum metrology with single spins in diamond under ambient conditions. National Science Review 5, 346–355 (2018). URL https://academic.oup.com/nsr/article/5/3/346/4430770.
  63. Maze, J. Quantum manipulation of nitrogen-vacancy centers in diamond: From basic properties to applications. Ph.D. thesis (2010).
  64. Barry, J. F. et al. Sensitivity optimization for NV-diamond magnetometry. Rev. Mod. Phys. 92, 68 (2020).
  65. Arshad, M. J. et al. Real-time adaptive estimation of decoherence timescales for a single qubit. Phys. Rev. Appl. 21, 024026 (2024).
  66. Joas, T. et al. Online adaptive quantum characterization of a nuclear spin. npj Quantum Information 7, 1–8 (2021). URL https://www.nature.com/articles/s41534-021-00389-z.
  67. Valeri, M. Experimental adaptive Bayesian estimation of multiple phases with limited data. npj Quantum Information 11 (2020).
  68. Paesani, S. et al. Experimental Bayesian Quantum Phase Estimation on a Silicon Photonic Chip. Physical Review Letters 118, 100503 (2017). URL https://link.aps.org/doi/10.1103/PhysRevLett.118.100503.
  69. Polino, E. et al. Experimental multiphase estimation on a chip. Optica 6, 288 (2019). URL https://www.osapublishing.org/abstract.cfm?URI=optica-6-3-288.
  70. Photonic quantum metrology. AVS Quantum Science 2, 024703 (2020). URL http://avs.scitation.org/doi/10.1116/5.0007577.
  71. Quantum Enhanced Multiple Phase Estimation. Physical Review Letters 111, 070403 (2013). URL https://link.aps.org/doi/10.1103/PhysRevLett.111.070403.
  72. Schmitt, S. et al. Optimal frequency measurements with quantum probes. npj Quantum Information 7, 55 (2021). URL http://www.nature.com/articles/s41534-021-00391-5.
  73. How to best sample a periodic probability distribution, or on the accuracy of Hamiltonian finding strategies. Quantum Information Processing 12, 611–623 (2013). URL http://link.springer.com/10.1007/s11128-012-0407-6.
  74. Sequential Bayesian Experiment Design for Optically Detected Magnetic Resonance of Nitrogen-Vacancy Centers. Physical Review Applied 14, 054036 (2020). URL https://link.aps.org/doi/10.1103/PhysRevApplied.14.054036.
  75. Sequential Bayesian experiment design for adaptive Ramsey sequence measurements. Journal of Applied Physics 130, 144401 (2021). URL https://aip.scitation.org/doi/10.1063/5.0055630.
  76. Robust online Hamiltonian learning. New Journal of Physics 14, 103013 (2012). URL https://dx.doi.org/10.1088/1367-2630/14/10/103013.
  77. Oshnik, N. et al. Robust magnetometry with single nitrogen-vacancy centers via two-step optimization. Physical Review A 106, 013107 (2022). URL https://link.aps.org/doi/10.1103/PhysRevA.106.013107.
  78. Resource-efficient adaptive Bayesian tracking of magnetic fields with a quantum sensor. Journal of Physics: Condensed Matter 33, 195801 (2021). URL https://iopscience.iop.org/article/10.1088/1361-648X/abe34f.
  79. Bonato, C. et al. Optimized quantum sensing with a single electron spin using real-time adaptive measurements. Nature Nanotechnology 11, 247–252 (2016). URL https://www.nature.com/articles/nnano.2015.261.
  80. Santagati, R. et al. Magnetic-Field Learning Using a Single Electronic Spin in Diamond with One-Photon Readout at Room Temperature. Physical Review X 9, 021019 (2019). URL https://link.aps.org/doi/10.1103/PhysRevX.9.021019.
  81. Zohar, I. et al. Real-time frequency estimation of a qubit without single-shot-readout. Quantum Science and Technology 8, 035017 (2023). URL https://iopscience.iop.org/article/10.1088/2058-9565/acd415.
  82. High-dynamic-range magnetometry with a single electronic spin in diamond. Nature Nanotechnology 7, 109–113 (2012). URL http://www.nature.com/articles/nnano.2011.225.
  83. Wang, J. et al. Experimental quantum Hamiltonian learning. Nature Physics 13, 551–555 (2017). URL http://www.nature.com/articles/nphys4074.
  84. Bayesian estimation for quantum sensing in the absence of single-shot detection. Physical Review B 99, 125413 (2019). URL https://link.aps.org/doi/10.1103/PhysRevB.99.125413.
  85. Adaptive tracking of a time-varying field with a quantum sensor. Physical Review A 95, 052348 (2017). URL http://link.aps.org/doi/10.1103/PhysRevA.95.052348.
  86. Adaptive Hamiltonian estimation using Bayesian experimental design. AIP Conference Proceedings 1443, 165–173 (2012). URL https://doi.org/10.1063/1.3703632.
  87. Repetitive readout enhanced by machine learning. Machine Learning: Science and Technology 1, 015003 (2020). URL https://iopscience.iop.org/article/10.1088/2632-2153/ab4e24.
  88. Tsukamoto, M. et al. Machine-learning-enhanced quantum sensors for accurate magnetic field imaging. Scientific Reports 12, 13942 (2022). URL http://arxiv.org/abs/2202.00380.
  89. Hamiltonian Learning and Certification Using Quantum Resources. Physical Review Letters 112, 190501 (2014). URL https://link.aps.org/doi/10.1103/PhysRevLett.112.190501.
  90. Cimini, V. et al. Experimental metrology beyond the standard quantum limit for a wide resources range. npj Quantum Information 9, 1–9 (2023). URL https://www.nature.com/articles/s41534-023-00691-y.
  91. Helstrom, C. W. Quantum detection and estimation theory. Journal of Statistical Physics 1, 231–252 (1969). URL https://doi.org/10.1007/BF01007479.
  92. Holevo, A. S. Statistical problems in quantum physics. In Maruyama, G. & Prokhorov, Y. V. (eds.) Proceedings of the Second Japan-USSR Symposium on Probability Theory, Lecture Notes in Mathematics, 104–119 (Springer, Berlin, Heidelberg, 1973).
  93. Incompatibility in quantum parameter estimation. New Journal of Physics 23, 063055 (2021). URL https://doi.org/10.1088/1367-2630/ac04ca. Publisher: IOP Publishing.
  94. Cimini, V. et al. Deep reinforcement learning for quantum multiparameter estimation. Advanced Photonics 5, 016005 (2023). URL https://iris.uniroma1.it/retrieve/e0ceb934-f61a-49c3-bb0a-7e1047b6f239/Cimini_Deep-reinforcement_2023.pdf.
  95. Brask, J. B. Gaussian states and operations – a quick reference (2022). URL http://arxiv.org/abs/2102.05748.
  96. A ‘Pretty Good’ Measurement for Distinguishing Quantum States. Journal of Modern Optics 41, 2385–2390 (1994). URL https://doi.org/10.1080/09500349414552221.
  97. Designing Optimal Quantum Detectors Via Semidefinite Programming. IEEE Transactions on Information Theory 49, 1007–1012 (2003). URL http://arxiv.org/abs/quant-ph/0205178.
  98. On quantum detection and the square-root measurement. IEEE Transactions on Information Theory 47, 858–872 (2006). URL https://doi.org/10.1109/18.915636.
  99. Izumi, S. et al. Displacement receiver for phase-shift-keyed coherent states. Physical Review A 86, 042328 (2012). URL https://link.aps.org/doi/10.1103/PhysRevA.86.042328.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up