Goal-oriented Estimation of Multiple Markov Sources in Resource-constrained Systems (2311.07346v3)
Abstract: This paper investigates goal-oriented communication for remote estimation of multiple Markov sources in resource-constrained networks. An agent decides the updating times of the sources and transmits the packet to a remote destination over an unreliable channel with delay. The destination is tasked with source reconstruction for actuation. We utilize the metric \textit{cost of actuation error} (CAE) to capture the state-dependent actuation costs. We aim for a sampling policy that minimizes the long-term average CAE subject to an average resource constraint. We formulate this problem as an average-cost constrained Markov Decision Process (CMDP) and relax it into an unconstrained problem by utilizing \textit{Lyapunov drift} techniques. Then, we propose a low-complexity \textit{drift-plus-penalty} (DPP) policy for systems with known source/channel statistics and a Lyapunov optimization-based deep reinforcement learning (LO-DRL) policy for unknown environments. Our policies significantly reduce the number of uninformative transmissions by exploiting the timing of the important information.
- G. Walsh and H. Ye, “Scheduling of networked control systems,” IEEE Control Systems Magazine, vol. 21, no. 1, pp. 57–65, 2001.
- P. Park, S. Coleri Ergen, C. Fischione, C. Lu, and K. H. Johansson, “Wireless network design for control systems: A survey,” IEEE Communications Surveys & Tutorials, vol. 20, no. 2, 2018.
- N. Pappas and M. Kountouris, “Goal-oriented communication for real-time tracking in autonomous systems,” in IEEE ICAS, 2021.
- J. Chakravorty and A. Mahajan, “Fundamental limits of remote estimation of autoregressive markov processes under communication constraints,” IEEE Transactions on Automatic Control, vol. 62, no. 3, 2017.
- G. Cocco, A. Munari, and G. Liva, “Remote monitoring of two-state markov sources via random access channels: an information freshness vs. state estimation entropy perspective,” IEEE Journal on Selected Areas in Information Theory, 2023.
- M. Pezzutto, L. Schenato, and S. Dey, “Transmission power allocation for remote estimation with multi-packet reception capabilities,” Automatica, vol. 140, 2022.
- M. Kountouris and N. Pappas, “Semantics-empowered communication for networked intelligent systems,” IEEE Communications Magazine, vol. 59, no. 6, 2021.
- P. Popovski et al., “A perspective on time toward wireless 6G,” Proceedings of the IEEE, vol. 110, no. 8, 2022.
- Y. Sun, Y. Polyanskiy, and E. Uysal, “Sampling of the wiener process for remote estimation over a channel with random delay,” IEEE Transactions on Information Theory, vol. 66, no. 2, 2019.
- V. Tripathi et al., “Wiswarm: Age-of-information-based wireless networking for collaborative teams of uavs,” in IEEE INFOCOM, 2023.
- P. Kutsevol, O. Ayan, N. Pappas, and W. Kellerer, “Experimental study of transport layer protocols for wireless networked control systems,” in IEEE SECON, 2023.
- G. Stamatakis, N. Pappas, and A. Traganitis, “Control of status updates for energy harvesting devices that monitor processes with alarms,” in IEEE Globecom Workshops, 2019.
- A. Maatouk, M. Assaad, and A. Ephremides, “The age of incorrect information: An enabler of semantics-empowered communication,” IEEE Transactions on Wireless Communications, vol. 22, no. 4, 2023.
- X. Zheng, S. Zhou, and Z. Niu, “Urgency of information for context-aware timely status updates in remote control systems,” IEEE Transactions on Wireless Communications, vol. 19, no. 11, 2020.
- J. S, N. Pappas, and R. V. Bhat, “Distortion minimization with age of information and cost constraints,” in 21st WiOpt, 2023.
- A. Nikkhah, A. Ephremides, and N. Pappas, “Age of actuation in a wireless power transfer system,” in IEEE INFOCOM Workshops, 2023.
- E. Fountoulakis, N. Pappas, and M. Kountouris, “Goal-oriented policies for cost of actuation error minimization in wireless autonomous systems,” IEEE Communications Letters, vol. 27, no. 9, 2023.
- M. Salimnejad, M. Kountouris, and N. Pappas, “Real-time reconstruction of markov sources and remote actuation over wireless channels,” IEEE Transactions on Communications, 2024.
- ——, “State-aware real-time tracking and remote reconstruction of a markov source,” Journal of Communications and Networks, 2023.
- M. Althoff and A. Mergel, “Comparison of markov chain abstraction and monte carlo simulation for the safety assessment of autonomous cars,” IEEE Transactions on Intelligent Transportation Systems, 2011.
- N. Ye, Y. Zhang, and C. M. Borror, “Robustness of the markov-chain model for cyber-attack detection,” IEEE transactions on reliability, 2004.
- D.-j. Ma, A. M. Makowski, and A. Shwartz, “Estimation and optimal control for constrained markov chains,” in IEEE CDC, 1986.
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv:1707.06347, 2017.