Smart Routing with Precise Link Estimation: DSEE-Based Anypath Routing for Reliable Wireless Networking (2405.10377v1)
Abstract: In dynamic and resource-constrained environments, such as multi-hop wireless mesh networks, traditional routing protocols often falter by relying on predetermined paths that prove ineffective in unpredictable link conditions. Shortest Anypath routing offers a solution by adapting routing decisions based on real-time link conditions. However, the effectiveness of such routing is fundamentally dependent on the quality and reliability of the available links, and predicting these variables with certainty is challenging. This paper introduces a novel approach that leverages the Deterministic Sequencing of Exploration and Exploitation (DSEE), a multi-armed bandit algorithm, to address the need for accurate and real-time estimation of link delivery probabilities. This approach augments the reliability and resilience of the Shortest Anypath routing in the face of fluctuating link conditions. By coupling DSEE with Anypath routing, this algorithm continuously learns and ensures accurate delivery probability estimation and selects the most suitable way to efficiently route packets while maintaining a provable near-logarithmic regret bound. We also theoretically prove that our proposed scheme offers better regret scaling with respect to the network size than the previously proposed Thompson Sampling-based Opportunistic Routing (TSOR).
- Opportunistic routing in multi-hop wireless networks. ACM SIGCOMM Computer Communication Review, 34(1):69–74, 2004.
- When does opportunistic routing make sense? In Third IEEE International Conference on Pervasive Computing and Communications Workshops, pages 350–356. IEEE, 2005.
- Opportunistic routing for wireless ad hoc and sensor networks: Present and future directions. IEEE Communications Magazine, 47(12):103–109, 2009.
- Multirate anypath routing in wireless mesh networks. In IEEE INFOCOM 2009, pages 37–45, 2009.
- Opportunistic routing metrics: A timely one-stop tutorial survey. J. Netw. Comput. Appl., 171:102802, 2020.
- Bandit algorithms. Cambridge University Press, 2020.
- Deterministic sequencing of exploration and exploitation for multi-armed bandit problems. IEEE Journal of Selected Topics in Signal Processing, 7:759–767, 2011.
- Toward packet routing with fully-distributed multi-agent deep reinforcement learning. In 2019 International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOPT), pages 1–8, 2019.
- Application of reinforcement learning to routing in distributed wireless networks: a review. Artificial Intelligence Review, 43:381 – 416, 2013.
- A bayesian multi-armed bandit algorithm for dynamic end-to-end routing in sdn-based networks with piecewise-stationary rewards. Algorithms, 16(5):233, Apr 2023.
- TSOR: Thompson sampling-based opportunistic routing. IEEE Transactions on Wireless Communications, 20(11):7272–7285, 2021.
- Adaptive opportunistic routing for wireless ad hoc networks. IEEE/ACM Transactions On Networking, 20(1):243–256, 2011.
- Adaptive shortest-path routing under unknown and stochastically varying link states. In 2012 10th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt), pages 232–237, 2012.
- Multi-agent reinforcement learning based opportunistic routing and channel assignment for mobile cognitive radio ad hoc network. Mobile Networks and Applications, 19:720–730, 2014.
- Routing in multi-radio, multi-hop wireless mesh networks. pages 114–128, 09 2004.
- Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6(1):4–22, 1985.
- Multi-armed bandit allocation indices. John Wiley & Sons, 2011.
- Reinforcement learning: An introduction. MIT press, 2018.
- Combinatorial network optimization with unknown variables: Multi-armed bandits with linear rewards and individual observations. IEEE/ACM Transactions on Networking, 20(5):1466–1478, 2012.
- Rajeev Agrawal. The continuum-armed bandit problem. SIAM Journal on Control and Optimization, 33(6):1926–1951, 1995.