Papers
Topics
Authors
Recent
Search
2000 character limit reached

Human-in-the-loop Learning for Dynamic Congestion Games

Published 24 Apr 2024 in cs.GT and cs.AI | (2404.15599v2)

Abstract: Today mobile users learn and share their traffic observations via crowdsourcing platforms (e.g., Waze). Yet such platforms simply cater to selfish users' myopic interests to recommend the shortest path, and do not encourage enough users to travel and learn other paths for future others. Prior studies focus on one-shot congestion games without considering users' information learning, while our work studies how users learn and alter traffic conditions on stochastic paths in a human-in-the-loop manner. Our analysis shows that the myopic routing policy leads to severe under-exploration of stochastic paths. This results in a price of anarchy (PoA) greater than $2$, as compared to the socially optimal policy in minimizing the long-term social cost. Besides, the myopic policy fails to ensure the correct learning convergence about users' traffic hazard beliefs. To address this, we focus on informational (non-monetary) mechanisms as they are easier to implement than pricing. We first show that existing information-hiding mechanisms and deterministic path-recommendation mechanisms in Bayesian persuasion literature do not work with even (\text{PoA}=\infty). Accordingly, we propose a new combined hiding and probabilistic recommendation (CHAR) mechanism to hide all information from a selected user group and provide state-dependent probabilistic recommendations to the other user group. Our CHAR successfully ensures PoA less than (\frac{5}{4}), which cannot be further reduced by any other informational (non-monetary) mechanism. Besides the parallel network, we further extend our analysis and CHAR to more general linear path graphs with multiple intermediate nodes, and we prove that the PoA results remain unchanged. Additionally, we carry out experiments with real-world datasets to further extend our routing graphs and verify the close-to-optimal performance of our CHAR.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (46)
  1. H. Li and L. Duan, “Distributed Learning for Dynamic Congestion Games,” in Proc. IEEE International Symposium on Information Theory (ISIT), 2024.
  2. H. Zhang, H. Ge, J. Yang, and Y. Tong, “Review of vehicle routing problems: Models, classification and solving algorithms,” Archives of Computational Methods in Engineering, vol. 29, no. 1, pp. 195–221, 2022.
  3. T. Haselton, “How to use Google Waze,” https://www.cnbc.com/2018/11/13/how-to-use-google-waze-for-directions-and-avoiding-traffic.html, 2018.
  4. FHWA, “Crowdsourcing for advancing operations,” https://www.fhwa.dot.gov/innovation/everydaycounts/edc_6/crowdsourcing.cfm, 2024.
  5. Y. Zhu and K. Savla, “Information design in non-atomic routing games with partial participation: Computation and properties,” IEEE Transactions on Control of Network Systems, 2022.
  6. S. Vasserman, M. Feldman, and A. Hassidim, “Implementing the wisdom of waze,” in Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015.
  7. H. Liu, K. Liu, and Q. Zhao, “Learning in a changing world: Restless multiarmed bandit with unknown dynamics,” IEEE Transactions on Information Theory, vol. 59, no. 3, pp. 1902–1916, 2012.
  8. A. Slivkins et al., “Introduction to multi-armed bandits,” Foundations and Trends® in Machine Learning, vol. 12, no. 1-2, pp. 1–286, 2019.
  9. S. Gupta, S. Chaudhari, G. Joshi, and O. Yağan, “Multi-armed bandits with correlated arms,” IEEE Transactions on Information Theory, vol. 67, no. 10, pp. 6711–6732, 2021.
  10. A. Bozorgchenani, S. Maghsudi, D. Tarchi, and E. Hossain, “Computation offloading in heterogeneous vehicular edge networks: On-line and off-policy bandit solutions,” IEEE Transactions on Mobile Computing, vol. 21, no. 12, pp. 4233-4248, 2021.
  11. H. Wang, Y. Yang, E. Wang, W. Liu, Y. Xu, and J. Wu, “Truthful user recruitment for cooperative crowdsensing task: A combinatorial multi-armed bandit approach,” IEEE Transactions on Mobile Computing, vol. 22, no. 7, pp. 4314-4331, 2022.
  12. F. Li, D. Yu, H. Yang, J. Yu, K. Holger, and X. Cheng, “Multi-armed-bandit-based spectrum scheduling algorithms in wireless networks: A survey,” IEEE Wireless Communications, vol. 27, no. 1, pp. 24-30, 2020.
  13. C. Shi and C. Shen, “Federated multi-armed bandits,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 11, 2021, pp. 9603–9611.
  14. L. Yang, Y.-Z. J. Chen, S. Pasteris, M. Hajiesmaili, J. Lui, and D. Towsley, “Cooperative stochastic bandits with asynchronous agents and constrained feedback,” Advances in Neural Information Processing Systems, vol. 34, pp. 8885–8897, 2021.
  15. L. Yang, Y.-Z. J. Chen, M. H. Hajiemaili, J. C. Lui, and D. Towsley, “Distributed bandits with heterogeneous agents,” in IEEE INFOCOM 2022-IEEE Conference on Computer Communications.   IEEE, 2022, pp. 200–209.
  16. J. Zhu and J. Liu, “Distributed multi-armed bandits,” IEEE Transactions on Automatic Control, 2023.
  17. C. Meng and A. Markopoulou, “On routing-optimal networks for multiple unicasts,” in 2014 IEEE International Symposium on Information Theory.   IEEE, 2014, pp. 111–115.
  18. S. Das, E. Kamenica, and R. Mirka, “Reducing congestion through information design,” in 2017 55th annual allerton conference on communication, control, and computing (allerton).   IEEE, 2017, pp. 1279–1284.
  19. E. Kamenica, “Bayesian persuasion and information design,” Annual Review of Economics, vol. 11, pp. 249–272, 2019.
  20. Y. Mansour, A. Slivkins, V. Syrgkanis, and Z. S. Wu, “Bayesian exploration: Incentivizing exploration in bayesian games,” Operations Research, vol. 70, no. 2, pp. 1105–1127, 2022.
  21. Y. Babichenko, I. Talgam-Cohen, H. Xu, and K. Zabarnyi, “Regret-minimizing bayesian persuasion,” Games and Economic Behavior, vol. 136, pp. 226–248, 2022.
  22. S. Gollapudi, K. Kollias, C. Maheshwari, and M. Wu, “Online learning for traffic navigation in congested networks,” in International Conference on Algorithmic Learning Theory.   PMLR, 2023, pp. 642–662.
  23. Y. Yang and G. I. Webb, “Discretization for naive-bayes learning: managing discretization bias and variance,” Machine learning, vol. 74, no. 1, pp. 39–74, 2009.
  24. F. Meunier and N. Wagner, “Equilibrium results for dynamic congestion games,” Transportation Science, vol. 44, no. 4, pp. 524–536, 2010.
  25. G. Carmona and K. Podczeck, “Pure strategy nash equilibria of large finite-player games and their relationship to non-atomic games,” Journal of Economic Theory, vol. 187, p. 105015, 2020.
  26. H. Li, and L. Duan, “Online pricing incentive to sample fresh information,” IEEE Transactions on Network Science and Engineering, vol. 10, no. 1, pp. 514–526, 2022.
  27. B. L. Ferguson, P. N. Brown, and J. R. Marden, “The effectiveness of subsidies and tolls in congestion games,” IEEE Transactions on Automatic Control, vol. 67, no. 6, pp. 2729–2742, 2022.
  28. F. Li, Y. Chai, H. Yang, P. Hu, and L. Duan, “The effectiveness of subsidies and tolls in congestion games,” IEEE/ACM Transactions on Networking, 2024.
  29. H. Tavafoghi and D. Teneketzis, “Informational incentives for congestion games,” in 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).   IEEE, 2017, pp. 1285–1292.
  30. J. Wang and M. Hu, “Efficient inaccuracy: User-generated information sharing in a queue,” Management Science, vol. 66, no. 10, pp. 4648–4666, 2020.
  31. F. Farhadi and D. Teneketzis, “Dynamic information design: a simple problem on optimal sequential information disclosure,” Dynamic Games and Applications, vol. 12, no. 2, pp. 443–484, 2022.
  32. Y. Li, C. Courcoubetis, and L. Duan, “Recommending paths: Follow or not follow?” in IEEE INFOCOM 2019-IEEE Conference on Computer Communications.   IEEE, 2019, pp. 928–936.
  33. M. Wu and S. Amin, “Learning an unknown network state in routing games,” IFAC-PapersOnLine, vol. 52, no. 20, pp. 345–350, 2019.
  34. H. Li and L. Duan, “When congestion games meet mobile crowdsourcing: Selective information disclosure,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 5, 2023, pp. 5739–5746.
  35. I. Kremer, Y. Mansour, and M. Perry, “Implementing the “wisdom of the crowd”,” Journal of Political Economy, vol. 122, no. 5, pp. 988–1012, 2014.
  36. X. Ban, R. Herring, P. Hao, and A. M. Bayen, “Delay pattern estimation for signalized intersections using sampled travel times,” Transportation Research Record, vol. 2130, no. 1, pp. 109–119, 2009.
  37. I. Alam, D. M. Farid, and R. J. Rossetti, “The prediction of traffic flow with regression analysis,” in Emerging Technologies in Data Mining and Information Security.   Springer, 2019, pp. 661–671.
  38. M. Venanzi, A. Rogers, and N. R. Jennings, “Crowdsourcing spatial phenomena using trust-based heteroskedastic gaussian processes,” in First AAAI Conference on Human Computation and Crowdsourcing, 2013.
  39. A. L. Smith and S. S. Villar, “Bayesian adaptive bandit-based designs using the gittins index for multi-armed trials with normally distributed endpoints,” Journal of Applied Statistics, vol. 45, no. 6, pp. 1052–1076, 2018.
  40. D. Bertsimas and J. Tsitsiklis, “Introduction to linear optimization,”    Athena Scientific Belmont, MA, 1997
  41. S. Ross, “Introduction to stochastic dynamic programming,” Academic press, 2014.
  42. F. Yao, “Efficient dynamic programming using quadrangle inequalities,” Proceedings of the twelfth annual ACM symposium on Theory of computing, pp. 429–435, 1980.
  43. H. J. Broersma and C. Hoede, “Path graphs,” Journal of graph theory, vol. 13, no. 4, pp. 427–444, 1989.
  44. B. BaiduMap, “Baidu maps open platform,” https://lbsyun.baidu.com/faq/api?title=webapi/traffic-roadseek, 2023.
  45. S. R. Eddy, “Profile hidden markov models.” Bioinformatics (Oxford, England), vol. 14, no. 9, pp. 755–763, 1998.
  46. Z. Chen, J. Wen, and Y. Geng, “Predicting future traffic using hidden markov models,” in 2016 IEEE 24th international conference on network protocols (ICNP).   IEEE, 2016, pp. 1–6.
Citations (2)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Summarize

Paper to Video (Beta)

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now

Authors (2)

  1. Hongbo Li 
  2. Lingjie Duan 

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up

Tweets

Sign up for free to view the 1 tweet with 0 likes about this paper.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up for Free