Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
153 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Result Diversification in Search and Recommendation: A Survey (2212.14464v4)

Published 29 Dec 2022 in cs.IR

Abstract: Diversifying return results is an important research topic in retrieval systems in order to satisfy both the various interests of customers and the equal market exposure of providers. There has been growing attention on diversity-aware research during recent years, accompanied by a proliferation of literature on methods to promote diversity in search and recommendation. However, diversity-aware studies in retrieval systems lack a systematic organization and are rather fragmented. In this survey, we are the first to propose a unified taxonomy for classifying the metrics and approaches of diversification in both search and recommendation, which are two of the most extensively researched fields of retrieval systems. We begin the survey with a brief discussion of why diversity is important in retrieval systems, followed by a summary of the various diversity concerns in search and recommendation, highlighting their relationship and differences. For the survey's main body, we present a unified taxonomy of diversification metrics and approaches in retrieval systems, from both the search and recommendation perspectives. In the later part of the survey, we discuss the open research questions of diversity-aware research in search and recommendation in an effort to inspire future innovations and encourage the implementation of diversity in real-world systems.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (101)
  1. S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme, “BPR: bayesian personalized ranking from implicit feedback,” in Proc. Conf. on Uncertainty in Artificial Intelligence, 2009.
  2. B. M. Sarwar, G. Karypis, J. A. Konstan, and J. Riedl, “Item-based collaborative filtering recommendation algorithms,” in Proc. Int. Conf. World Wide Web, 2001.
  3. X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang, “Lightgcn: Simplifying and powering graph convolution network for recommendation,” in Proc. Conf. on Research and Development in Information Retrieval, 2020.
  4. S. Rosen, “The economics of superstars,” in he American Economic Review, 1981, pp. 845–858.
  5. Q. Wu, Y. Liu, C. Miao, Y. Zhao, L. Guan, and H. Tang, “Recent advances in diversified recommendation,” CoRR, vol. abs/1905.06589, 2019.
  6. J. Sun, W. Guo, D. Zhang, Y. Zhang, F. Regol, Y. Hu, H. Guo, R. Tang, H. Yuan, X. He, and M. Coates, “A framework for recommending accurate and diverse items using bayesian graph convolutional neural networks,” in KDD.   ACM, 2020, pp. 2030–2039.
  7. H. K. Azad and A. Deepak, “Query expansion techniques for information retrieval: A survey,” Inf. Process. Manag., vol. 56, no. 5, pp. 1698–1735, 2019.
  8. L. Azzopardi, “Cognitive biases in search: A review and reflection of cognitive biases in information retrieval,” in CHIIR ’21: ACM SIGIR Conference on Human Information Interaction and Retrieval, Canberra, ACT, Australia, March 14-19, 2021, F. Scholer, P. Thomas, D. Elsweiler, H. Joho, N. Kando, and C. Smith, Eds.   ACM, 2021, pp. 27–37.
  9. W. Huang, B. Liu, and H. Tang, “Privacy protection for recommendation system: A survey,” Journal of Physics: Conference Series, vol. 1325, no. 1, p. 012087, oct 2019.
  10. J. Chen, H. Dong, X. Wang, F. Feng, M. Wang, and X. He, “Bias and debias in recommender system: A survey and future directions,” CoRR, vol. abs/2010.03240, 2020.
  11. C. Faloutsos and D. W. Oard, “A survey of information retrieval and filtering methods,” USA, Tech. Rep., 1995.
  12. S. Zhang, L. Yao, A. Sun, and Y. Tay, “Deep learning based recommender system: A survey and new perspectives,” ACM Comput. Surv., vol. 52, no. 1, pp. 5:1–5:38, 2019.
  13. M. Kunaver and T. Pozrl, “Diversity in recommender systems - A survey,” Knowl. Based Syst., 2017.
  14. J. Chakraborty and V. Verma, “A survey of diversification techniques in recommendation systems,” in 2016 International Conference on Data Mining and Advanced Computing (SAPIENCE), 2016, pp. 35–40.
  15. J. G. Carbonell and J. Goldstein, “The use of mmr, diversity-based reranking for reordering documents and producing summaries,” in SIGIR.   ACM, 1998, pp. 335–336.
  16. C. L. A. Clarke, M. Kolla, G. V. Cormack, O. Vechtomova, A. Ashkan, S. Büttcher, and I. MacKinnon, “Novelty and diversity in information retrieval evaluation,” in SIGIR.   ACM, 2008, pp. 659–666.
  17. F. Radlinski, P. N. Bennett, B. Carterette, and T. Joachims, “Redundancy, diversity and interdependent document relevance,” SIGIR Forum, vol. 43, no. 2, pp. 46–52, 2009.
  18. M. Zhang and N. Hurley, “Avoiding monotony: improving the diversity of recommendation lists,” in RecSys.   ACM, 2008, pp. 123–130.
  19. C. Ziegler, S. M. McNee, J. A. Konstan, and G. Lausen, “Improving recommendation lists through topic diversification,” in Proceedings of the 14th international conference on World Wide Web, WWW.   ACM, 2005, pp. 22–32.
  20. L. Chen, G. Zhang, and E. Zhou, “Fast greedy MAP inference for determinantal point process to improve recommendation diversity,” in NeurIPS, 2018, pp. 5627–5638.
  21. Y. Huang, W. Wang, L. Zhang, and R. Xu, “Sliding spectrum decomposition for diversified recommendation,” in KDD.   ACM, 2021, pp. 3041–3049.
  22. W. Chen, P. Ren, F. Cai, F. Sun, and M. de Rijke, “Improving end-to-end sequential recommendations with intent-aware diversification,” in CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, 2020.
  23. P. Cheng, S. Wang, J. Ma, J. Sun, and H. Xiong, “Learning to recommend accurate and diverse items,” in WWW.   ACM, 2017, pp. 183–192.
  24. X. Li, W. Jiang, W. Chen, J. Wu, G. Wang, and K. Li, “Directional and explainable serendipity recommendation,” in Proceedings of The Web Conference 2020, 2020, pp. 122–132.
  25. L. Gan, D. Nurbakova, L. Laporte, and S. Calabretto, “Enhancing recommendation diversity using determinantal point processes on knowledge graphs,” in SIGIR.   ACM, 2020, pp. 2001–2004.
  26. L. Chen, G. Zhang, and H. Zhou, “Improving the diversity of top-n recommendation via determinantal point process,” CoRR, vol. abs/1709.05135, 2017.
  27. Y. Liang, T. Qian, Q. Li, and H. Yin, “Enhancing domain-level and user-level adaptivity in diversified recommendation,” in Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2021.
  28. J. Parapar and F. Radlinski, “Diverse user preference elicitation with multi-armed bandits,” in WSDM.   ACM, 2021, pp. 130–138.
  29. J. Wasilewski and N. Hurley, “Incorporating diversity in a learning to rank recommender system,” in FLAIRS Conference.   AAAI Press, 2016, pp. 572–578.
  30. S. Vargas, P. Castells, and D. Vallet, “Intent-oriented diversity in recommender systems,” in Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, 2011, pp. 1211–1212.
  31. D. Stamenkovic, A. Karatzoglou, I. Arapakis, X. Xin, and K. Katevas, “Choosing the best of both worlds: Diverse and novel recommendations through multi-objective reinforcement learning,” in WSDM.   ACM, 2022, pp. 957–965.
  32. W. Chen, P. Ren, F. Cai, F. Sun, and M. De Rijke, “Multi-interest diversification for end-to-end sequential recommendation,” ACM Transactions on Information Systems (TOIS), vol. 40, no. 1, pp. 1–30, 2021.
  33. J. Han and H. Yamana, “Geographic diversification of recommended pois in frequently visited areas,” ACM Transactions on Information Systems (TOIS), vol. 38, no. 1, pp. 1–39, 2019.
  34. W. Gong, X. Zhang, Y. Chen, Q. He, A. Beheshti, X. Xu, C. Yan, and L. Qi, “DAWAR: diversity-aware web apis recommendation for mashup creation based on correlation graph,” in SIGIR.   ACM, 2022, pp. 395–404.
  35. K. Tsukuda and M. Goto, “Dualdiv: diversifying items and explanation styles in explainable hybrid recommendation,” in RecSys.   ACM, 2019, pp. 398–402.
  36. J. R. Haritsa, “The KNDN problem: A quest for unity in diversity,” IEEE Data Eng. Bull., vol. 32, no. 4, pp. 15–22, 2009.
  37. J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. Riedl, “Evaluating collaborative filtering recommender systems,” ACM Trans. Inf. Syst., vol. 22, no. 1, pp. 5–53, 2004.
  38. M. Ge, C. Delgado-Battenfeld, and D. Jannach, “Beyond accuracy: evaluating recommender systems by coverage and serendipity,” in Proceedings of the 2010 ACM Conference on Recommender Systems, RecSys.   ACM, 2010, pp. 257–260.
  39. B. Paudel, T. Haas, and A. Bernstein, “Fewer flops at the top: Accuracy, diversity, and regularization in two-class collaborative filtering,” in Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys.   ACM, 2017, pp. 215–223.
  40. G. Balloccu, L. Boratto, G. Fenu, and M. Marras, “Post processing recommender systems with knowledge graphs for recency, popularity, and diversity of explanations,” in SIGIR.   ACM, 2022, pp. 646–656.
  41. G. Adomavicius and Y. Kwon, “Improving aggregate recommendation diversity using ranking-based techniques,” IEEE Trans. Knowl. Data Eng., vol. 24, no. 5, pp. 896–911, 2012.
  42. K. Raman, P. N. Bennett, and K. Collins-Thompson, “Understanding intrinsic diversity in web search: Improving whole-session relevance,” ACM Transactions on Information Systems (TOIS), vol. 32, no. 4, pp. 1–45, 2014.
  43. M. Wilhelm, A. Ramanathan, A. Bonomo, S. Jain, E. H. Chi, and J. Gillenwater, “Practical diversified recommendations on youtube with determinantal point processes,” in CIKM.   ACM, 2018, pp. 2165–2173.
  44. H. Kwon, J. Han, and K. Han, “ART (attractive recommendation tailor): How the diversity of product recommendations affects customer purchase preference in fashion industry?” in CIKM.   ACM, 2020, pp. 2573–2580.
  45. Y. Zheng, C. Gao, L. Chen, D. Jin, and Y. Li, “DGCN: diversified recommendation with graph convolutional networks,” in WWW.   ACM / IW3C2, 2021, pp. 401–412.
  46. J. Zhou, E. Agichtein, and S. Kallumadi, “Diversifying multi-aspect search results using simpson’s diversity index,” in CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, 2020.
  47. Y. He, H. Zou, H. Yu, Q. Wang, and S. Gao, “Diversity-aware recommendation by user interest domain coverage maximization,” in 2019 IEEE International Conference on Data Mining (ICDM).   IEEE, 2019, pp. 1084–1089.
  48. X. Yin, J. X. Huang, Z. Li, and X. Zhou, “A survival modeling approach to biomedical search result diversification using wikipedia,” IEEE Trans. Knowl. Data Eng., vol. 25, no. 6, pp. 1201–1212, 2013.
  49. R. Li and J. X. Yu, “Scalable diversified ranking on large graphs,” IEEE Trans. Knowl. Data Eng., vol. 25, no. 9, pp. 2133–2146, 2013.
  50. C. Zhai, W. W. Cohen, and J. D. Lafferty, “Beyond independent relevance: methods and evaluation metrics for subtopic retrieval,” in SIGIR.   ACM, 2003, pp. 10–17.
  51. X. Qin, Z. Dou, and J. Wen, “Diversifying search results using self-attention network,” in CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, 2020.
  52. S. Liang, F. Cai, Z. Ren, and M. de Rijke, “Efficient structured learning for personalized diversification,” IEEE Trans. Knowl. Data Eng., vol. 28, no. 11, pp. 2958–2973, 2016.
  53. S. Liang, E. Yilmaz, H. Shen, M. D. Rijke, and W. B. Croft, “Search result diversification in short text streams,” ACM Transactions on Information Systems (TOIS), vol. 36, no. 1, pp. 1–35, 2017.
  54. E. H. Simpson, “Measurement of diversity,” nature, vol. 163, no. 4148, pp. 688–688, 1949.
  55. A. Antikacioglu and R. Ravi, “Post processing recommender systems for diversity,” in Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2017, pp. 707–716.
  56. J. Sanz-Cruzado and P. Castells, “Enhancing structural diversity in social networks by recommending weak ties,” in Proceedings of the 12th ACM Conference on Recommender Systems, RecSys.   ACM, 2018, pp. 233–241.
  57. C. Gini, “Variabilit‘a e mutabilit‘a,” 1912.
  58. J. Parapar and F. Radlinski, “Towards unified metrics for accuracy and diversity for recommender systems,” in RecSys ’21: Fifteenth ACM Conference on Recommender Systems.   ACM, 2021, pp. 75–84.
  59. S. Li, Y. Zhou, D. Zhang, Y. Zhang, and X. Lan, “Learning to diversify recommendations based on matrix factorization,” in DASC/PiCom/DataCom/CyberSciTech.   IEEE Computer Society, 2017, pp. 68–74.
  60. R. L. T. Santos, C. Macdonald, and I. Ounis, “Exploiting query reformulations for web search result diversification,” in WWW.   ACM, 2010, pp. 881–890.
  61. H. Yu, “Optimize what you evaluate with: Search result diversification based on metric optimization,” in AAAI.   AAAI Press, 2022, pp. 10 399–10 407.
  62. Z. Jiang, Z. Dou, W. X. Zhao, J. Nie, M. Yue, and J. Wen, “Supervised search result diversification via subtopic attention,” IEEE Trans. Knowl. Data Eng., vol. 30, no. 10, pp. 1971–1984, 2018.
  63. F. Cai, R. Reinanda, and M. D. Rijke, “Diversifying query auto-completion,” ACM Transactions on Information Systems (TOIS), vol. 34, no. 4, pp. 1–33, 2016.
  64. Y. Yang, A. Lad, N. Lao, A. Harpale, B. Kisiel, and M. Rogati, “Utility-based information distillation over temporally sequenced documents,” in SIGIR 2007: Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Amsterdam, The Netherlands, July 23-27, 2007.   ACM, 2007, pp. 31–38.
  65. Y. Yang and A. Lad, “Modeling expected utility of multi-session information distillation,” in Advances in Information Retrieval Theory, Second International Conference on the Theory of Information Retrieval, ICTIR 2009, Cambridge, UK, September 10-12, 2009, Proceedings, ser. Lecture Notes in Computer Science, vol. 5766.   Springer, 2009, pp. 164–175.
  66. R. Agrawal, S. Gollapudi, A. Halverson, and S. Ieong, “Diversifying search results,” in WSDM.   ACM, 2009, pp. 5–14.
  67. T. Sakai and Z. Zeng, “Retrieval evaluation measures that agree with users’ serp preferences: Traditional, preference-based, and diversity measures,” ACM Transactions on Information Systems (TOIS), vol. 39, no. 2, pp. 1–35, 2020.
  68. T. Sakai, “Evaluation with informational and navigational intents,” in Proceedings of the 21st international conference on World Wide Web, 2012, pp. 499–508.
  69. T. Sakai and R. Song, “Evaluating diversified search results using per-intent graded relevance,” in Proceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval, 2011, pp. 1043–1052.
  70. T. Sakai, N. Craswell, R. Song, S. Robertson, Z. Dou, and C. Lin, “Simple evaluation metrics for diversified search results,” in Proceedings of the 3rd International Workshop on Evaluating Information Access, EVIA 2010.   National Institute of Informatics (NII), 2010, pp. 42–50.
  71. X. Wang, J. Wen, Z. Dou, T. Sakai, and R. Zhang, “Search result diversity evaluation based on intent hierarchies,” IEEE Trans. Knowl. Data Eng., vol. 30, no. 1, pp. 156–169, 2018.
  72. C. Yu, L. V. S. Lakshmanan, and S. Amer-Yahia, “It takes variety to make a world: diversification in recommender systems,” in EDBT 2009, 12th International Conference on Extending Database Technology, Saint Petersburg, Russia, March 24-26, 2009, Proceedings, ser. ACM International Conference Proceeding Series, vol. 360.   ACM, 2009, pp. 368–378.
  73. J. C. Gower, “A general coefficient of similarity and some of its properties,” Biometrics, vol. 27, no. 4, pp. 857–871, 1971.
  74. A. Moffat and J. Zobel, “Rank-biased precision for measurement of retrieval effectiveness,” ACM Trans. Inf. Syst., vol. 27, no. 1, pp. 2:1–2:27, 2008.
  75. E. Amigó, J. Gonzalo, and F. Verdejo, “A general evaluation measure for document organization tasks,” in SIGIR.   ACM, 2013, pp. 643–652.
  76. E. Amigó, D. Spina, and J. C. de Albornoz, “An axiomatic analysis of diversity evaluation metrics: Introducing the rank-biased utility metric,” in SIGIR.   ACM, 2018, pp. 625–634.
  77. B. Carterette, “An analysis of np-completeness in novelty and diversity ranking,” in ICTIR, ser. Lecture Notes in Computer Science, vol. 5766.   Springer, 2009, pp. 200–211.
  78. C. L. A. Clarke, M. Kolla, and O. Vechtomova, “An effectiveness measure for ambiguous and underspecified queries,” in Advances in Information Retrieval Theory, Second International Conference on the Theory of Information Retrieval, ICTIR 2009, Cambridge, UK, September 10-12, 2009, Proceedings, ser. Lecture Notes in Computer Science, vol. 5766.   Springer, 2009, pp. 188–199.
  79. S. Maropaki, S. Chester, C. Doulkeridis, and K. Nørvåg, “Diversifying top-k point-of-interest queries via collective social reach,” in CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, 2020.
  80. L. Chen and H. Shi, “Dexdeepfm: Ensemble diversity enhanced extreme deep factorization machine model,” ACM Transactions on Knowledge Discovery from Data (TKDD), vol. 16, no. 5, pp. 1–17, 2022.
  81. Y. Gu, G. Liu, J. Qi, H. Xu, G. Yu, and R. Zhang, “The moving K diversified nearest neighbor query,” IEEE Trans. Knowl. Data Eng., vol. 28, no. 10, pp. 2778–2792, 2016.
  82. O. Macchi, “The coincidence approach to stochastic point processes,” Advances in Applied Probability, vol. 7, no. 1, p. 83–122, 1975.
  83. C. Zhang, H. Liang, and K. Wang, “Trip recommendation meets real-world constraints: Poi availability, diversity, and traveling time uncertainty,” ACM Transactions on Information Systems (TOIS), vol. 35, no. 1, pp. 1–28, 2016.
  84. C. Li, H. Feng, and M. d. Rijke, “Cascading hybrid bandits: Online learning to rank for relevance and diversity,” in Fourteenth ACM Conference on Recommender Systems, 2020.
  85. Q. Ding, Y. Liu, C. Miao, F. Cheng, and H. Tang, “A hybrid bandit framework for diversified recommendation,” in AAAI.   AAAI Press, 2021, pp. 4036–4044.
  86. L. Qin, S. Chen, and X. Zhu, “Contextual combinatorial bandit and its application on diversified online recommendation,” in SDM.   SIAM, 2014, pp. 461–469.
  87. L. Wang, C. Wang, K. Wang, and X. He, “Biucb: A contextual bandit algorithm for cold-start and diversified recommendation,” in ICBK.   IEEE Computer Society, 2017, pp. 248–253.
  88. J. Parapar and F. Radlinski, “Diverse user preference elicitation with multi-armed bandits,” in WSDM ’21, The Fourteenth ACM International Conference on Web Search and Data Mining, Virtual Event, Israel, March 8-12, 2021.   ACM, 2021, pp. 130–138.
  89. F. Radlinski, R. Kleinberg, and T. Joachims, “Learning diverse rankings with multi-armed bandits,” in Proceedings of the 25th international conference on Machine learning, ser. ACM International Conference Proceeding Series, vol. 307.   ACM, 2008, pp. 784–791.
  90. G. Zheng, F. Zhang, Z. Zheng, Y. Xiang, N. J. Yuan, X. Xie, and Z. Li, “DRN: A deep reinforcement learning framework for news recommendation,” in WWW.   ACM, 2018, pp. 167–176.
  91. A. Kulesza and B. Taskar, “Determinantal point processes for machine learning,” Found. Trends Mach. Learn., vol. 5, no. 2-3, pp. 123–286, 2012.
  92. A. Borodin, “Determinantal point processes,” 2009.
  93. W. R. Thompson, “On the likelihood that one unknown probability exceeds another in view of the evidence of two samples,” Biometrika, vol. 25, no. 3/4, pp. 285–294, 1933.
  94. R. Weber, “On the Gittins Index for Multiarmed Bandits,” The Annals of Applied Probability, vol. 2, no. 4, pp. 1024 – 1033, 1992.
  95. N. Lathia, S. Hailes, L. Capra, and X. Amatriain, “Temporal diversity in recommender systems,” in SIGIR.   ACM, 2010, pp. 210–217.
  96. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nat., vol. 518, no. 7540, pp. 529–533, 2015.
  97. A. Grotov and M. de Rijke, “Online learning to rank for information retrieval: SIGIR 2016 tutorial,” in Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, SIGIR 2016, Pisa, Italy, July 17-21, 2016.   ACM, 2016, pp. 1215–1218.
  98. K. Hofmann, A. Schuth, S. Whiteson, and M. de Rijke, “Reusing historical interaction data for faster online learning to rank for IR,” in Sixth ACM International Conference on Web Search and Data Mining, WSDM 2013, Rome, Italy, February 4-8, 2013.   ACM, 2013, pp. 183–192.
  99. Y. Yue and T. Joachims, “Interactively optimizing information retrieval systems as a dueling bandits problem,” in Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009, ser. ACM International Conference Proceeding Series, vol. 382.   ACM, 2009, pp. 1201–1208.
  100. M. Wu, Y. Chang, Z. Zheng, and H. Zha, “Smoothing DCG for learning to rank: a novel approach using smoothed hinge functions,” in Proceedings of the 18th ACM Conference on Information and Knowledge Management, CIKM 2009, Hong Kong, China, November 2-6, 2009.   ACM, 2009, pp. 1923–1926.
  101. V. Do and N. Usunier, “Optimizing generalized gini indices for fairness in rankings,” in SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022.   ACM, 2022, pp. 737–747.
Citations (7)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now