Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Understanding User Behavior in Carousel Recommendation Systems for Click Modeling and Learning to Rank (2307.01866v1)

Published 4 Jul 2023 in cs.IR and cs.HC

Abstract: Carousels (also-known as multilists) have become the standard user interface for e-commerce platforms replacing the ranked list, the previous standard for recommender systems. While the research community has begun to focus on carousels, there are many unanswered questions and undeveloped areas when compared to the literature for ranked lists, which includes information retrieval research on the presentation of web search results. This work is an extended abstract for the RecSys 2023 Doctoral Symposium outlining a PhD project, with the main contribution of addressing the undeveloped areas in carousel recommenders: 1) the formulation of new click models and 2) learning to rank with click data. We present two significant barriers for this contribution and the field: lack of public datasets and lack of eye tracking user studies of browsing behavior. Clicks, the standard feedback collected by recommender systems, are insufficient to understand the whole interaction process of a user with a recommender requiring system designers to make assumptions, especially on browsing behavior. Eye tracking provides a means to elucidate the process and test these assumptions. Thus, to address these barriers and encourage future work, we will conduct an eye tracking user study within a carousel movie recommendation setting and make the dataset publicly available. Moreover, the insights learned on browsing behavior will help motivate the formulation of new click models and learning to rank.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (43)
  1. Learning user interaction models for predicting web search result preferences. In Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, Seattle Washington USA, 3–10. https://doi.org/10.1145/1148170.1148175
  2. Carousel Personalization in Music Streaming Apps with Contextual Bandits. In Fourteenth ACM Conference on Recommender Systems. ACM, Virtual Event Brazil, 420–425. https://doi.org/10.1145/3383313.3412217
  3. The netflix prize. In Proceedings of KDD cup and workshop, Vol. 2007. New York, 35.
  4. Eye-tracking product recommenders’ usage. In Proceedings of the fourth ACM conference on Recommender systems - RecSys ’10. ACM Press, Barcelona, Spain, 29. https://doi.org/10.1145/1864708.1864717
  5. Rocío Cañamares and Pablo Castells. 2018. Should I Follow the Crowd?: A Probabilistic Analysis of the Effectiveness of Popularity in Recommender Systems. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, Ann Arbor MI USA, 415–424. https://doi.org/10.1145/3209978.3210014
  6. Olivier Chapelle and Ya Zhang. 2009. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th international conference on World wide web. ACM, Madrid Spain, 1–10. https://doi.org/10.1145/1526709.1526711
  7. Eye-tracking-based personality prediction with recommendation interfaces. User Modeling and User-Adapted Interaction (June 2022). https://doi.org/10.1007/s11257-022-09336-9
  8. Optimizing two-dimensional search results presentation. In Proceedings of the fourth ACM international conference on Web search and data mining. ACM, Hong Kong China, 257–266. https://doi.org/10.1145/1935826.1935873
  9. An experimental comparison of click position-bias models. In Proceedings of the international conference on Web search and web data mining - WSDM ’08. ACM Press, Palo Alto, California, USA, 87. https://doi.org/10.1145/1341531.1341545
  10. Edward Cutrell and Zhiwei Guan. 2007. What are you looking for?: an eye-tracking study of information usage in web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, San Jose California USA, 407–416. https://doi.org/10.1145/1240624.1240690
  11. Eye Tracking as a Source of Implicit Feedback in Recommender Systems: A Preliminary Analysis. In 2023 Symposium on Eye Tracking Research and Applications. ACM, Tubingen Germany, 1–3. https://doi.org/10.1145/3588015.3589511
  12. Learning to Rank in the Position Based Model with Bandit Feedback. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. ACM, Virtual Event Ireland, 2405–2412. https://doi.org/10.1145/3340531.3412723
  13. Measuring the User Satisfaction in a Recommendation Interface with Multiple Carousels. In ACM International Conference on Interactive Media Experiences. ACM, Virtual Event USA, 212–217. https://doi.org/10.1145/3452918.3465493
  14. A Methodology for the Offline Evaluation of Recommender Systems in a User Interface with Multiple Carousels. In Adjunct Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. ACM, Utrecht Netherlands, 10–15. https://doi.org/10.1145/3450614.3461680
  15. A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research. ACM Transactions on Information Systems 39, 2 (Jan. 2021), 20:1–20:49. https://doi.org/10.1145/3434185
  16. Offline Evaluation of Recommender Systems in a User Interface With Multiple Carousels. Frontiers in Big Data 5 (June 2022), 910030. https://doi.org/10.3389/fdata.2022.910030
  17. Antonino Freno. 2017. Practical Lessons from Developing a Large-Scale Recommender System at Zalando. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, Como Italy, 251–259. https://doi.org/10.1145/3109859.3109897
  18. Analysis of User Behavior in Interfaces with Recommended Items: An Eye-tracking Study. In Proceedings of the 5th Joint Workshop on Interfaces and Human Decision Making for Recommender Systems co-located with ACM Conference on Recommender Systems (RecSys 2018). 5. http://ceur-ws.org/Vol-2225/paper5.pdf
  19. Eye-tracking analysis of user behavior in WWW search. In Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, Sheffield United Kingdom, 478–479. https://doi.org/10.1145/1008992.1009079
  20. Zhiwei Guan and Edward Cutrell. 2007. An eye tracking study of the effect of target rank on web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, San Jose California USA, 417–420. https://doi.org/10.1145/1240624.1240691
  21. Click chain model in web search. In Proceedings of the 18th international conference on World wide web. ACM, Madrid Spain, 11–20. https://doi.org/10.1145/1526709.1526712
  22. Efficient multiple-click models in web search. In Proceedings of the Second ACM International Conference on Web Search and Data Mining. ACM, Barcelona Spain, 124–131. https://doi.org/10.1145/1498759.1498818
  23. F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems 5, 4 (Jan. 2015), 1–19. https://doi.org/10.1145/2827872
  24. Exploring Multi-List User Interfaces for Similar-Item Recommendations. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. ACM, Utrecht Netherlands, 224–228. https://doi.org/10.1145/3450613.3456809
  25. Thorsten Joachims. 2002. Optimizing search engines using clickthrough data. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, Edmonton Alberta Canada, 133–142. https://doi.org/10.1145/775047.775067
  26. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, Salvador Brazil, 154–161. https://doi.org/10.1145/1076034.1076063
  27. Evaluating the accuracy of implicit feedback from clicks and query reformulations in Web search. ACM Transactions on Information Systems 25, 2 (April 2007), 7. https://doi.org/10.1145/1229179.1229181
  28. Unbiased Learning-to-Rank with Biased Feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM, Cambridge United Kingdom, 781–789. https://doi.org/10.1145/3018661.3018699
  29. Yvonne Kammerer and Peter Gerjets. 2010. How the interface design influences users’ spontaneous trustworthiness evaluations of web search results: comparing a list and a grid interface. In Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications - ETRA ’10. ACM Press, Austin, Texas, 299. https://doi.org/10.1145/1743666.1743736
  30. Are people biased in their use of search engines? Commun. ACM 51, 2 (Feb. 2008), 49–52. https://doi.org/10.1145/1314215.1314224
  31. Towards Measuring and Inferring User Interest from Gaze. In Proceedings of the 26th International Conference on World Wide Web Companion (WWW ’17 Companion). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 525–533. https://doi.org/10.1145/3041021.3054182
  32. Variational Autoencoders for Collaborative Filtering. In Proceedings of the 2018 World Wide Web Conference (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 689–698. https://doi.org/10.1145/3178876.3186150
  33. Page-level Optimization of e-Commerce Item Recommendations. In Fifteenth ACM Conference on Recommender Systems. ACM, Amsterdam Netherlands, 495–504. https://doi.org/10.1145/3460231.3474242
  34. Estimating Credibility of User Clicks with Mouse Movement and Eye-Tracking Information. In Natural Language Processing and Chinese Computing, Chengqing Zong, Jian-Yun Nie, Dongyan Zhao, and Yansong Feng (Eds.). Vol. 496. Springer Berlin Heidelberg, Berlin, Heidelberg, 263–274. https://doi.org/10.1007/978-3-662-45924-9_24 Series Title: Communications in Computer and Information Science.
  35. Explore, exploit, and explain: personalizing explainable recommendations with bandits. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, Vancouver British Columbia Canada, 31–39. https://doi.org/10.1145/3240323.3240354
  36. Classifeye: Classification of Personal Characteristics Based on Eye Tracking Data in a Recommender System Interface. In HUMANIZE: Joint Proceedings of the ACM IUI 2021 Workshops, Vol. 2903. CEUR-WS, 12. http://ceur-ws.org/Vol-2903/IUI21WS-HUMANIZE-4.pdf
  37. The determinants of web page viewing behavior: an eye-tracking study. In Proceedings of the Eye tracking research & applications symposium on Eye tracking research & applications - ETRA’2004. ACM Press, San Antonio, Texas, 147–154. https://doi.org/10.1145/968363.968391
  38. Behnam Rahdari and Peter Brusilovsky. 2022. Simulation-Based Evaluation of Interactive Recommender Systems. (Sept. 2022).
  39. From Ranked Lists to Carousels: A Carousel Click Model. http://arxiv.org/abs/2209.13426 arXiv:2209.13426 [cs, math].
  40. Predicting clicks: estimating the click-through rate for new ads. In Proceedings of the 16th international conference on World Wide Web. ACM, Banff Alberta Canada, 521–530. https://doi.org/10.1145/1242572.1242643
  41. Comprehension effects of signalling relationships between documents in search engines. Computers in Human Behavior 26, 3 (May 2010), 419–426. https://doi.org/10.1016/j.chb.2009.11.013
  42. “Serving Each User”: Supporting Different Eating Goals Through a Multi-List Recommender Interface. In Fifteenth ACM Conference on Recommender Systems. ACM, Amsterdam Netherlands, 124–132. https://doi.org/10.1145/3460231.3474232
  43. Gaze Prediction for Recommender Systems. Proceedings of the 10th ACM Conference on Recommender Systems - RecSys ’16 (2016), 131–138. https://doi.org/10.1145/2959100.2959150 ISBN: 9781450340359.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now