Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Sampling Individually-Fair Rankings that are Always Group Fair (2306.11964v1)

Published 21 Jun 2023 in cs.CY, cs.DS, cs.IR, cs.LG, and stat.ML

Abstract: Rankings on online platforms help their end-users find the relevant information -- people, news, media, and products -- quickly. Fair ranking tasks, which ask to rank a set of items to maximize utility subject to satisfying group-fairness constraints, have gained significant interest in the Algorithmic Fairness, Information Retrieval, and Machine Learning literature. Recent works, however, identify uncertainty in the utilities of items as a primary cause of unfairness and propose introducing randomness in the output. This randomness is carefully chosen to guarantee an adequate representation of each item (while accounting for the uncertainty). However, due to this randomness, the output rankings may violate group fairness constraints. We give an efficient algorithm that samples rankings from an individually-fair distribution while ensuring that every output ranking is group fair. The expected utility of the output ranking is at least $\alpha$ times the utility of the optimal fair solution. Here, $\alpha$ depends on the utilities, position-discounts, and constraints -- it approaches 1 as the range of utilities or the position-discounts shrinks, or when utilities satisfy distributional assumptions. Empirically, we observe that our algorithm achieves individual and group fairness and that Pareto dominates the state-of-the-art baselines.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (76)
  1. “Diversifying Search Results” In WSDM ACM, 2009, pp. 5–14
  2. Pranjal Awasthi, Matthäus Kleindessner and Jamie Morgenstern “Equalized Odds Postprocessing under Imperfect Group Information” In International Conference on Artificial Intelligence and Statistics, 2020, pp. 1770–1780 PMLR
  3. “Incremental Cluster-Based Retrieval Using Compressed Cluster-Skipping Inverted Files” In ACM Transactions on Information Systems (TOIS) 26.3 ACM New York, NY, USA, 2008, pp. 1–36
  4. Mohammad Ahsanullah, Valery B. Nevzorov and Mohammad Shakil “An Introduction to Order Statistics” Springer, 2013
  5. “Sequential Submodular Maximization and Applications to Ranking an Assortment of Products” In Operations Research INFORMS, 2022
  6. “Diversity constraints in public housing allocation” In 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2018), 2018
  7. “Random Sampling from a Search Engine’s Index” In Journal of the ACM (JACM) 55.5 ACM New York, NY, USA, 2008, pp. 1–74
  8. Asia J. Biega, Krishna P. Gummadi and Gerhard Weikum “Equity of Attention: Amortizing Individual Fairness in Rankings” In SIGIR ACM, 2018, pp. 405–414
  9. Richard A. Brualdi “Notes on the Birkhoff Algorithm for Doubly Stochastic Matrices” In Canadian Mathematical Bulletin 25.2 Cambridge University Press, 1982, pp. 191–199 DOI: 10.4153/CMB-1982-026-3
  10. “Learning to Rank Using Gradient Descent” In Proceedings of the 22nd international conference on Machine learning, 2005, pp. 89–96
  11. Christopher J.C. Burges “From RankNet to LambdaRank to LambdaMART: An Overview” In Learning Microsoft Research Technical Report, 2010
  12. “Advances in TF-Ranking” https://ai.googleblog.com/2021/07/advances-in-tf-ranking.html In Google AI: Blog, 2021
  13. Carlos Castillo “Fairness and Transparency in Ranking” In SIGIR Forum 52.2 New York, NY, USA: Association for Computing Machinery, 2019, pp. 64–71 DOI: 10.1145/3308774.3308783
  14. “Controlling Polarization in Personalization: An Algorithmic Framework” In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19 Atlanta, GA, USA: Association for Computing Machinery, 2019, pp. 160–169 DOI: 10.1145/3287560.3287601
  15. “Fair Classification with Noisy Protected Attributes” In ICML 120, Proceedings of Machine Learning Research PMLR, 2021
  16. “Matroids, Matchings, and Fairness” In AISTATS 89, Proceedings of Machine Learning Research PMLR, 2019, pp. 2212–2220
  17. “Implicit Diversity in Image Summarization” In Proc. ACM Hum.-Comput. Interact. 4.CSCW2 New York, NY, USA: Association for Computing Machinery, 2020 DOI: 10.1145/3415210
  18. Cyril W Cleverdon “The Significance of the Cranfield Tests on Index Languages” In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, 1991, pp. 3–12
  19. Moses Charikar, Konstantin Makarychev and Yury Makarychev “Near-Optimal Algorithms for Unique Games” In Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing, STOC ’06 Seattle, WA, USA: Association for Computing Machinery, 2006, pp. 205–214 DOI: 10.1145/1132516.1132547
  20. L.Elisa Celis, Anay Mehrotra and Nisheeth K. Vishnoi “Fair Classification with Adversarial Perturbations” In Advances in Neural Information Processing Systems, 2021 URL: https://openreview.net/forum?id=LEqVjnffcWo
  21. L.Elisa Celis, Damian Straszak and Nisheeth K. Vishnoi “Ranking with Fairness Constraints” In 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018) 107, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2018, pp. 28:1–28:15 DOI: 10.4230/LIPIcs.ICALP.2018.28
  22. “Fairness in Matching under Uncertainty” arXiv, 2023 DOI: 10.48550/ARXIV.2302.03810
  23. Kushal Dave, Steve Lawrence and David M Pennock “Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews” In Proceedings of the 12th international conference on World Wide Web, 2003, pp. 519–528
  24. “On Fair Selection in the Presence of Implicit Variance” In Proceedings of the 21st ACM Conference on Economics and Computation, EC ’20 Virtual Event, Hungary: Association for Computing Machinery, 2020, pp. 649–675 DOI: 10.1145/3391403.3399482
  25. Robert Epstein and Ronald E Robertson “The Search Engine Manipulation Effect (SEME) And Its Possible Impact on the Outcomes of Elections” In Proceedings of the National Academy of Sciences 112.33 National Acad Sciences, 2015, pp. E4512–E4521 DOI: 10.1073/pnas.1419828112
  26. “Probabilistic Fair Clustering” In NeurIPS, 2020
  27. “Fair algorithms for selecting citizens’ assemblies” In Nature 596.7873, 2021, pp. 548–552
  28. Sahin Cem Geyik, Stuart Ambler and Krishnaram Kenthapadi “Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search” In KDD ACM, 2019, pp. 2221–2231
  29. “Maxmin-Fair Ranking: Individual Fairness under Group-Fairness Constraints” In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’21 Virtual Event, Singapore: Association for Computing Machinery, 2021, pp. 436–446 DOI: 10.1145/3447548.3467349
  30. Sruthi Gorantla, Amit Deshpande and Anand Louis “On the Problem of Underranking in Group-Fair Ranking” In ICML 139, Proceedings of Machine Learning Research PMLR, 2021, pp. 3777–3787
  31. Avijit Ghosh, Ritam Dutt and Christo Wilson “When Fair Ranking Meets Uncertain Inference” In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 1033–1043 DOI: 10.1145/3404835.3462850
  32. M. Grötschel, L. Lovász and A. Schrijver “Geometric Methods in Combinatorial Optimization” In Progress in Combinatorial Optimization Academic Press, 1984, pp. 167–183 DOI: https://doi.org/10.1016/B978-0-12-566780-7.50016-7
  33. “Bias in online freelance marketplaces: Evidence from taskrabbit and fiverr” In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, 2017, pp. 1914–1933
  34. “Cumulated Gain-Based Evaluation of IR Techniques” In ACM Trans. Inf. Syst. 20.4 New York, NY, USA: Association for Computing Machinery, 2002, pp. 422–446 DOI: 10.1145/582415.582418
  35. “Scaling Personalized Web Search” In Proceedings of the 12th international conference on World Wide Web, 2003, pp. 271–279 ACM
  36. Deborah K King “Multiple jeopardy, multiple consciousness: The context of a Black feminist ideology” In Signs: Journal of women in culture and society 14.1 University of Chicago Press, 1988, pp. 42–72
  37. Jon M. Kleinberg “Authoritative Sources in a Hyperlinked Environment” In J. ACM 46.5 New York, NY, USA: Association for Computing Machinery, 1999, pp. 604–632 DOI: 10.1145/324133.324140
  38. Matthew Kay, Cynthia Matuszek and Sean A. Munson “Unequal Representation and Gender Stereotypes in Image Search Results for Occupations” In CHI ACM, 2015, pp. 3819–3828
  39. “Selection Problems in the Presence of Implicit Bias” In 9th Innovations in Theoretical Computer Science Conference (ITCS 2018) 94, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2018, pp. 33:1–33:17 DOI: 10.4230/LIPIcs.ITCS.2018.33
  40. Till Kletti, Jean-Michel Renders and Patrick Loiseau “Introducing the Expohedron for Efficient Pareto-Optimal Fairness-Utility Amortizations in Repeated Rankings” In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, WSDM ’22 Virtual Event, AZ, USA: Association for Computing Machinery, 2022, pp. 498–507 DOI: 10.1145/3488560.3498490
  41. Jon Kleinberg, Emily Ryu and Éva Tardos “Ordered Submodularity and its Applications to Diversifying Recommendations” In arXiv preprint arXiv:2203.00233, 2022
  42. Elizabeth D. Liddy “Automatic Document Retrieval” In Encyclopedia of Language and Linguistics Elsevier, 2005
  43. Tie-Yan Liu “Learning to Rank for Information Retrieval” Hanover, MA, USA: Now Publishers Inc., 2009, pp. 225–331 DOI: 10.1561/1500000016
  44. Lap Chi Lau, R. Ravi and Mohit Singh “Iterative Methods in Combinatorial Optimization”, Cambridge Texts in Applied Mathematics Cambridge University Press, 2011 URL: https://books.google.com/books?id=TjE7g0Yr0ScC
  45. Alexandre Louis Lamy and Ziyuan Zhong “Noise-Tolerant Fair Classification” In NeurIPS, 2019, pp. 294–305
  46. “Mitigating Bias in Set Selection with Noisy Protected Attributes” In FAccT ACM, 2021, pp. 237–248
  47. “Fairness for Robust Learning to Rank” In CoRR abs/2112.06288, 2021 arXiv: https://arxiv.org/abs/2112.06288
  48. Christopher Mims “Why Social Media Is So Good at Polarizing Us” https://www.wsj.com/articles/why-social-media-is-so-good-at-polarizing-us-11603105204 In The Wall Street Journal, 2020
  49. Hussein Mozannar, Mesrob I. Ohannessian and Nathan Srebro “Fair Learning with Private Demographic Data” In ICML 119, Proceedings of Machine Learning Research PMLR, 2020, pp. 7066–7075
  50. Anay Mehrotra, Bary S.R. Pradelski and Nisheeth K. Vishnoi “Selection in the Presence of Implicit Bias: The Advantage of Intersectional Constraints” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 599–609 DOI: 10.1145/3531146.3533124
  51. “Randomized Algorithms” Cambridge university press, 1995
  52. Christopher Manning, Prabhakar Raghavan and Hinrich Schütze “Introduction to Information Retrieval” In Natural Language Engineering 16.1 Cambridge university press, 2010, pp. 100–103
  53. Anay Mehrotra and Nisheeth K. Vishnoi “Fair Ranking with Noisy Protected Attributes” In Thirty-Sixth Conference on Neural Information Processing Systems, 2022 URL: https://openreview.net/forum?id=mTra5BIUyRV
  54. Meghana Nasre, Prajakta Nimbhorkar and Nada Pulath “Classified Rank-Maximal Matchings and Popular Matchings – Algorithms and Hardness” In Graph-Theoretic Concepts in Computer Science Cham: Springer International Publishing, 2019, pp. 244–257
  55. Safiya Umoja Noble “Algorithms of Oppression: How Search Engines Reinforce Racism” NYU Press, 2018
  56. Harrie Oosterhuis “Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness” In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 1023–1032 DOI: 10.1145/3404835.3462830
  57. “Fair Ranking: A Critical Review, Challenges, and Future Directions” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 1929–1942 DOI: 10.1145/3531146.3533238
  58. Atasi Panda, Anand Louis and Prajakta Nimbhorkar “Bipartite Matchings with Group Fairness and Individual Fairness Constraints” In CoRR abs/2208.09951, 2022
  59. Evaggelia Pitoura, Kostas Stefanidis and Georgia Koutrika “Fairness in Rankings and Recommendations: An Overview” In The VLDB Journal, 2021 URL: https://doi.org/10.1007/s00778-021-00697-y
  60. Alexander Schrijver “Combinatorial Optimization: Polyhedra and Efficiency”, Algorithms and Combinatorics v. 1, 2, and 3 Springer, 2003 URL: https://books.google.com/books?id=mqGeSQ6dJycC
  61. “Fairness of Exposure in Rankings” In KDD ACM, 2018, pp. 2219–2228
  62. “Policy Learning for Fairness in Ranking” In NeurIPS, 2019, pp. 5427–5437
  63. Ashudeep Singh, David Kempe and Thorsten Joachims “Fairness in Ranking under Uncertainty” In Advances in Neural Information Processing Systems 34 Curran Associates, Inc., 2021, pp. 11896–11908 URL: https://proceedings.neurips.cc/paper/2021/file/63c3ddcc7b23daa1e42dc41f9a44a873-Paper.pdf
  64. Pavan Kumar C Singitham, Mahathi S Mahabhashyam and Prabhakar Raghavan “Efficiency-quality tradeoffs for vector score aggregation” In Proceedings of the Thirtieth international conference on Very large data bases-Volume 30, 2004, pp. 624–635
  65. “Optimisation Methods for Ranking Functions with Multiple Parameters” In Proceedings of the 15th ACM international conference on Information and knowledge management, 2006, pp. 585–593
  66. Dirk Johannes Thomas “Matching Problems with Additional Resource Constraints”, 2016 DOI: 10.25353/ubtr-xxxx-7644-a670/
  67. “Robust Optimization for Fairness with Noisy Protected Groups” In NeurIPS, 2020
  68. Jason Weston, Samy Bengio and Nicolas Usunier “Large Scale Image Annotation: Learning to Rank with Joint Word-Image Embeddings” In Machine learning 81.1 Springer, 2010, pp. 21–35
  69. Jialu Wang, Yang Liu and Caleb Levy “Fair Classification with Group-Dependent Label Noise” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 526–536 DOI: 10.1145/3442188.3445915
  70. Ke Yang, Vasilis Gkatzelis and Julia Stoyanovich “Balanced Ranking with Diversity Constraints” In IJCAI ijcai.org, 2019, pp. 6035–6042
  71. Ke Yang, Joshua R. Loftus and Julia Stoyanovich “Causal Intersectionality and Fair Ranking” In FORC 192, LIPIcs Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021, pp. 7:1–7:20
  72. “Measuring Fairness in Ranked Outputs” In Proceedings of the 29th International Conference on Scientific and Statistical Database Management, Chicago, IL, USA, June 27-29, 2017 Chicago, IL, USA: ACM, 2017, pp. 22:1–22:6 DOI: 10.1145/3085504.3085526
  73. “Reducing Disparate Exposure in Ranking: A Learning To Rank Approach” In WWW ACM / IW3C2, 2020, pp. 2849–2855
  74. “FA*IR: A Fair Top-k Ranking Algorithm” In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17 Singapore, Singapore: Association for Computing Machinery, 2017, pp. 1569–1578 DOI: 10.1145/3132847.3132938
  75. Meike Zehlike, Ke Yang and Julia Stoyanovich “Fairness in Ranking, Part I: Score-Based Ranking” Just Accepted In ACM Comput. Surv. New York, NY, USA: Association for Computing Machinery, 2022 URL: https://doi.org/10.1145/3533379
  76. Meike Zehlike, Ke Yang and Julia Stoyanovich “Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems” Just Accepted In ACM Comput. Surv. New York, NY, USA: Association for Computing Machinery, 2022 DOI: 10.1145/3533380
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Sruthi Gorantla (9 papers)
  2. Anay Mehrotra (25 papers)
  3. Amit Deshpande (35 papers)
  4. Anand Louis (35 papers)
Citations (3)

Summary

We haven't generated a summary for this paper yet.