Sampling Individually-Fair Rankings that are Always Group Fair (2306.11964v1)
Abstract: Rankings on online platforms help their end-users find the relevant information -- people, news, media, and products -- quickly. Fair ranking tasks, which ask to rank a set of items to maximize utility subject to satisfying group-fairness constraints, have gained significant interest in the Algorithmic Fairness, Information Retrieval, and Machine Learning literature. Recent works, however, identify uncertainty in the utilities of items as a primary cause of unfairness and propose introducing randomness in the output. This randomness is carefully chosen to guarantee an adequate representation of each item (while accounting for the uncertainty). However, due to this randomness, the output rankings may violate group fairness constraints. We give an efficient algorithm that samples rankings from an individually-fair distribution while ensuring that every output ranking is group fair. The expected utility of the output ranking is at least $\alpha$ times the utility of the optimal fair solution. Here, $\alpha$ depends on the utilities, position-discounts, and constraints -- it approaches 1 as the range of utilities or the position-discounts shrinks, or when utilities satisfy distributional assumptions. Empirically, we observe that our algorithm achieves individual and group fairness and that Pareto dominates the state-of-the-art baselines.
- “Diversifying Search Results” In WSDM ACM, 2009, pp. 5–14
- Pranjal Awasthi, Matthäus Kleindessner and Jamie Morgenstern “Equalized Odds Postprocessing under Imperfect Group Information” In International Conference on Artificial Intelligence and Statistics, 2020, pp. 1770–1780 PMLR
- “Incremental Cluster-Based Retrieval Using Compressed Cluster-Skipping Inverted Files” In ACM Transactions on Information Systems (TOIS) 26.3 ACM New York, NY, USA, 2008, pp. 1–36
- Mohammad Ahsanullah, Valery B. Nevzorov and Mohammad Shakil “An Introduction to Order Statistics” Springer, 2013
- “Sequential Submodular Maximization and Applications to Ranking an Assortment of Products” In Operations Research INFORMS, 2022
- “Diversity constraints in public housing allocation” In 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2018), 2018
- “Random Sampling from a Search Engine’s Index” In Journal of the ACM (JACM) 55.5 ACM New York, NY, USA, 2008, pp. 1–74
- Asia J. Biega, Krishna P. Gummadi and Gerhard Weikum “Equity of Attention: Amortizing Individual Fairness in Rankings” In SIGIR ACM, 2018, pp. 405–414
- Richard A. Brualdi “Notes on the Birkhoff Algorithm for Doubly Stochastic Matrices” In Canadian Mathematical Bulletin 25.2 Cambridge University Press, 1982, pp. 191–199 DOI: 10.4153/CMB-1982-026-3
- “Learning to Rank Using Gradient Descent” In Proceedings of the 22nd international conference on Machine learning, 2005, pp. 89–96
- Christopher J.C. Burges “From RankNet to LambdaRank to LambdaMART: An Overview” In Learning Microsoft Research Technical Report, 2010
- “Advances in TF-Ranking” https://ai.googleblog.com/2021/07/advances-in-tf-ranking.html In Google AI: Blog, 2021
- Carlos Castillo “Fairness and Transparency in Ranking” In SIGIR Forum 52.2 New York, NY, USA: Association for Computing Machinery, 2019, pp. 64–71 DOI: 10.1145/3308774.3308783
- “Controlling Polarization in Personalization: An Algorithmic Framework” In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19 Atlanta, GA, USA: Association for Computing Machinery, 2019, pp. 160–169 DOI: 10.1145/3287560.3287601
- “Fair Classification with Noisy Protected Attributes” In ICML 120, Proceedings of Machine Learning Research PMLR, 2021
- “Matroids, Matchings, and Fairness” In AISTATS 89, Proceedings of Machine Learning Research PMLR, 2019, pp. 2212–2220
- “Implicit Diversity in Image Summarization” In Proc. ACM Hum.-Comput. Interact. 4.CSCW2 New York, NY, USA: Association for Computing Machinery, 2020 DOI: 10.1145/3415210
- Cyril W Cleverdon “The Significance of the Cranfield Tests on Index Languages” In Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, 1991, pp. 3–12
- Moses Charikar, Konstantin Makarychev and Yury Makarychev “Near-Optimal Algorithms for Unique Games” In Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing, STOC ’06 Seattle, WA, USA: Association for Computing Machinery, 2006, pp. 205–214 DOI: 10.1145/1132516.1132547
- L.Elisa Celis, Anay Mehrotra and Nisheeth K. Vishnoi “Fair Classification with Adversarial Perturbations” In Advances in Neural Information Processing Systems, 2021 URL: https://openreview.net/forum?id=LEqVjnffcWo
- L.Elisa Celis, Damian Straszak and Nisheeth K. Vishnoi “Ranking with Fairness Constraints” In 45th International Colloquium on Automata, Languages, and Programming (ICALP 2018) 107, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2018, pp. 28:1–28:15 DOI: 10.4230/LIPIcs.ICALP.2018.28
- “Fairness in Matching under Uncertainty” arXiv, 2023 DOI: 10.48550/ARXIV.2302.03810
- Kushal Dave, Steve Lawrence and David M Pennock “Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews” In Proceedings of the 12th international conference on World Wide Web, 2003, pp. 519–528
- “On Fair Selection in the Presence of Implicit Variance” In Proceedings of the 21st ACM Conference on Economics and Computation, EC ’20 Virtual Event, Hungary: Association for Computing Machinery, 2020, pp. 649–675 DOI: 10.1145/3391403.3399482
- Robert Epstein and Ronald E Robertson “The Search Engine Manipulation Effect (SEME) And Its Possible Impact on the Outcomes of Elections” In Proceedings of the National Academy of Sciences 112.33 National Acad Sciences, 2015, pp. E4512–E4521 DOI: 10.1073/pnas.1419828112
- “Probabilistic Fair Clustering” In NeurIPS, 2020
- “Fair algorithms for selecting citizens’ assemblies” In Nature 596.7873, 2021, pp. 548–552
- Sahin Cem Geyik, Stuart Ambler and Krishnaram Kenthapadi “Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search” In KDD ACM, 2019, pp. 2221–2231
- “Maxmin-Fair Ranking: Individual Fairness under Group-Fairness Constraints” In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’21 Virtual Event, Singapore: Association for Computing Machinery, 2021, pp. 436–446 DOI: 10.1145/3447548.3467349
- Sruthi Gorantla, Amit Deshpande and Anand Louis “On the Problem of Underranking in Group-Fair Ranking” In ICML 139, Proceedings of Machine Learning Research PMLR, 2021, pp. 3777–3787
- Avijit Ghosh, Ritam Dutt and Christo Wilson “When Fair Ranking Meets Uncertain Inference” In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 1033–1043 DOI: 10.1145/3404835.3462850
- M. Grötschel, L. Lovász and A. Schrijver “Geometric Methods in Combinatorial Optimization” In Progress in Combinatorial Optimization Academic Press, 1984, pp. 167–183 DOI: https://doi.org/10.1016/B978-0-12-566780-7.50016-7
- “Bias in online freelance marketplaces: Evidence from taskrabbit and fiverr” In Proceedings of the 2017 ACM conference on computer supported cooperative work and social computing, 2017, pp. 1914–1933
- “Cumulated Gain-Based Evaluation of IR Techniques” In ACM Trans. Inf. Syst. 20.4 New York, NY, USA: Association for Computing Machinery, 2002, pp. 422–446 DOI: 10.1145/582415.582418
- “Scaling Personalized Web Search” In Proceedings of the 12th international conference on World Wide Web, 2003, pp. 271–279 ACM
- Deborah K King “Multiple jeopardy, multiple consciousness: The context of a Black feminist ideology” In Signs: Journal of women in culture and society 14.1 University of Chicago Press, 1988, pp. 42–72
- Jon M. Kleinberg “Authoritative Sources in a Hyperlinked Environment” In J. ACM 46.5 New York, NY, USA: Association for Computing Machinery, 1999, pp. 604–632 DOI: 10.1145/324133.324140
- Matthew Kay, Cynthia Matuszek and Sean A. Munson “Unequal Representation and Gender Stereotypes in Image Search Results for Occupations” In CHI ACM, 2015, pp. 3819–3828
- “Selection Problems in the Presence of Implicit Bias” In 9th Innovations in Theoretical Computer Science Conference (ITCS 2018) 94, Leibniz International Proceedings in Informatics (LIPIcs) Dagstuhl, Germany: Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik, 2018, pp. 33:1–33:17 DOI: 10.4230/LIPIcs.ITCS.2018.33
- Till Kletti, Jean-Michel Renders and Patrick Loiseau “Introducing the Expohedron for Efficient Pareto-Optimal Fairness-Utility Amortizations in Repeated Rankings” In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, WSDM ’22 Virtual Event, AZ, USA: Association for Computing Machinery, 2022, pp. 498–507 DOI: 10.1145/3488560.3498490
- Jon Kleinberg, Emily Ryu and Éva Tardos “Ordered Submodularity and its Applications to Diversifying Recommendations” In arXiv preprint arXiv:2203.00233, 2022
- Elizabeth D. Liddy “Automatic Document Retrieval” In Encyclopedia of Language and Linguistics Elsevier, 2005
- Tie-Yan Liu “Learning to Rank for Information Retrieval” Hanover, MA, USA: Now Publishers Inc., 2009, pp. 225–331 DOI: 10.1561/1500000016
- Lap Chi Lau, R. Ravi and Mohit Singh “Iterative Methods in Combinatorial Optimization”, Cambridge Texts in Applied Mathematics Cambridge University Press, 2011 URL: https://books.google.com/books?id=TjE7g0Yr0ScC
- Alexandre Louis Lamy and Ziyuan Zhong “Noise-Tolerant Fair Classification” In NeurIPS, 2019, pp. 294–305
- “Mitigating Bias in Set Selection with Noisy Protected Attributes” In FAccT ACM, 2021, pp. 237–248
- “Fairness for Robust Learning to Rank” In CoRR abs/2112.06288, 2021 arXiv: https://arxiv.org/abs/2112.06288
- Christopher Mims “Why Social Media Is So Good at Polarizing Us” https://www.wsj.com/articles/why-social-media-is-so-good-at-polarizing-us-11603105204 In The Wall Street Journal, 2020
- Hussein Mozannar, Mesrob I. Ohannessian and Nathan Srebro “Fair Learning with Private Demographic Data” In ICML 119, Proceedings of Machine Learning Research PMLR, 2020, pp. 7066–7075
- Anay Mehrotra, Bary S.R. Pradelski and Nisheeth K. Vishnoi “Selection in the Presence of Implicit Bias: The Advantage of Intersectional Constraints” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 599–609 DOI: 10.1145/3531146.3533124
- “Randomized Algorithms” Cambridge university press, 1995
- Christopher Manning, Prabhakar Raghavan and Hinrich Schütze “Introduction to Information Retrieval” In Natural Language Engineering 16.1 Cambridge university press, 2010, pp. 100–103
- Anay Mehrotra and Nisheeth K. Vishnoi “Fair Ranking with Noisy Protected Attributes” In Thirty-Sixth Conference on Neural Information Processing Systems, 2022 URL: https://openreview.net/forum?id=mTra5BIUyRV
- Meghana Nasre, Prajakta Nimbhorkar and Nada Pulath “Classified Rank-Maximal Matchings and Popular Matchings – Algorithms and Hardness” In Graph-Theoretic Concepts in Computer Science Cham: Springer International Publishing, 2019, pp. 244–257
- Safiya Umoja Noble “Algorithms of Oppression: How Search Engines Reinforce Racism” NYU Press, 2018
- Harrie Oosterhuis “Computationally Efficient Optimization of Plackett-Luce Ranking Models for Relevance and Fairness” In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 1023–1032 DOI: 10.1145/3404835.3462830
- “Fair Ranking: A Critical Review, Challenges, and Future Directions” In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22 Seoul, Republic of Korea: Association for Computing Machinery, 2022, pp. 1929–1942 DOI: 10.1145/3531146.3533238
- Atasi Panda, Anand Louis and Prajakta Nimbhorkar “Bipartite Matchings with Group Fairness and Individual Fairness Constraints” In CoRR abs/2208.09951, 2022
- Evaggelia Pitoura, Kostas Stefanidis and Georgia Koutrika “Fairness in Rankings and Recommendations: An Overview” In The VLDB Journal, 2021 URL: https://doi.org/10.1007/s00778-021-00697-y
- Alexander Schrijver “Combinatorial Optimization: Polyhedra and Efficiency”, Algorithms and Combinatorics v. 1, 2, and 3 Springer, 2003 URL: https://books.google.com/books?id=mqGeSQ6dJycC
- “Fairness of Exposure in Rankings” In KDD ACM, 2018, pp. 2219–2228
- “Policy Learning for Fairness in Ranking” In NeurIPS, 2019, pp. 5427–5437
- Ashudeep Singh, David Kempe and Thorsten Joachims “Fairness in Ranking under Uncertainty” In Advances in Neural Information Processing Systems 34 Curran Associates, Inc., 2021, pp. 11896–11908 URL: https://proceedings.neurips.cc/paper/2021/file/63c3ddcc7b23daa1e42dc41f9a44a873-Paper.pdf
- Pavan Kumar C Singitham, Mahathi S Mahabhashyam and Prabhakar Raghavan “Efficiency-quality tradeoffs for vector score aggregation” In Proceedings of the Thirtieth international conference on Very large data bases-Volume 30, 2004, pp. 624–635
- “Optimisation Methods for Ranking Functions with Multiple Parameters” In Proceedings of the 15th ACM international conference on Information and knowledge management, 2006, pp. 585–593
- Dirk Johannes Thomas “Matching Problems with Additional Resource Constraints”, 2016 DOI: 10.25353/ubtr-xxxx-7644-a670/
- “Robust Optimization for Fairness with Noisy Protected Groups” In NeurIPS, 2020
- Jason Weston, Samy Bengio and Nicolas Usunier “Large Scale Image Annotation: Learning to Rank with Joint Word-Image Embeddings” In Machine learning 81.1 Springer, 2010, pp. 21–35
- Jialu Wang, Yang Liu and Caleb Levy “Fair Classification with Group-Dependent Label Noise” In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21 Virtual Event, Canada: Association for Computing Machinery, 2021, pp. 526–536 DOI: 10.1145/3442188.3445915
- Ke Yang, Vasilis Gkatzelis and Julia Stoyanovich “Balanced Ranking with Diversity Constraints” In IJCAI ijcai.org, 2019, pp. 6035–6042
- Ke Yang, Joshua R. Loftus and Julia Stoyanovich “Causal Intersectionality and Fair Ranking” In FORC 192, LIPIcs Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2021, pp. 7:1–7:20
- “Measuring Fairness in Ranked Outputs” In Proceedings of the 29th International Conference on Scientific and Statistical Database Management, Chicago, IL, USA, June 27-29, 2017 Chicago, IL, USA: ACM, 2017, pp. 22:1–22:6 DOI: 10.1145/3085504.3085526
- “Reducing Disparate Exposure in Ranking: A Learning To Rank Approach” In WWW ACM / IW3C2, 2020, pp. 2849–2855
- “FA*IR: A Fair Top-k Ranking Algorithm” In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17 Singapore, Singapore: Association for Computing Machinery, 2017, pp. 1569–1578 DOI: 10.1145/3132847.3132938
- Meike Zehlike, Ke Yang and Julia Stoyanovich “Fairness in Ranking, Part I: Score-Based Ranking” Just Accepted In ACM Comput. Surv. New York, NY, USA: Association for Computing Machinery, 2022 URL: https://doi.org/10.1145/3533379
- Meike Zehlike, Ke Yang and Julia Stoyanovich “Fairness in Ranking, Part II: Learning-to-Rank and Recommender Systems” Just Accepted In ACM Comput. Surv. New York, NY, USA: Association for Computing Machinery, 2022 DOI: 10.1145/3533380
- Sruthi Gorantla (9 papers)
- Anay Mehrotra (25 papers)
- Amit Deshpande (35 papers)
- Anand Louis (35 papers)