Towards Measuring Fairness in Grid Layout in Recommender Systems

Abstract: There has been significant research in the last five years on ensuring the providers of items in a recommender system are treated fairly, particularly in terms of the exposure the system provides to their work through its results. However, the metrics developed to date have all been designed and tested for linear ranked lists. It is unknown whether and how existing fair ranking metrics for linear layouts can be applied to grid-based displays. Moreover, depending on the device (phone, tab, or laptop) users use to interact with systems, column size is adjusted using column reduction approaches in a grid-view. The visibility or exposure of recommended items in grid layouts varies based on column sizes and column reduction approaches as well. In this paper, we extend existing fair ranking concepts and metrics to study provider-side group fairness in grid layouts, present an analysis of the behavior of these grid adaptations of fair ranking metrics, and study how their behavior changes across different grid ranking layout designs and geometries. We examine how fairness scores change with different ranking layouts to yield insights into (1) the consistency of fair ranking measurements across layouts; (2) whether rankings optimized for fairness in a linear ranking remain fair when the results are displayed in a grid; and (3) the impact of column reduction approaches to support different device geometries on fairness measurement. This work highlights the need to use layout-specific user attention models when measuring fairness of rankings, and provide practitioners with a first set of insights on what to expect when translating existing fair ranking metrics to the grid layouts in wide use today.

• The paper introduces adapted fairness measures for grid layouts, moving beyond traditional linear ranking assessments.
• It reveals that grid design and device-induced layout changes significantly influence recommendation exposure.
• The study highlights the need for tailored user attention models in recommender systems to ensure equitable content visibility.

Introduction

Recommender systems, which power most of the content we engage with on digital platforms, from streaming services to e-commerce websites, hold a significant impact on which items gain visibility and which languish in obscurity. At the heart of these systems lies an algorithm that selects and ranks content based on presumed user preference, simultaneously influencing the exposure that content creators receive. Amidst growing scrutiny, ensuring that these recommendation algorithms distribute exposure fairly has become a pressing concern.

Evaluating Fairness in Grid Layouts

Traditionally, fairness in recommendations has been assessed through linear rankings, where items are listed vertically, as if on a search engine results page. However, in real-world applications, content is just as likely to be arranged in grid layouts, responding dynamically to different devices with varying screen sizes. Implicit in this dynamic presentation is an assumption that has largely gone untested—that fairness metrics developed for linear layouts will hold up when applied to grids.

The study discussed here tackles this challenge head-on, attempting to bridge this gap. It aims to adapt existing fairness concepts and metrics to grid layouts, examining the reliability of these adapted metrics across various layout designs. Specifically, it evaluates the visibility and subsequent fairness of item exposure as influenced by the user's device, be it a phone, tablet, or desktop, which determines the grid's column size and format.

Fairness Measurement Insights

A cornerstone finding from this research is that the measures of fairness can greatly differ depending on layout design and geometry. This is substantial as it suggests that an algorithm appearing to distribute exposure fairly in a linear arrangement may fail to do so in a grid format or vice versa. The study delineates several types of grid layouts and introduces adaptations to three existing user browsing models, namely Row-Skipping and Slower-Decay, to enable fairness assessment in grid-based recommenders.

The inquiry extends to observing how variations in grid layouts due to changes in device screens impact these fairness evaluations. The analysis reveals that both the type of grid layout and the column adjustment approaches—for example, truncating off-screen items or re-wrapping them into new rows—affect the perceived fairness of recommendations.

Practical Implications for Providers

For practitioners, this research signals a crucial need to consider layout-specific user attention models when evaluating the fairness of their recommender systems. It underscores that a one-size-fits-all approach in fairness metrics does not translate well across different geometries and adaptive layouts. Furthermore, the study shows that different approaches to column size adjustments for various devices come with their own implications on the fairness measurements.

The implications of these findings are far-reaching. Content providers and platform designers must carefully consider how the display geometry interacts with the user browsing behavior, as this interplay significantly mediates the exposure an item gets. Establishing fair exposure becomes a multi-faceted problem where display layout, device type, and user interaction behavior all demand meticulous scrutiny.

Conclusion

In conclusion, this paper presents a groundbreaking step towards understanding and integrating fairness in grid layout recommender systems. It enlightens us on the complexities of evaluating fairness in non-linear display formats and calls for a tailored approach, taking into account the nuances of grid layouts and user attention patterns. Such insights are pivotal for developing fair recommender systems that serve both users and content providers with equity.