Papers
Topics
Authors
Recent
Assistant
AI Research Assistant
Well-researched responses based on relevant abstracts and paper content.
Custom Instructions Pro
Preferences or requirements that you'd like Emergent Mind to consider when generating responses.
Gemini 2.5 Flash
Gemini 2.5 Flash 189 tok/s
Gemini 2.5 Pro 53 tok/s Pro
GPT-5 Medium 36 tok/s Pro
GPT-5 High 36 tok/s Pro
GPT-4o 75 tok/s Pro
Kimi K2 160 tok/s Pro
GPT OSS 120B 443 tok/s Pro
Claude Sonnet 4.5 37 tok/s Pro
2000 character limit reached

An engine not a camera: Measuring performative power of online search (2405.19073v2)

Published 29 May 2024 in cs.CY and cs.IR

Abstract: The power of digital platforms is at the center of major ongoing policy and regulatory efforts. To advance existing debates, we designed and executed an experiment to measure the performative power of online search providers. Instantiated in our setting, performative power quantifies the ability of a search engine to steer web traffic by rearranging results. To operationalize this definition we developed a browser extension that performs unassuming randomized experiments in the background. These randomized experiments emulate updates to the search algorithm and identify the causal effect of different content arrangements on clicks. Analyzing tens of thousands of clicks, we discuss what our robust quantitative findings say about the power of online search engines, using the Google Shopping antitrust investigation as a case study. More broadly, we envision our work to serve as a blueprint for how the recent definition of performative power can help integrate quantitative insights from online experiments with future investigations into the economic power of digital platforms.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. When Should You Adjust Standard Errors for Clustering? The Quarterly Journal of Economics, 138(1):1–35, 2022.
  2. A general framework for counterfactual learning-to-rank. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, page 5–14, 2019.
  3. Location, location, location: An analysis of profitability of position in online advertising markets. Journal of Marketing Research, 48(6):1057–1073, 2011.
  4. Bidding to the top: Vcg and equilibria of position-based auctions. In Proceedings of the 4th International Conference on Approximation and Online Algorithms, page 15–28, 2006.
  5. Position Auctions with Consumer Search. The Quarterly Journal of Economics, 126(3):1213–1270, 2011.
  6. Jack Bandy. Problematic machine behavior: A systematic literature review of algorithm audits. Proc. ACM Hum.-Comput. Interact., 5, 2021.
  7. A game-theoretic approach to recommendation systems with strategic content providers. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, page 1118–1128, 2018.
  8. The EU digital markets act. Technical Report KJ-02-21-116-EN-N (online), 2021.
  9. Bias and debias in recommender system: A survey and future directions. ACM Trans. Inf. Syst., 41(3), 2023.
  10. Scalar is not enough: Vectorization-based unbiased learning to rank. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022.
  11. Click Models for Web Search. Springer Cham, 2015.
  12. CMA. Online search: Consumer and firm behavior - a review of the existing literature, 2017.
  13. An experimental comparison of click position-bias models. In Proceedings of the 2008 International Conference on Web Search and Data Mining, page 87–94, 2008.
  14. Competition Policy for the digital era: Final report. Publications Office of the European Union, 2019.
  15. Antitrust, amazon, and algorithmic auditing. Arxiv preprint arxiv:2403.18623, 2024.
  16. Visual positions of links and clicks on wikipedia. In Proceedings of the 25th International Conference Companion on World Wide Web, page 27–28, 2016.
  17. European Commission. Google shopping commission decision, 2017.
  18. R.A. Fisher. The design of experiments. 1935.
  19. Beyond rankings: Exploring the impact of serp features on organic click-through rates. Arxiv preprint arxiv:2306.01785, 2023.
  20. Jason Furman. Unlocking digital competition, report of the digital competition expert panel, 2019. https://doi.org/10.17639/wjcs-jc14.
  21. Google the gatekeeper: How search components affect clicks and attention. Proceedings of the International AAAI Conference on Web and Social Media, 17(1):245–256, 2023.
  22. The predictability of social events. Journal of Political Economy, 62(6):465–478, 1954.
  23. An eye tracking study of the effect of target rank on web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, page 417–420. Association for Computing Machinery, 2007.
  24. Performative prediction: Past and future. ArXiv preprint arXiv:2310.16608, 2023.
  25. Performative Power. In Advances in Neural Information Processing Systems, pages 22969–22981, 2022.
  26. Algorithmic collective action in machine learning. In Proceedings of the 40th International Conference on Machine Learning, page 12570–12586, 2023.
  27. Modeling content creator incentives on algorithm-curated platforms. In The Eleventh International Conference on Learning Representations, 2023.
  28. Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction. Cambridge University Press, 2015.
  29. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, page 154–161, 2005.
  30. Reactivity in measuring sensitive online behavior. Internet Research, 2022. ISSN 1066-2243.
  31. Donald MacKenzie. An engine, not a camera: How financial models shape markets. MIT Press, 2008.
  32. Auditing algorithms: Understanding algorithmic systems from the outside in. Foundations and Trends in Human–Computer Interaction, 14(4):272–344, 2021.
  33. Oskar Morgenstern. Wirtschaftsprognose: Eine Untersuchung ihrer Voraussetzungen und Möglichkeiten. Springer, 1928.
  34. Position effects in search advertising and their moderators: A regression discontinuity approach. Marketing Science, 34(3):388–407, 2015.
  35. Nathan Newman. Search, antitrust, and the economics of the control of user data. Yale J. on Reg., 31:401, 2014.
  36. NSF. The National Internet Observatory, 2021. https://nationalinternetobservatory.org.
  37. OECD. The evolving concept of market power in the digital economy, 2022. OECD Competition Policy Roundtable Background Note.
  38. In Google We Trust: Users’ Decisions on Rank, Position, and Relevance. Journal of Computer-Mediated Communication, 12(3):801–823, 2007.
  39. Performative prediction. In International Conference on Machine Learning (ICML), pages 7599–7609, 2020.
  40. Attribute-based propensity for unbiased learning in recommender systems: Algorithm and case studies. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 2359–2367, 2020.
  41. Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages. In Proceedings of the 27th International Web Conference , 2018.
  42. Donald B. Rubin. Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75(371):591–593, 1980.
  43. Methods for sample size determination in cluster randomized trials. International Journal of Epidemiology, 44(3):1051–1067, 2015.
  44. Herbert A. Simon. Bandwagon and underdog effects and the possibility of election predictions. The Public Opinion Quarterly, 18(3):245–253, 1954.
  45. Stigler Committee. Final report: Stigler committee on digital platforms, September 2019.
  46. The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society: Series A (Statistics in Society), 174(2):369–386, 2011.
  47. Chad Syverson. Macroeconomics and market power: Context, implications, and open questions. Journal of Economic Perspectives, 33(3):23–43, August 2019.
  48. Nudge: Improving Decisions about Health, Wealth, and Happiness. Yale University Press, 2008.
  49. Raluca M Ursu. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science, 37(4):530–552, 2018.
  50. Visual encoding and debiasing for ctr prediction. In Proceedings of the ACM International Conference on Information & Knowledge Management, page 4615–4619, 2022.
  51. Towards disentangling relevance and bias in unbiased learning to rank. In Proc. 29292929th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 5618–5627, 2023.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Dice Question Streamline Icon: https://streamlinehq.com

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
Lightbulb Streamline Icon: https://streamlinehq.com

Continue Learning

We haven't generated follow-up questions for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Generate Now
List To Do Tasks Checklist Streamline Icon: https://streamlinehq.com

Collections

Sign up for free to add this paper to one or more collections.

User Circle Single Streamline Icon: https://streamlinehq.com Sign Up
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets

This paper has been mentioned in 1 tweet and received 0 likes.

Upgrade to Pro to view all of the tweets about this paper:

Start a free 7-day Pro trial