Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
184 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Improving Ego-Cluster for Network Effect Measurement (2308.05945v3)

Published 11 Aug 2023 in cs.SI and stat.ME

Abstract: The network effect, wherein one user's activity impacts another user, is common in social network platforms. Many new features in social networks are specifically designed to create a network effect, enhancing user engagement. For instance, content creators tend to produce more when their articles and posts receive positive feedback from followers. This paper discusses a new cluster-level experimentation methodology for measuring creator-side metrics in the context of A/B experiments. The methodology is designed to address cases where the experiment randomization unit and the metric measurement unit differ. It is a crucial part of LinkedIn's overall strategy to foster a robust creator community and ecosystem. The method is developed based on widely-cited research at LinkedIn but significantly improves the efficiency and flexibility of the clustering algorithm. This improvement results in a stronger capability for measuring creator-side metrics and an increased velocity for creator-related experiments.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (26)
  1. Designing and deploying online field experiments. In Proceedings of the 23rd international conference on World wide web (WWW ’14). Association for Computing Machinery, Seoul, Korea, 283–292. https://doi.org/10.1145/2566486.2567967
  2. Joan Fisher Box. 1980. R.A. Fisher and the Design of Experiments, 1922–1926. The American Statistician 34, 1 (Feb. 1980), 1–7. https://doi.org/10.1080/00031305.1980.10482701
  3. Nicholas Chamandy. 2016. Experimentation in a Ridesharing Marketplace. https://eng.lyft.com/experimentation-in-a-ridesharing-marketplace-b39db027a66e
  4. Automatic Detection and Diagnosis of Biased Online Experiments. arXiv:1808.00114 [stat] (July 2018). http://arxiv.org/abs/1808.00114 arXiv: 1808.00114.
  5. Improving the sensitivity of online controlled experiments by utilizing pre-experiment data. In Proceedings of the sixth ACM international conference on Web search and data mining - WSDM ’13. ACM Press, Rome, Italy, 123. https://doi.org/10.1145/2433396.2433413
  6. Shrinkage Estimators in Online Experiments. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining - KDD ’19 (2019), 2914–2922. https://doi.org/10.1145/3292500.3330771 arXiv: 1904.12918.
  7. Diagnosing Sample Ratio Mismatch in Online Controlled Experiments: A Taxonomy and Rules of Thumb for Practitioners. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining - KDD ’19. ACM Press, Anchorage, AK, USA, 2156–2164. https://doi.org/10.1145/3292500.3330722
  8. Network A/B Testing: From Sampling to Estimation. In Proceedings of the 24th International Conference on World Wide Web (Florence, Italy) (WWW ’15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 399–409. https://doi.org/10.1145/2736277.2741081
  9. Top Challenges from the first Practical Online Controlled Experiments Summit. ACM SIGKDD Explorations Newsletter 21, 1 (May 2019), 20–35. https://doi.org/10.1145/3331651.3331655
  10. Focusing on the Long-term: It’s Good for Users and Business. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’15. ACM Press, Sydney, NSW, Australia, 1849–1858. https://doi.org/10.1145/2783258.2788583
  11. Paul W Holland. 1986. Statistics and causal inference. Journal of the American statistical Association 81, 396 (1986), 945–960.
  12. Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
  13. Network Experimentation at Scale. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (Virtual Event, Singapore) (KDD ’21). Association for Computing Machinery, New York, NY, USA, 3106–3116. https://doi.org/10.1145/3447548.3467091
  14. David Kastelman and Raghav Ramesh. [n.d.]. Switchback Tests and Randomized Experimentation Under Network Effects at DoorDash | by DoorDash | Medium. https://medium.com/@DoorDash/switchback-tests-and-randomized-experimentation-under-network-effects-at-doordash-f1d938ab7c2a
  15. Online controlled experiments at large scale. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’13. ACM Press, Chicago, Illinois, USA, 1168. https://doi.org/10.1145/2487575.2488217
  16. Seven rules of thumb for web site experimenters. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’14. ACM Press, New York, New York, USA, 1857–1866. https://doi.org/10.1145/2623330.2623341
  17. Controlled experiments on the web: survey and practical guide. Data Mining and Knowledge Discovery 18, 1 (Feb. 2009), 140–181. https://doi.org/10.1007/s10618-008-0114-1
  18. Trustworthy and Powerful Online Marketplace Experimentation with Budget-split Design. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. ACM, Virtual Event Singapore, 3319–3329. https://doi.org/10.1145/3447548.3467193
  19. Near linear time algorithm to detect community structures in large-scale networks. Physical Review E 76, 3 (sep 2007). https://doi.org/10.1103/physreve.76.036106
  20. Donald B. Rubin. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology 66, 5 (1974), 688–701. https://doi.org/10.1037/h0037350
  21. Dmitriy Ryaboy. 2015. Twitter experimentation: technical overview. https://blog.twitter.com/engineering/en_us/a/2015/twitter-experimentation-technical-overview.html
  22. Using Ego-Clusters to Measure Network Effects at LinkedIn. https://doi.org/10.48550/ARXIV.1903.08755
  23. Detecting Network Effects: Randomizing Over Randomized Experiments. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’17). Association for Computing Machinery, New York, NY, USA, 1027–1035. https://doi.org/10.1145/3097983.3098192
  24. Ajit C. Tamhane. 2009. Statistical Analysis of Designed Experiments: Theory and Applications. John Wiley & Sons. Google-Books-ID: 5fHm_OWUQmQC.
  25. Overlapping experiment infrastructure: More, better, faster experimentation. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. 17–26.
  26. From Infrastructure to Culture: A/B Testing Challenges in Large Scale Social Networks. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’15. ACM Press, Sydney, NSW, Australia, 2227–2236. https://doi.org/10.1145/2783258.2788602

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now