An Empirical Investigation of Personalization Factors on TikTok

Abstract: TikTok currently is the fastest growing social media platform with over 1 billion active monthly users of which the majority is from generation Z. Arguably, its most important success driver is its recommendation system. Despite the importance of TikTok's algorithm to the platform's success and content distribution, little work has been done on the empirical analysis of the algorithm. Our work lays the foundation to fill this research gap. Using a sock-puppet audit methodology with a custom algorithm developed by us, we tested and analysed the effect of the language and location used to access TikTok, follow- and like-feature, as well as how the recommended content changes as a user watches certain posts longer than others. We provide evidence that all the tested factors influence the content recommended to TikTok users. Further, we identified that the follow-feature has the strongest influence, followed by the like-feature and video view rate. We also discuss the implications of our findings in the context of the formation of filter bubbles on TikTok and the proliferation of problematic content.

• The paper demonstrates that following behavior significantly influences TikTok’s recommendation system.
• It employs sock-puppet audits and controlled experiments with over 30,000 posts to quantify the impact of likes, follows, and video view rates.
• Results indicate that location and explicit user interactions enhance content personalization, raising concerns about filter bubbles.

An Empirical Investigation of Personalization Factors on TikTok

Overview and Motivation

The paper "An Empirical Investigation of Personalization Factors on TikTok" conducts an extensive examination of TikTok's recommendation system (RS) to uncover how different personalization factors affect content distribution on users' "For You" pages. It employs a sock-puppet audit methodology to determine the influence of explicit user actions— such as liking, following, and video viewing preferences— as well as the impact of location and language settings. The study is driven by the increasing reliance on algorithmic-driven content distribution on platforms like TikTok, which has significant implications for societal discourse, the creation of filter bubbles, and the spread of problematic content.

Methodology

Data Collection Approach

The research utilizes a custom web-based bot to mimic user behavior and isolate personalized factors impacting TikTok's RS. Virtual agent-based auditing, or "sock-puppet" auditing, effectively creates a controlled environment to test different user characteristics or actions while measuring their influence on the recommended content. The bots, running in incognito mode using Selenium ChromeDriver, interacted with the platform, executed scripted actions, and collected data across different setups:

• Scenarios: Each experimental group tested specific factors, such as liking random posts, following specific content creators, or setting different language and location preferences.
• Dataset: The final dataset comprised over 30,000 unique posts, 34,905 distinct hashtags, 21,278 creators, and 20,302 sounds.

Experimental Scenarios

1. Like-Feature: Various strategies for liking posts—random, hashtag-based personas—were used to test its influence compared to scenarios with passive behavior.
2. Follow-Feature: Control scenarios emphasized following random content creators to determine its effect.
3. Video View Rate (VVR): Simulations varied the percentage of video watch time to identify correlations with RS behavior similar to known metrics in platforms like YouTube.
4. Language and Location: Experiments isolated the effects of user location and language settings on the recommendation system.

Figure 1: Difference of feeds per test run for test scenario 7 before accounting for drops.

Results and Analysis

Influence of Personalization Factors

• Following Content Creators: Demonstrated the strongest influence on the RS, indicating that explicit content engagement profoundly affects personalization.
• Video View Rate: While impactful, differences were marginally higher than likes, implying the RS places significant emphasis on inferred user interest through viewing habits.
• Liking Behavior: Strategic likes based on defined personas exhibited noticeable shifts in feed similarity, underscoring the RS's responsiveness to explicit user preferences.

Location and Language

The analysis indicated that location strongly influences RS behavior, with language settings exerting less impact. Users accessing TikTok from a dominant language area experienced more significant alignment with content typical of that region.

Figure 2: Results of test scenario 12.

Discussion

The study emphasizes the nuances of user control over TikTok's RS through explicit actions but also highlights the reduced transparency in quantifying viewing actions, which cannot be undone unlike follows or likes. The findings suggest a need for more user agency and transparency in how interests are inferred and reflected in RS behavior. This is crucial given the risks of filter bubbles and exposure to extremist content.

Conclusion

This investigation lays foundational knowledge for understanding TikTok's RS, emphasizing the strong influence of certain user actions on content personalization. It calls for future research into additional user factors affecting algorithmic content presentation and advocates for enhanced transparency to empower users in managing their digital experience on the platform.