Social-Annotate: Self-Healing Browser Extension to Annotate and Collect Social Media Data

Abstract: Human-annotated data remains foundational for machine learning and social media analysis. However, traditional data collection often relies on cumbersome pipelines that isolate content from its original source, compromising ecological validity. To address these challenges, we present Social-Annotate, a flexible browser extension that facilitates direct data collection on online platforms. By injecting customizable forms into webpages, the tool captures annotations while users interact with the native environment. Social-Annotate offers no-code design interface for the survey forms for non-technical users. Since injecting custom elements directly into host platforms creates a brittle dependency on evolving interfaces, we integrate a self-healing agent powered by LLMs. This automated pipeline autonomously detects structural changes, regenerates valid target selectors, and validates them within a live browser environment. Our extensible platform readily supports 12 platforms including social media like $\mathbb{X}$, Instagram, TikTok and P2P messaging platforms WhatsApp and Telegram. Social-Annotate significantly reduces data collection overhead and developer maintenance, enabling researchers of all technical backgrounds to focus on data analysis rather than engineering. Moreover, Social-Annotate provides an ecosystem for conducting intervention studies by dynamic content manipulation.

• The paper presents a novel browser extension, Social-Annotate, that enables in-context social media annotation with self-healing capabilities using LLM-driven selector extraction.
• The paper details a configurable, no-code interface and a cross-platform architecture that supports ecological validity and seamless integration into social media studies.
• The paper demonstrates robust performance by restoring data collection across platforms as DOM structures evolve, reducing maintenance and manual intervention.

Social-Annotate: Robust, In-Context Social Media Annotation via Self-Healing Browser Extension

Motivation and Context

Collection of ecologically valid, annotated social media data is critical for robust computational social science and machine learning applications, particularly in domains such as misinformation detection, hate speech moderation, and coordinated activity analysis. However, restrictive data access regimes (the so-called "APIcalypse") and dynamic front-end interfaces significantly undermine both the scalability and long-term reliability of standard data annotation tools. The conventional split between data presentation and annotation—often requiring external platforms and manual interaction—introduces substantial overhead and exacerbates bias by fragmenting the user experience away from its native context. Existing solutions are either brittle to platform updates or require considerable technical expertise for configuration and maintenance.

System Overview

The "Social-Annotate" framework (2607.01460) addresses these infrastructural and methodological challenges by introducing a general-purpose, extensible browser extension for direct in-situ annotation of social media content. The system is architected to facilitate no-code survey injection, supports high configurability, and crucially, incorporates a self-healing agent leveraging LLM-powered reasoning to autonomously adapt to shifts in platform DOM structures.

Figure 1: Platform design schematic for Social-Annotate, illustrating architectural modules and user interface aspects for extension configuration and operation.

The primary architectural components include:

• Platform-Aware Content Scripts: Customized for supported platforms, these scripts inject survey forms at user-defined DOM locations to maximize ecological validity.
• Modular Configuration Interface: Enables both non-technical and advanced users to tailor survey content, consent forms, export destinations (JSONL or API endpoints), and data manipulation logic through an options page.

  Figure 2: Options page with interfaces for detailed configuration of studies, informed consent, and live preview of survey forms, supporting flexible experimental designs.

• Content Manipulation Module: For interventional studies, Social-Annotate supports both dynamic API-driven and static file-based content rewriting, operating in either blind or aware experimental modes to control user awareness of intervention.

Self-Healing Agent: LLM-Induced Structural Robustness

A central innovation is the self-healing agent that ensures resilience to frequent and often opaque changes to social media platform interfaces. This agent employs the following pipeline:

1. Platform Abstraction: Typed DOM selector schemas and site-specific validation logic are encapsulated for supported targets.
2. LLM-Driven Selector Extraction: State-of-the-art LLMs (e.g., Claude, Gemini) parse current HTML snapshots to generate valid, structured selectors via constrained response schemas.
3. Browser-Level Validation: Using Playwright, proposed selectors are tested in a live Chromium instance, verifying not only injection but also form accessibility and correct metadata extraction, simulating realistic user submission workflows.

   Figure 3: Pipeline of the self-healing agent for dynamic repair of survey injection points, integrating validation, LLM extraction, browser testing, and operator-verified deployment.

The agent's efficacy is evaluated against historical and contemporary interface snapshots for platforms such as $\mathbb{X}$/Twitter and Bluesky, achieving full functional annotation and metadata extraction for all but one edge case (the absence of timestamp attributes in early $\mathbb{X}$/Twitter snapshots). The key quantitative claim is that the agent restored survey injection and data collection capability in all tested scenarios except those explicitly out of scope due to platform-specific client-side rendering or excessive DOM complexity.

Intervention and Cross-Platform Deployment

Annotation is further enriched by the intervention framework. For studies requiring content or UI manipulation, Social-Annotate provides both serverless (mapping file) and API-based pathways for substituting post/user data, enabling both opaque (blind) and transparent (aware) settings—vital for causal inference experiments in user perception and social influence.

Figure 4: Cross-platform deployment of the annotation workflow, demonstrating native survey injection across six social platforms while retaining original content dynamics.

Figure 5: Content and profile intervention instances, visualizing original, blind-manipulated, and aware-manipulated content to facilitate controlled behavioral studies.

The extension currently supports direct deployment and annotation spanning 12 major social and communication platforms, including both public and peer-to-peer messaging environments. This strategic generality allows for the unification of annotation protocols and content interventions at scale, overcoming fragmentation and maximizing dataset representativeness.

Theoretical and Practical Implications

The direct in-context annotation approach implemented in Social-Annotate addresses key methodological critiques regarding ecological validity and cognitive bias in data collection. By integrating seamlessly with native social media interfaces and minimizing the technical barrier to customization, the platform enables richer, context-sensitive annotation that is both scalable and maintainable by a wide spectrum of users, including non-programmers.

From a data engineering perspective, the LLM-based self-healing agent operationalizes a paradigm shift in web-based data tool maintenance. Rather than relying on brittle, hard-coded selector maintenance or constant manual intervention, the approach leverages the generalization abilities and code synthesis strengths of foundation models, bringing annotation pipelines closer to "fire-and-forget" robustness.

Looking ahead, further integration of in-browser, client-side LLMs promises to enable complex, privacy-preserving content transformations and metrics interventions without external data transfer, enhancing user privacy and regulatory alignment. As coding agents and on-device AI continue to mature, the technical overhead for extending annotation tools to new platforms or experimental protocols is expected to diminish substantially, moving closer to adaptive, fully autonomous instrumentation in web science.

Conclusion

Social-Annotate (2607.01460) introduces a formalized architecture to restore and expand the capabilities of in-context data annotation for social media research. By combining a highly configurable, cross-platform browser extension with a robust LLM-driven self-healing pipeline, the framework establishes a scalable foundation for ecologically valid annotation studies and controlled interventions. These contributions are positioned to both democratize data collection infrastructure and prepare the field for increasingly complex, AI-mediated research protocols.