ChatGPT Plugin Shop Ecosystem

• ChatGPT Plugin Shop is a marketplace enabling API integration and custom GPT development for dynamic access to external services.
• It operates via a manifest-driven, API-mediated model that uses metadata to facilitate secure and efficient external API calls.
• Empirical studies reveal significant security and compliance challenges, highlighting the need for improved governance and metadata consistency.

The ChatGPT Plugin Shop denotes the marketplace layer through which OpenAI exposed third-party extensions for ChatGPT, first through the plugin store and later through the GPT Store for custom GPTs. In the plugin-store form, the system centered on API-backed extensions invoked by ChatGPT during conversation; in the GPT Store form, it centered on custom GPTs configured through instructions, knowledge, capabilities, and optional Actions that connect to external services. Across both forms, the platform functions as an LLM-centered app ecosystem in which conversational interaction is coupled to external APIs, metadata, legal disclosures, and marketplace discovery mechanisms (Yan et al., 2024).

1. Platform concept and historical forms

The ecosystem has been described in two closely related but distinct platform forms. The earlier form was the OpenAI plugin store, where ChatGPT could invoke third-party plugins distributed through an official marketplace. The later form was the GPT Store, where creators could publish custom GPTs built on top of ChatGPT through prompt-tuning, reference resources, browsing, DALL·E image generation, data analysis, and Actions for external API interactions (Zhao et al., 2024).

In the GPT Store framing, GPTs are treated as custom applications that can be built without coding and described through seven metadata fields: name, description, instructions, conversation starters, knowledge, capabilities, and actions. The decisive field for third-party integration is Actions, because actions are the mechanism by which a GPT reaches out to external systems, fetches live data, executes tasks, or processes user data through outside APIs (Yan et al., 18 Jun 2025). The paper on GPTs-ThirdSpy formalizes three metadata-based GPT classes:

$$g \in \mathcal{G}_p \Leftrightarrow \{knowledge\} \notin g \land \{actions\} \notin g$$

$$g \in \mathcal{G}_k \Leftrightarrow \{knowledge\} \in g \land \{actions\} \notin g$$

$$g \in \mathcal{G}_a \Leftrightarrow \{actions\} \in g$$

Only action-based GPTs are directly relevant to third-party service integration in that formulation (Yan et al., 18 Jun 2025).

The marketplace analogy is explicit in several studies. One paper portrays the GPT Store as an early “app store” for customized LLMs, while another treats the plugin ecosystem as an LLM-centered app platform in which OpenAI mediates between users and third-party developers (Zhao et al., 2024, Yan et al., 2024). A plausible implication is that “ChatGPT Plugin Shop” is best understood not as a single immutable product name but as a broader ecosystem concept spanning the plugin store, the GPT Store, and the surrounding third-party indexing and auditing infrastructure.

2. Integration architecture and execution model

The plugin-store architecture was manifest-driven, API-mediated, and LLM-orchestrated. To be listed, a plugin had to provide a manifest file hosted under the plugin’s API domain at /.well-known/ai-plugin.json. This manifest acted as the required registration and configuration artifact for OpenAI and contained metadata such as plugin name, logo, description, legal-document URL, API endpoint(s), developer email, and authentication settings or token-related data (Yan et al., 2024).

That paper distinguishes a broader set of manifest fields required by OpenAI, denoted $O_r$, from the subset exposed to users, denoted $O_p \subseteq O_r$, with hidden fields $O_h = O_r - O_p$ (Yan et al., 2024). The runtime flow was reconstructed as follows: the user enters a prompt, ChatGPT matches the prompt against plugin descriptions in the manifest, chooses the appropriate plugin, extracts relevant parameters, sends an API request to the plugin server, receives JSON, and converts the JSON into a natural-language answer. The integration pattern is therefore not app embedding in the conventional UI sense, but LLM-as-router plus third-party API execution (Yan et al., 2024).

The GPT Store retained the external-service layer, but embedded it in the GPT abstraction. In the GPTs-ThirdSpy account, a GPT is identified by a unique 9-character alphanumeric identifier or GizmoID, and privacy settings exposed through the GPT interaction page indicate whether the GPT uses Actions and therefore external APIs. The automation pipeline for extracting third-party service data proceeds by identifying the GPT by GizmoID, opening the interaction page, clicking the dropdown menu, locating the “Privacy settings” button, extracting third-party domain entries and privacy-policy links, fetching and parsing privacy-policy pages, and storing the structured results in a dataset (Yan et al., 18 Jun 2025).

A further architectural perspective appears in the security work on the plugin shop, which models a plugin as

$p : (u, m, a)$

where $u \in \mathcal{U}$ denotes user-visible metadata, $m \in \mathcal{M}$ manifest data, and $a \in \mathcal{A}$ APIs and their response data (Yan et al., 2024). This representation makes explicit that the ecosystem’s trust boundary is distributed across store-visible metadata, configuration artifacts, and externally hosted services rather than confined to the LLM itself.

3. Scale, distribution, and marketplace structure

Large-scale measurements depict the ecosystem as extensive but highly skewed. GPTZoo reports 730,420 GPT instances after cleaning and deduplication, standardized in JSON and annotated with 21 metadata attributes spanning basic information and functional details, market feedback and user interaction, and development resources (Hou et al., 2024). The same paper notes that the official OpenAI GPT Store reportedly contains more than 3,000,000 GPTs, while the public interface supports keyword search and exposes only limited metadata such as creator, description, ratings, number of conversations, conversation starters, and capabilities (Hou et al., 2024).

Usage is strongly concentrated. GPTZoo reports 414,720 GPTs with 0 chats, 262,473 GPTs with 10 chats, 756 GPTs with over 50,000 chats, 2,100 GPTs with over 100,000 chats, 1,039 GPTs with over 500,000 chats, 5,026 GPTs with over 1,000,000 chats, and 24 GPTs with over 5,000,000 chats (Hou et al., 2024). “GPTs Window Shopping” similarly finds that only 28.5% of custom GPTs had more than 10 conversations, and that the top 12 GPTs in the store explorer page represented 0.36% of recorded GPTs while accounting for about 33.7% of all conversations (Zhao et al., 2024). This suggests pronounced winner-take-most dynamics in discovery and engagement.

The plugin-store dataset analyzed in the security study contains 1,038 plugins collected over the last four months of the store’s life, from after new registrations stopped in November 2023 until the store closed in April 2024 (Yan et al., 2024). That ecosystem was classified into 21 categories, with the most populated being Data Research (12.9%), Tools (11.2%), Business (10.1%), Developer Code (9.7%), Entertainment (6.7%), and Image Video (6.0%), while Law accounted for 0.8% (Yan et al., 2024). The same study reports that 417 of those plugins still existed as GPTs in the newer GPT Store, 70 corresponded to plugins that had previously leaked manifests, and 41 still had externally accessible APIs after migration (Yan et al., 2024).

The GPT landscape also exhibits sparse marketplace signals. “GPTs Window Shopping” reports an average rating of 4.13 among rated GPTs, but 90.78% of GPTs had no ratings at all (Zhao et al., 2024). Categories were weakly used: None accounted for 152,539 GPTs (45.62%), Other for 88,276 (26.40%), while Programming represented 2.48% (Zhao et al., 2024). English dominated description and conversation-starter language with 289,562 GPTs (86.605%), followed by Spanish, Japanese, Chinese, French, Portuguese, Korean, and German in much smaller shares (Zhao et al., 2024).

4. Developer-facing functionality and software-engineering use

Research on actual developer interactions shows that the ecosystem is not limited to isolated chatbot use. DevGPT is a dataset of shared developer–ChatGPT conversations linked to software-development artifacts including source code files, commits, issues, pull requests, GitHub discussions, and Hacker News threads (Xiao et al., 2023). In its 20231012 snapshot, DevGPT contains 4,733 shared ChatGPT links drawn from 3,559 GitHub or Hacker News references; after deduplication this corresponds to 3,794 unique shared links, because 940 links were referenced from multiple sources. Those links expand into 29,778 prompt/answer pairs and 19,106 code snippets (Xiao et al., 2023).

The largest artifact category in DevGPT is GitHub code files, with 1,843 files mentioning shared links in code comments; these correspond to 2,708 shared links, 2,540 accessible links, 1,184 conversations with code, 22,799 prompts, and 14,132 code snippets. Other categories include GitHub commits, GitHub issues, GitHub pull requests, Hacker News, and GitHub discussions. Commits account for 694 files/messages and 694 shared links, with 1,922 prompts and 1,828 code snippets (Xiao et al., 2023). The paper emphasizes that developers use ChatGPT in situated, task-driven ways connected to debugging, maintenance, problem solving, and workflow support rather than only for de novo code generation.

The code-bearing portion of the developer ecosystem is linguistically concentrated. DevGPT reports 6,084 Python snippets, 4,802 JavaScript snippets, and 4,332 Bash snippets as the three most frequent languages (Xiao et al., 2023). The paper does not define a new metric for ChatGPT quality or developer success; its methodological contribution is the construction of a traceable conversational dataset through repeated harvesting of shared ChatGPT URLs, preservation of access consistency, extraction of prompt/response pairs and code snippets, and linkage back to the referencing artifact and context (Xiao et al., 2023).

A related concern is whether ChatGPT can serve as a reliable recommender of software dependencies in a plugin- or library-oriented workflow. “Is ChatGPT a Good Software Librarian?” evaluates GPT-3.5 Turbo on 10,000 Python Stack Overflow questions and extracts 11,136 library occurrences and 764 unique libraries from generated code (Latendresse et al., 2024). ChatGPT recommended third-party libraries in 48.3% of occurrences, compared with 38.7% in human-written accepted answers, a difference of 9.6 percentage points (Latendresse et al., 2024). It favored mature and widely adopted libraries, with median forks 98 versus 7, median stars 430 versus 23.5, median dependents 98 versus 14, median SourceRank 22 versus 12, median age 159 months versus 125.1 months, and median version frequency 0.24 versus 0.16 (Latendresse et al., 2024).

The same study identifies substantive operational and legal risks for recommendation-driven ecosystems. Among ChatGPT’s library recommendations, 14.2% had copyleft licenses and 10.4% had unspecified licenses, while ChatGPT did not explicitly communicate license restrictions (Latendresse et al., 2024). In addition, 6.5% of recommendations—730 occurrences out of 11,136—did not work out of the box. The breakdown is Hard-coded 33.0% (241), Alias 30.5% (223), Module 16.3% (119), Placeholder 14.8% (108), Deprecated 1.4% (10), Mistake 1.0% (7), and Other 3.0% (22) (Latendresse et al., 2024). Within a ChatGPT Plugin Shop context, these results indicate that recommendation quality depends not only on model output but also on registry validation, licensing visibility, and dependency hygiene.

5. Privacy, compliance, and third-party service disclosure

The GPT Store’s third-party service layer has been studied most directly through GPTs-ThirdSpy, an automated framework designed to extract privacy settings of GPTs from the official GPT Store (Yan et al., 18 Jun 2025). The paper argues that existing third-party metadata sources are stale or incomplete and that real-time extraction from the official store is essential for trustworthy analysis of data sharing, compliance, and security risk. Traditional browser automation tools such as Selenium and Puppeteer were reported to fail against the store’s anti-automation mechanisms and dynamic loading, so the framework uses AppleScript plus cliclick on macOS to simulate coordinate-based clicks (Yan et al., 18 Jun 2025).

On a sample of the 500 most popular GPTs from GPTsHunter, GPTs-ThirdSpy finds 109 GPTs using third-party services. Among the 500 GPTs, 409 use no third-party services, 79 use exactly one third-party service, and 12 use two or more services (Yan et al., 18 Jun 2025). The paper interprets this as evidence that most GPTs are self-contained, but a meaningful minority rely on external APIs for real-time data, specialized computation, or cross-platform functionality. A plausible implication is that the external-service subset constitutes the principal compliance and security surface of the GPT Store.

The privacy-policy results are especially revealing. Of the 109 GPTs using third-party services, 92 privacy-policy links are accessible, 9 are broken links, 5 point only to the service homepage rather than a dedicated privacy policy, 2 time out, and 2 return server errors (Yan et al., 18 Jun 2025). One noted case used a placeholder privacy link such as https://app.example.com/privacy_policy (Yan et al., 18 Jun 2025). The paper argues that this reveals a gap between formal platform requirements and practical compliance: a disclosure field exists, but validity and content quality do not appear to be enforced consistently.

A comparable pattern appears in the plugin-store security study. That work reports 767 legal-document links accessible programmatically, 649 of which contained legal attributes, but 271 were inaccessible or unusable; among accessible legal documents, 391 were terms of service, 116 privacy policies, and 142 other legal documents (Yan et al., 2024). It also notes that many links pointed to company homepages, GitHub repositories, portfolio sites, or unrelated pages (Yan et al., 2024). Across both generations of the ecosystem, the recurring issue is not merely whether a legal or privacy URL exists, but whether it actually supplies relevant, accessible, and reviewable disclosure.

6. Security vulnerabilities, governance, and research infrastructure

The strongest empirical finding across the literature is that the ecosystem’s security posture has been weak relative to its attack surface. The plugin-store study “Exploring ChatGPT App Ecosystem: Distribution, Deployment and Security” defines five exposure types: non-empty manifest leakage, metadata discrepancies, single-auth external API calls, multi-auth external API calls, and token leakage (Yan et al., 2024). It finds 368 plugins leaking manifest files, 69 plugins with inconsistent metadata, 173 plugins with broken access control vulnerabilities in their APIs, and 271 plugins with inaccessible legal-document links (Yan et al., 2024). Sensitive developer credentials including API keys, API locations, and OAuth tokens were reported as leaked in some cases (Yan et al., 2024).

The 2025 security study of the ChatGPT plugin system reaches closely aligned conclusions. It reports collecting 1033 ChatGPT plugins, finding 373 valid manifest files through probing, together with 104 hidden redirects, 19 GitHub-based inaccessible addresses, 12 OpenAI-protected addresses, 6 Google Doc links, and 518 native URLs unreachable (Ren, 21 Jul 2025). Its three-layer model comprises a Manifest Analysis Layer, API Request Analysis Layer, and Consistency and Integrity Analysis Layer, and it formalizes manifest exposure as

$$g \in \mathcal{G}_k \Leftrightarrow \{knowledge\} \in g \land \{actions\} \notin g$$0

with protected files

$$g \in \mathcal{G}_k \Leftrightarrow \{knowledge\} \in g \land \{actions\} \notin g$$1

(Ren, 21 Jul 2025).

The API findings indicate repeated authorization weaknesses. Among the 373 plugins with leaked manifests, the study reports five request cases: Case 1: 8 plugins where a token was required, a valid token was used, and the request succeeded; Case 2: 74 where a token was required and the request failed appropriately; Case 3: 24 where a token was required but an invalid token still succeeded; Case 4: 141 where no token was required and the request succeeded; and Case 5: 98 where no token was required and the request failed (Ren, 21 Jul 2025). It also states that 173 plugins returned valid responses to external probing and 172 failed under the probe conditions, with failures attributed to 58 lack authorization, 66 client errors, and 49 rate limiting (Ren, 21 Jul 2025). The paper interprets no-token APIs as the most common and most exposed configuration.

Metadata integrity problems were also nontrivial. The 2025 study found 69 out of 373 explorable manifest files with consistency problems: 34 inconsistent names, 8 different descriptions, and 27 mismatched legal-document URLs (Ren, 21 Jul 2025). Examples include the use of “MixerBox OnePlayer” for up to 17 different plugins and name stuffing such as “Digital Pet” → “A Digital Pet” to improve ranking position (Ren, 21 Jul 2025). The earlier plugin-store study similarly reported 34 name inconsistencies, 8 description inconsistencies, and 27 legal-document URL mismatches (Yan et al., 2024). These patterns support the view that marketplace governance requires not only API hardening but also stronger controls on naming, disclosure, and metadata fidelity.

Some remediation occurred after disclosure. The plugin-store paper reports that after reporting to OpenAI and revisiting on April 9, 2024, manifest leaks dropped from 368 to 282, metadata inconsistencies from 69 to 61, single-auth accessible APIs from 141 to 89, multi-auth accessible APIs from 24 to 17, and token-accessible APIs from 8 to 5 (Yan et al., 2024). The later security study reports comparable reductions: file leaks 368 → 282, data inconsistencies 69 → 61, no-auth API leaks 141 → 89, OAuth-related leaks 27 → 17, and Bearer-token leaks 5 → 3 (Ren, 21 Jul 2025). These results indicate that platform intervention can reduce exposure, but they do not suggest full remediation.

The ecosystem is now accompanied by a growing research infrastructure for measurement. GPTZoo provides large-scale metadata, partial extraction of instructions, knowledge files, and third-party services, and a CLI for keyword-based search and analysis over more than 730k GPT records (Hou et al., 2024). GPTs-ThirdSpy provides real-time extraction of third-party service and privacy-policy information from the official GPT Store (Yan et al., 18 Jun 2025). DevGPT provides traceable developer–ChatGPT conversations anchored to software artifacts (Xiao et al., 2023). Taken together, these resources establish the ChatGPT Plugin Shop as a measurable software ecosystem characterized by centralized discovery, heterogeneous customization, external-service dependencies, sparse marketplace signals, and persistent security and compliance challenges.