Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

A First Look at GPT Apps: Landscape and Vulnerability (2402.15105v3)

Published 23 Feb 2024 in cs.CR and cs.CL

Abstract: Following OpenAI's introduction of GPTs, a surge in GPT apps has led to the launch of dedicated LLM app stores. Nevertheless, given its debut, there is a lack of sufficient understanding of this new ecosystem. To fill this gap, this paper presents a first comprehensive longitudinal (5-month) study of the evolution, landscape, and vulnerability of the emerging LLM app ecosystem, focusing on two GPT app stores: \textit{GPTStore.AI} and the official \textit{OpenAI GPT Store}. Specifically, we develop two automated tools and a TriLevel configuration extraction strategy to efficiently gather metadata (\ie names, creators, descriptions, \etc) and user feedback for all GPT apps across these two stores, as well as configurations (\ie system prompts, knowledge files, and APIs) for the top 10,000 popular apps. Our extensive analysis reveals: (1) the user enthusiasm for GPT apps consistently rises, whereas creator interest plateaus within three months of GPTs' launch; (2) nearly 90\% system prompts can be easily accessed due to widespread failure to secure GPT app configurations, leading to considerable plagiarism and duplication among apps. Our findings highlight the necessity of enhancing the LLM app ecosystem by the app stores, creators, and users.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (35)
  1. 2023. Can ai really be protected from text-based attacks? https://techcrunch.com/2023/02/24/can-language-models-really-be-protected-from-text-based-attacks/.
  2. 2023. Reverse-engineering gpts for fun and data. https://andrei.fyi/blog/reverse-engineering-gpts/.
  3. Adversa.AI. 2023. Llm red teaming gpt’s: Prompt leaking, api leaking, documents leaking. https://adversa.ai/blog/llm-red-teaming-gpts-prompt-leaking-api-leaking-documents-leaking/.
  4. Alfred V. Aho and Jeffrey D. Ullman. 1972. The Theory of Parsing, Translation and Compiling, volume 1. Prentice-Hall, Englewood Cliffs, NJ.
  5. Same app, different app stores: A comparative study. In 2017 IEEE/ACM 4th International Conference on Mobile Software Engineering and Systems (MOBILESoft), pages 79–90. IEEE.
  6. Aspectdroid: Android app analysis system. In Proceedings of the Sixth ACM Conference on Data and Application Security and Privacy, pages 145–147.
  7. American Psychological Association. 1983. Publications Manual. American Psychological Association, Washington, DC.
  8. Rie Kubota Ando and Tong Zhang. 2005. A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6:1817–1853.
  9. Galen Andrew and Jianfeng Gao. 2007. Scalable training of L1subscript𝐿1L_{1}italic_L start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT-regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, pages 33–40.
  10. Paolo Calciati and Alessandra Gorla. 2017. How do apps evolve in their permission requests? a preliminary study. In 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pages 37–41. IEEE.
  11. Bogdan Carbunar and Rahul Potharaju. 2015. A longitudinal study of the google app market. In Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, pages 242–249.
  12. Alternation. Journal of the Association for Computing Machinery, 28(1):114–133.
  13. Ar-miner: mining informative reviews for developers from mobile app marketplace. In Proceedings of the 36th international conference on software engineering, pages 767–778.
  14. James W. Cooley and John W. Tukey. 1965. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 19(90):297–301.
  15. Madfraud: Investigating ad fraud in android applications. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services, pages 123–134.
  16. Privacy pass: Bypassing internet challenges anonymously. Proc. Priv. Enhancing Technol., 2018(3):164–180.
  17. Androidleaks: Automatically detecting potential privacy leaks in android applications on a large scale. In Trust and Trustworthy Computing: 5th International Conference, TRUST 2012, Vienna, Austria, June 13-15, 2012. Proceedings 5, pages 291–307. Springer.
  18. Dan Gusfield. 1997. Algorithms on Strings, Trees and Sequences. Cambridge University Press, Cambridge, UK.
  19. Automated document classification for news article in bahasa indonesia based on term frequency inverse document frequency (tf-idf) approach. In 2014 6th international conference on information technology and electrical engineering (ICITEE), pages 1–4. IEEE.
  20. Androzoo++: Collecting millions of android apps and their metadata for the research community. arXiv preprint arXiv:1709.05281.
  21. A measurement-based study on application popularity in android and ios app stores. In Proceedings of the 2015 Workshop on Mobile Big Data, pages 13–18.
  22. Prompt injection attack against llm-integrated applications. arXiv preprint arXiv:2306.05499.
  23. A review on selenium web driver with python. Annals of the Romanian Society for Cell Biology, pages 16760–16768.
  24. Measurement, modeling, and analysis of the mobile app ecosystem. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 2(2):1–33.
  25. Semantic cosine similarity. In The 7th international student conference on advanced science and technology ICAST, volume 4, page 1.
  26. Mohammad Sadegh Rasooli and Joel R. Tetreault. 2015. Yara parser: A fast and accurate dependency parser. Computing Research Repository, arXiv:1503.06733. Version 2.
  27. Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
  28. Opening a pandora’s box: Things you should know in the era of custom gpts. arXiv preprint arXiv:2401.00905.
  29. Vincent F Taylor and Ivan Martinovic. 2017. To update or not to update: Insights from a two-year study of android app evolution. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pages 45–57.
  30. A measurement study of google play. In The 2014 ACM international conference on Measurement and modeling of computer systems, pages 221–233.
  31. Understanding the evolution of mobile app ecosystems: A longitudinal measurement study of google play. In The World Wide Web Conference, pages 1988–1999.
  32. An explorative study of the mobile app ecosystem from app developers’ perspective. In Proceedings of the 26th international conference on World Wide Web, pages 163–172.
  33. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
  34. Assessing prompt injection risks in 200+ custom gpts. arXiv preprint arXiv:2311.11538.
  35. A measurement study of wechat mini-apps. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 5(2):1–25.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Zejun Zhang (7 papers)
  2. Li Zhang (693 papers)
  3. Xin Yuan (198 papers)
  4. Anlan Zhang (4 papers)
  5. Mengwei Xu (62 papers)
  6. Feng Qian (41 papers)
Citations (10)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now