Are Large Language Models Ready for Business Integration? A Study on Generative AI Adoption (2502.19423v1)

Abstract: The explorations and applications of AI in various domains becomes increasingly vital as it continues to evolve. While much attention has been focused on LLMs such as ChatGPT, this research examines the readiness of other LLMs such as Google Gemini (previously Google BARD), a conversational AI chatbot, for potential business applications. Gemini is an example of a Generative AI (Gen AI) that demonstrates capabilities encompassing content generation, language translation, and information retrieval. This study aims to assess its efficacy for text simplification in catering to the demands of modern businesses. A dataset of 42,654 reviews from distinct Disneyland branches was employed. The chatbot's API was utilised with a uniform prompt to generate simplified re-views. Results presented a spectrum of responses, including 75% successful simplifications, 25% errors, and instances of model self-reference. Quantitative analysis encompassing response categorisation, error prevalence, and response length distribution was conducted. Furthermore, NLP metrics were applied to gauge the quality of the generated content with the original reviews. The findings offer insights into Gen AI models performance, highlighting proficiency in simplifying re-views while unveiling certain limitations in coherence and consistency since only about 7.79% of the datasets was simplified. This research contributes to the ongoing discourse on AI adoption in business contexts. The study's out-comes provide implications for future development and implementation of AI-driven tools in businesses seeking to enhance content creation and communication processes. As AI continues to transform industries, an understanding of the readiness and limitations of AI models is essential for informed decision-making, automations and effective integration.