AI Revolution on Chat Bot: Evidence from a Randomized Controlled Experiment (2401.10956v1)

Abstract: In recent years, generative AI has undergone major advancements, demonstrating significant promise in augmenting human productivity. Notably, LLMs (LLM), with ChatGPT-4 as an example, have drawn considerable attention. Numerous articles have examined the impact of LLM-based tools on human productivity in lab settings and designed tasks or in observational studies. Despite recent advances, field experiments applying LLM-based tools in realistic settings are limited. This paper presents the findings of a field randomized controlled trial assessing the effectiveness of LLM-based tools in providing unmonitored support services for information retrieval.