From Searchable to Non-Searchable: Generative AI and Information Diversity in Online Information Seeking

Abstract: Conversational generative AI systems such as ChatGPT are transforming how people seek and engage with information online. Unlike traditional search engines, these systems support open-ended, conversational inquiry, yet it remains unclear whether they ultimately expand or constrain the diversity of knowledge that users encounter in online search spaces, a primary foundation for knowledge work, learning, and innovation. Using over 200,000 real-world human-ChatGPT interactions, we examine how generative-AI-mediated inquiry reshapes diversity in both user inputs and system outputs through the lens of searchability - whether queries could plausibly be answered by traditional search engines. We find that almost 80% of ChatGPT user queries are non-searchable and span a broader knowledge space and topics than searchable queries, indicating expanded modes of inquiry. However, for comparable searchable queries, AI responses are less diverse than Google search results in the majority of topics. Moreover, the diversity of AI responses predicts subsequent changes in users' inquiry diversity, revealing a feedback loop between AI outputs and human exploration. These findings highlight a tension between expanded inquiry and constrained information exposure, with implications for designing hybrid search and generative-AI systems that better support exploratory knowledge seeking.

• The paper demonstrates that 79% of ChatGPT queries are non-searchable and occupy a broader semantic space than searchable queries.
• It employs a large dataset and embedding-based metrics to compare output diversity, showing that ChatGPT delivers less varied responses than Google for factual queries.
• Temporal analysis reveals a feedback loop where reduced system output diversity constrains subsequent user query diversity over conversation turns.

Generative AI, Searchability, and the Dynamics of Information Diversity in Online Inquiry

Introduction

The paper "From Searchable to Non-Searchable: Generative AI and Information Diversity in Online Information Seeking" (2604.10258) presents a rigorous empirical investigation into how conversational generative AI systems such as ChatGPT are reconfiguring user inquiry and response diversity compared to traditional search engines. Utilizing a corpus of over 200,000 real-world ChatGPT conversations, the authors operationalize the notion of "searchability"—whether a query could plausibly be answered by conventional web search—and analyze both the breadth of user inquiries and the diversity of system-generated outputs. Through comparative and temporal analyses, the study delineates the emergent tension: while generative AI expands the spectrum of user queries, it tends to deliver less diverse informational outputs for comparable, searchable queries, with measurable feedback effects on subsequent user exploration.

Empirical Setting and Methodology

The core of the study draws from the WildChat-4.8M dataset, leveraging over 230k ChatGPT conversations from 2023-2025, stratified temporally to ensure representative sampling. User queries are classified as searchable or non-searchable via a combination of LLM-assisted pre-labeling, human annotation, and supervised text classification, with validated accuracy above 90%. The diversity of information—input and output—is quantified as the extent of dispersion in SentenceTransformer-based semantic embedding spaces, aligning with established precedents for measuring knowledge breadth in the sciences.

Figure 1: Diversity and temporal trends in user input searchability, including semantic space coverage (A), topic prevalence (B), and weekly shifts in the proportion of searchable queries (C).

Main Findings

Expansion of Inquiry Scope

A principal finding is that the overwhelming majority (79%) of ChatGPT user queries are non-searchable, denoting information needs that are not effectively addressed by conventional search engines. The diversity metric (embedding-space radius) demonstrates that non-searchable queries occupy a significantly larger semantic space, with a maximum diversity score of 1.0826 compared to 1.0421 for searchable ones. Only four out of twenty-four high-level topics show dominance among searchable queries; non-searchable usage is highly prevalent for generative, creative, and dialogic tasks such as image creation, fiction, and role-play. Moreover, the longitudinal analysis reveals a declining fraction of searchable queries over time, suggesting a normative shift in user information-seeking behaviors.

Contraction of Output Diversity for Searchable Queries

Despite an expansion of the inquiry spectrum, ChatGPT's responses to comparable, searchable queries are less diverse than those returned by Google Search. Across topics, Google Top-5 results systematically present higher coverage within semantic space (mean diversity: 1.052 for Google vs. 1.027 for ChatGPT, p < 0.001). This contraction of output diversity is robust to quantile-based sensitivity checks, indicating persistent narrowing across the majority of topics. A striking exception is observed in creative domains, where ChatGPT exceeds Google in output diversity (1.190 vs. 1.114, p < 0.001).

Figure 2: Diversity of responses for identical searchable queries, comparing ChatGPT answers and the top five Google results, with topic-wise effect sizes.

Temporal Dynamics and Feedback Effects

The paper further substantiates a feedback loop between system output diversity and user exploratory behavior. Multi-turn conversation analysis shows a monotonic decrease in input diversity across turns (4.6% drop from turn 0 to 70), attributed to the increasing contextual conditioning on prior system responses. Regression analyses establish a statistically significant link: a one standard deviation increment in output diversity predicts a 0.5% increase in the diversity of the user's next query, after controlling for previous input diversity. This finding exposes a mechanism by which generative models can shape, and potentially limit, the criteria for subsequent user exploration.

Figure 3: Trajectory of input diversity over conversation turns (A) and regression outcomes associating ChatGPT output diversity with subsequent user input diversity (B).

Theoretical and Practical Implications

This work elucidates an essential duality in generative-AI–mediated information seeking. On one hand, conversational AI lowers barriers to open-ended, creative, and non-traditional inquiry, facilitating broader user curiosity and engagement. On the other, the generated answers are—on aggregate—less semantically varied than those surfaced by search engine ranking algorithms, especially for factual or informational domains. This narrowing has both immediate and downstream implications: it can reinforce uniformity in informational exposure and restrict the user’s subsequent conceptual latitude, with potential consequences for learning, innovation, and susceptibility to bias.

The authors advocate for the development of hybrid search paradigms, where generative AI and retrieval-based systems are combined to synergistically support both exploration and diversity. Existing empirical evidence on the collective narrowing of perspectives in LLM-powered search [doshi2024generative, lee2024empirical] is complemented here by quantification of feedback effects, highlighting the manipulative potential of AI output design on user exploration.

Limitations and Directions for Future Research

The primary limitation arises from data silos: direct comparison of user behaviors between search engines and ChatGPT is constrained by the lack of unified interaction logs from the same time period and user base. Thus, observed differences may partially reflect demographic or behavioral selection effects. The ongoing proliferation of hybrid systems—blending retrieval-centric and generative modalities—necessitates further longitudinal and experimental studies to track the evolution of both inquiry scope and output diversity as users' habits and system affordances co-evolve.

Additionally, the link between diversity measures in embedding space and downstream knowledge acquisition or creative outcomes should be explored in greater depth, potentially leveraging controlled user studies or intervention designs.

Conclusion

The paper provides a comprehensive, data-driven account of the impact of generative conversational AI on knowledge diversity in online information seeking. ChatGPT expands opportunity for non-searchable, open-ended inquiry, but the diversity of its generated outputs is systematically lower than that of algorithmic web search for comparable queries. The diversity of system responses not only reflects, but also shapes, ongoing user exploration, underscoring the need for thoughtful system design. Integrating generative and search-based approaches will be essential for fostering both breadth and depth in future knowledge work environments.