Unknown LLM Decision Process in Unconstrained Recommendations

Determine the decision-making mechanism by which large language models select a candidate application when responding to unconstrained information-retrieval prompts that do not specify evaluation criteria (for example, requests to "find any text editor"), in order to clarify how such selections are made.

Background

The paper analyzes software-related AI queries to assess when LLMs are necessary versus when lower-cost alternatives would suffice, using a taxonomy that distinguishes factoid, convenience, and complex queries. A key concern is whether some information-retrieval uses rely on deep reasoning or merely replicate simple lookup behavior, which has implications for sustainability via digital sobriety.

To illustrate this, the authors compare an unconstrained prompt ("find any text editor") with a criteria-based prompt ("find the best text editor for software development"). They explicitly note that for the unconstrained case, the internal basis for the model’s selection is unknown, raising a transparency question that affects how such usage should be judged in terms of necessity and sustainability.

References

No complex thinking is required to retrieve a potential candidate application without further criteria; we do not know how the AI is making its decision, and as such its results are no different than a google search to retrieve a text editor.

— Sustainable AI Assistance Through Digital Sobriety  (2603.29222 - Jennings et al., 31 Mar 2026) in Section 4, Findings (paragraph discussing illustrative prompts)

Capturing this structure once, as a library of skills the agent can draw on, would let it reuse the route instead of working it out again for every task; what remains open is how the agent should select from such a library once it holds many skills and only a few fit the task at hand.

— EconSkills: Studying Skill Transfer and Retrieval for Web Agents on Live Economic Data  (2609.19523 - Quan et al., 17 Sep 2026) in Section 1, Introduction

Second, reasoning trace extraction was confined to the Google Gemini family. Whether similar shifts in the adopted objective emerge in reasoning LLMs from other providers, such as Anthropic, remains an open empirical question.

— Shopping by algorithm: How agentic AI deploys human heuristics as a surrogate consumer  (2609.28372 - Wadi et al., 23 Sep 2026) in Web Appendix E, subsection “Objective shift”