Elicitron: LLM-Driven Requirements Elicitation

• Elicitron is an LLM agent–based framework that simulates user interactions to uncover both explicit and latent product requirements.
• It employs both parallel and serial agent generation strategies, with context-aware prompts enhancing diversity and extreme use case exploration.
• The framework automates interview transcript analysis using explicit criteria and chain-of-thought reasoning to reliably classify latent needs at low cost.

Searching arXiv for the specified paper and closely related work. Elicitron is an LLM agent–based simulation framework for design requirements elicitation that uses LLMs to generate simulated user agents, let those agents experience and describe interacting with a product, interview them to surface user needs, including latent needs, and automatically analyze interview transcripts to extract and classify needs (Ataei et al., 2024). It was built to address limitations of conventional requirements elicitation methods—user interviews, focus groups, and prototyping—which are described as time-consuming and costly, limited in coverage, and weak at uncovering latent needs. In this framework, latent needs are defined, following Lin et al. 2007 and Maalej et al. 2015, as unarticulated, unexpected factors that strongly influence product desirability. The framework is positioned as a scalable method for exploring diverse and extreme usage contexts, structuring outputs as actions, observations, challenges, and needs, and automating analysis at very low cost.

1. Definition and motivation

Elicitron was introduced for early-stage product development, where requirements elicitation is both critical and difficult. The stated motivation is that conventional methods often fail to capture the full spectrum of user needs, which may lead to products that fall short of expectations. The framework therefore uses LLM-generated agents to scale up user perspectives and to explore a broader range of user needs and unforeseen use cases than can typically be reached through direct human recruitment (Ataei et al., 2024).

The emphasis on latent needs is central. These needs are defined as unarticulated, unexpected factors that strongly influence product desirability. They matter because they often drive innovative features and major design changes, yet are notoriously hard to discover via traditional interviews, especially under “ordinary” conditions. Elicitron addresses this by systematically exploring extreme and diverse usage contexts, including disabled users and users in extreme environments, and by structuring agent outputs so that they are directly usable in design.

A plausible implication is that Elicitron treats elicitation not as a single interview stage, but as a simulation-and-analysis pipeline in which user diversity, contextualization, and automated synthesis are jointly optimized. This suggests a shift from small-sample qualitative inquiry toward high-throughput computational exploration of design requirements.

2. System architecture and workflow

Elicitron’s architecture has four main components: Agent Generation, Product Experience Generation, Agent Interview, and Latent Needs Identification and Reporting (Ataei et al., 2024). Across the pipeline, LLM outputs are constrained using a Pydantic schema and validation to ensure structured, consistent data.

In the Agent Generation stage, each simulated user agent is represented by a Name, a Description, and a Reasoning chain explaining why the agent was created. Two generation strategies are described. In Parallel agent generation, the LLM is called $N$ times independently to create $N$ agents. This is fast, but it lacks awareness of other generated agents and therefore tends toward low diversity and redundancy. To mitigate that problem, the framework can overgenerate agents, compute description embeddings with text-embedding-ada-002, cluster them with KMeans, and select representative agents. Formally, given agents $A = \{a_1,\dots,a_N\}$ with embeddings $v_i$, the procedure computes embeddings, applies $\text{KMeans}(V, k)$, and selects one representative agent from each cluster to form $A'$. In Serial agent generation (context-aware), one prompt asks the LLM to generate $N$ agents in a single response while keeping prior agents in context, explicitly biasing generation toward “different” kinds of users. This context-aware process improves diversity but is slower and limited by output token length, around 20 agents per call under typical caps.

In Product Experience Generation, each agent “hallucinates” a structured product use scenario. For each step the agent produces an Action, an Observation, and a Challenge. The paper’s example for an “Elderly with arthritis” agent includes grasping a tent zipper, observing that the zipper is small and lacks grip, and reporting significant struggle to open or close the tent. This Action–Observation–Challenge structure is explicitly said to be inspired by chain-of-thought reasoning, with the purpose of making implicit issues explicit and traceable.

In Agent Interview, Elicitron simulates a user interview using either designer-crafted or LLM-generated question pools. Questioning is contextualized: each agent’s product experience narrative is included in the prompt, and responses are generated from the standpoint of that specific persona and scenario. In the tent-design experiment, the interview consisted of one free-style question—“If you were to purchase an ideal tent, what main characteristics would you look for?”—and 11 category-specific questions covering size, shape, weight, material, safety, durability, aesthetics, ergonomics, cost, setup, and transport.

In Latent Needs Identification and Reporting, an LLM extracts candidate needs from interview text, reasons step-by-step about each candidate against human-provided latent-need criteria, labels each need as latent or non-latent, and generates a report summarizing needs and latent needs. The criteria borrowed from Lin 2007 are binary: a need is latent if it either represents a significant change to the product design outside the listed categories, or reflects an exceptionally innovative and clearly expressed insight regarding the product and/or how it is used.

3. Diversity-oriented agent generation

The first validation experiment evaluates how agent generation strategy affects diversity of agents and identified needs (Ataei et al., 2024). Three conditions were compared for a tent design problem: serial generation, parallel generation, and parallel generation plus KMeans filtering. Each condition generated 20 agents, and those agents answered 12 interview questions. Diversity was measured using embeddings of role descriptions and of responses to each question.

Three embedding-based metrics were used. Convex hull volume measures the hypervolume of the smallest convex set containing all embeddings, with larger volume interpreted as broader coverage in embedding space. Mean distance to centroid is defined for embeddings $\{v_i\}_{i=1}^N$, with centroid $c = \frac{1}{N}\sum_i v_i$, as

$$\text{MeanDist} = \frac{1}{N}\sum_{i=1}^N \|v_i - c\|.$$

Larger mean distance indicates greater spread. Silhouette score is defined for clustering with intra-cluster distance $N$0 and nearest-cluster distance $N$1 as

$N$2

In this work, lower silhouette scores across $N$3 are interpreted as more scattered points and thus higher diversity of concepts; silhouette is also used to choose $N$4 for KMeans.

The reported quantitative results, all normalized to $N$5–$N$6, favor context-aware serial generation. For convex hull volume, the mean over all 13 embedding sets was $N$7 for serial generation, $N$8 for parallel generation, and $N$9 for parallel generation plus filtering. For mean distance to centroid, the corresponding means were $A = \{a_1,\dots,a_N\}$0, $A = \{a_1,\dots,a_N\}$1, and $A = \{a_1,\dots,a_N\}$2. Serial generation also produced lower silhouette scores, indicating that its embeddings were harder to cluster tightly.

The qualitative analysis reinforces the same conclusion. When all 60 agents from the three methods were clustered with KMeans using $A = \{a_1,\dots,a_N\}$3, the four summarized groups were: Service and Conservation; Outdoor Recreation and Camping; Adventure and Exploration; and Family Camping and Outdoor Activities. Serial generation covered all four groups, including Service and Conservation, whereas both parallel methods failed to generate any Service and Conservation agents. The explicit interpretation is that context-aware serial generation produces more truly diverse roles and corresponding needs.

This suggests that diversity in elicitation is not merely a function of sample size, but also of prompt context and inter-agent dependence. In Elicitron, persistent context acts as a diversity prior.

4. Simulated empathic lead user interviews

The second experiment evaluates whether Elicitron can mimic empathic lead user interviews and discover latent needs in a tent-design benchmark derived from Lin 2007 (Ataei et al., 2024). The conceptual basis is the lead user literature associated with Von Hippel and Urban, together with empathic design methods that expose users to extraordinary conditions so that latent needs become visible. Elicitron operationalizes this by creating LLM agents that explicitly embody atypical or extreme conditions, then eliciting their needs through contextualized interviews.

Three Elicitron conditions were used, each with 20 agents. Condition 1 used automatic serial generation without special steering. Condition 2 used serial generation plus a steering prompt defining a “typical user” and instructing the LLM to create non-typical users. Condition 3 used manually defined empathic lead user personas. The baseline was Lin’s empathic lead user interviews with 20 human participants. Examples of generated or defined agents include Young Outdoor Adventurer, Festival-goer, Military Personnel, Humanitarian Worker, Person with Physical Disability, High-Altitude Climber, Elderly with arthritis, Visually impaired, Wheelchair accessible camper, and Ultramarathon runner. All agents underwent product experience generation and answered the same 12 interview questions as the human participants in Lin’s study.

Human experts labeled phrases in agent responses as latent or non-latent using the Lin criteria. Inter-rater agreement was measured with F1-score, defined for this information-retrieval-like task as

$A = \{a_1,\dots,a_N\}$4

where $A = \{a_1,\dots,a_N\}$5 denotes phrases both raters label as latent, $A = \{a_1,\dots,a_N\}$6 denotes phrases only rater 1 labels as latent, and $A = \{a_1,\dots,a_N\}$7 denotes phrases only rater 2 labels as latent. The reported values are $A = \{a_1,\dots,a_N\}$8, $A = \{a_1,\dots,a_N\}$9, $v_i$0, yielding $v_i$1, which the authors interpret as reliable agreement.

The mean latent needs per participant or agent were higher for all Elicitron conditions than for Lin’s human empathic lead user interviews. The reported Elicitron values are $v_i$2, $v_i$3 for Condition 1; $v_i$4, $v_i$5 for Condition 2; and $v_i$6, $v_i$7 for Condition 3. A comparison between Condition 3 and Condition 1 yields $v_i$8, $v_i$9, indicating that manual empathic lead user agents identified significantly more latent needs than purely automatic agents. The difference between Condition 3 and Condition 2 was not statistically significant.

The paper also characterizes stylistic differences between simulated and human interviews. Simulated agents naturally introduced rich extreme contexts, such as setup in darkness, high wind or snow, or limited dexterity. Their narratives were systematically structured by the Action–Observation–Challenge format. Their needs often embedded conceptual design solutions, including self-erecting tents and adaptive anchoring systems. The reported example latent needs include a tactile floor path and sloped base for a visually impaired user, seated-position accessibility for tent controls for a wheelchair accessible camper, and an adaptive anchoring system for a high-altitude climber. These are presented as non-obvious, innovative design changes satisfying the latent-need criteria.

5. Automated latent-need analysis

The third experiment studies whether LLMs can analyze interview text and classify latent needs automatically (Ataei et al., 2024). The dataset consisted of 20 latent needs and 20 non-latent needs derived from human expert labels in Experiment 2. Three prompt-based strategies were evaluated: zero-shot detection, detection with latent needs criteria, and detection with criteria plus chain-of-thought.

The reported evaluation metrics are standard classification measures: $\text{KMeans}(V, k)$0

$\text{KMeans}(V, k)$1

$\text{KMeans}(V, k)$2

The results show a marked improvement when explicit criteria are included and a further improvement when step-by-step reasoning is required. Zero-shot detection achieved Precision $\text{KMeans}(V, k)$3, Recall $\text{KMeans}(V, k)$4, and F1 $\text{KMeans}(V, k)$5. Detection with criteria achieved Precision $\text{KMeans}(V, k)$6, Recall $\text{KMeans}(V, k)$7, and F1 $\text{KMeans}(V, k)$8. Detection with criteria plus chain-of-thought achieved Precision $\text{KMeans}(V, k)$9, Recall $A'$0, and F1 $A'$1.

The paper’s examples illustrate the decision logic. For a latent need involving a photography-enhancing tent with a wide-angle interior and designated openings for wildlife observation, the model reasons that the concept is a significant redesign, does not fit simple categories such as size or durability, and introduces an innovative way of using the tent as a wildlife observation platform; it therefore satisfies both criteria. For a non-latent need involving a more tear-resistant tent floor, the model reasons that the request falls within durability and is a straightforward improvement rather than a fundamentally new usage or design concept.

A plausible implication is that Elicitron treats interpretability as an operational feature rather than a post hoc diagnostic. The chain-of-thought prompt is used not merely to improve accuracy, but also to make the latent-need labeling rationale inspectable.

6. Positioning, implications, and limitations

Elicitron is positioned relative to several established research areas. In requirements engineering and elicitation, it augments interviews, focus groups, surveys, and prototyping with LLM-driven simulation. In empathic design and latent needs, it extends methods such as photo diaries, physical simulations, and empathic guidelines by operationalizing empathic lead user perspectives with LLM agents. In lead user methods, it simulates users whose needs are ahead of the market through persona generation and steering. In LLM agents and role-play, it integrates persona simulation into a full requirements-engineering workflow. In design diversity metrics, it applies convex hull volume, mean distance to centroid, and silhouette score to elicitation participants rather than designs. In automated text analysis and classification, it uses explicit criteria and chain-of-thought to classify latent needs with high fidelity (Ataei et al., 2024).

The work identifies several practical implications for product development. Under the GPT-4 Turbo pricing used in the paper, Experiments 1 and 3 cost only a few cents, and Experiment 2, using 80,000 tokens, cost approximately $A'$2 total. The framework is therefore presented as making large-scale early-stage requirements elicitation economically feasible, enabling more iterations and broader exploration within the same budget. By generating diverse personas and extreme conditions, surfacing latent needs through contextualized interviews, and automatically classifying and summarizing results, Elicitron helps reveal unexpected use cases, edge cases, and inclusivity-related concerns that might otherwise be neglected.

The limitations are also explicit. The quality of insights depends on model capability, training data, and behavioral biases, so simulated users may reproduce biases and stereotypes. Human judgment remains necessary for prioritizing needs and assessing feasibility and market impact. LLM agents are not real users, and their outputs should therefore be validated in follow-up studies with actual users. Serial generation also encounters practical token limits, around 20 agents per call, although it can be scaled with multiple calls. Ethical considerations are described implicitly: sensitive demographics and disabilities require careful representation, and simulated findings should not be deployed without real-user validation.

The paper identifies several future directions: user studies that evaluate how Elicitron aids designers in real projects; multi-agent interactions that allow agents to interact or debate; multimodal inputs and outputs involving images, CAD models, or sketches; and improved prioritization and ranking of latent needs using quantitative criteria or multi-criteria decision methods. Taken together, these directions indicate that Elicitron is not only a prompt-engineering artifact but an extensible computational framework for design requirements elicitation in which generation, simulation, interviewing, and analysis are treated as an end-to-end system.