No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot

Abstract: Conversational LLM-based programming assistants'' provide a range of benefits to developers. However, recent studies demonstrate the variety in individual developers' needs regarding programming assistants, and challenges encountered by only specific groups of developers. In this study, we explore the role of cognitive diversity in shaping interactions with GitHub Copilot chat. Through a mixed-methods think aloud study with 27 professional developers and students, we characterize 5 distinctinteraction modes'' and 10 underlying needs in developers' interactions, forming a conceptual model. We characterize links between these modes, needs, and developers' problem-solving styles and experience profiles, showing how cognitive diversity may shape developers' interactions. We provide insights and recommendations for researchers and practitioners on how to design, research, and employ programming assistants to better account for diverse developer needs.

• The paper presents a conceptual model that links developers' cognitive profiles with distinct interaction modes in Copilot chat.
• It employs a mixed-methods think-aloud study with 27 participants and topic modeling to uncover nuanced developer needs and behaviors.
• Findings highlight the need for adaptive, context-aware AI designs to effectively support diverse developer experiences and improve tool usability.

Cognitive Diversity and Developer-AI Interaction: A Conceptual Model and Empirical Study

Overview

The paper "No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot" (2606.19216) presents an empirical investigation into the role of cognitive diversity—primarily problem-solving styles and experience profiles—in shaping developer interaction with conversational LLM-based assistants, specifically GitHub Copilot chat. Utilizing a mixed-methods think-aloud study with 27 professional developers and students, the authors derive a conceptual model that characterizes five distinct interaction modes and ten developer needs influencing behavior. Their analysis explores the relationships among these modes, underlying needs, individual cognitive profiles, perceived usability, and cognitive load.

Methodology

Participants were recruited from diverse professional and educational backgrounds to maximize cognitive and identity diversity, though gender diversity in the sample was limited. Cognitive profiles were assessed via a pre-study survey combining self-reported experience and GenderMag-based measurements of problem-solving styles. Tasks were drawn from a modified React codebase, increasing in complexity, and participants interacted with Copilot chat in well-controlled environments.

Think-aloud protocols and retrospective interviews generated rich qualitative data. Topic modelling (LDA for individual prompts, NMF for prompt sequences) was applied to annotation tokens, revealing distributions over prompt types and interaction modes. Developer needs were inductively derived via open and axial coding of transcripts, and quantitative analysis (using CLR-transformed compositional data and Spearman correlations) mapped interaction mode distributions to cognitive profiles and perceived outcomes.

Cognitive Profiles and Diversity

Empirical evidence confirmed substantial diversity among participants across experience and problem-solving style spectra.

Figure 1: Distributions in participants' cognitive profiles, consisting of an experience profile and problem-solving styles.

This diversity enabled the characterization of nuanced relationships between individual traits and interaction behavior in conversational AI contexts.

Interaction Modes

Five interaction modes emerged from topic modelling, each associated with distinct prompting patterns, strategy preferences, and levels of delegation or control:

• Navigator: Using Copilot to locate features or code, often with uncontextualized queries.
• Autopilot: Delegating entire tasks or debugging problems with minimal intervention ("one and done" strategy).
• Deputy: Incremental, collaborative prompts, often seeking advice or explanations.
• Technician: Consulting Copilot for granular code changes via low- or mid-level instructions, maintaining high control.
• Scholar: Using Copilot for code explanations and information support, typically phrased in concrete, code-level terms.

Figure 2: Interaction mode distributions per participant-task combination. Average task-mode distributions reveal substantial behavioral diversity and mode switching.

Figure 3: Expected distributions per mode of the number of prompts per task, prompt length, and stripped prompt length (omitting code and instructions).

Deputy mode dominated overall usage, while Technician mode—aligned with hands-on control—was more prevalent among highly experienced participants.

Developer Needs in Conversational Interaction

Ten core developer needs were identified, encompassing ability, agency, experimentation, assurance, productivity, ease, learning, understanding, delegation, and control. These needs frequently reinforce or compete with each other, shaping developers' interaction modes dynamically and contextually.

Figure 4: Developer needs (purple) and interaction modes (blue) in Copilot chat, with relationships to individual and contextual factors; links may be positive ($+$), negative ($-$), or mixed ($\sim$).

For instance, increased task complexity elicits greater need for ability (favoring delegation), while professional context heightens agency and assurance, driving self-reliance and control. Ease and productivity encourage delegation but reduce desire for understanding or control; contrasted with learning and understanding, which promote interaction modes favoring explanations and incremental builds.

The conceptual model reveals these interdependencies and suggests that "one-size-fits-all" interaction paradigms are insufficient for inclusive developer-AI support.

Quantitative Correlation Analysis

Significant correlations were observed between interaction mode distributions and experience profiles, problem-solving styles, perceived usefulness, ease of use, and cognitive load:

Figure 5: Spearman rank correlations between interaction modes and experience profile, problem-solving styles, perceived usefulness and ease of use, and cognitive load. Significant correlations and effect strengths are annotated.

• Experience profiles: More experienced developers prefer Technician mode (control-oriented), less use Navigator, Autopilot, and Deputy modes (delegation-oriented).
• Problem-solving styles: Positive (though weak and often insignificant) associations between self-efficacy and Autopilot mode; motivation correlated positively with Technician mode, negatively with Scholar and Deputy modes.
• Perceived outcomes: Autopilot mode correlated with lower cognitive load; Technician mode correlated with increased physical effort; Scholar mode with higher perceived effort.

Overall, the empirical mapping supports the conceptual model's prediction that individual differences and contextual factors fundamentally shape developer-AI interaction, confirming that cognitive diversity is not universally addressed by current LLM-based assistants.

Implications for AI Assistant Design and Research

The findings highlight limitations of current conversational programming assistants in supporting the full spectrum of cognitive diversity. Interaction modes and underlying needs are dynamic, often shaped by context, task difficulty, and individual developer profiles. Design strategies that merely optimize for productivity or ease may fail to support agency, understanding, or learning needs, promoting inequities and dissatisfaction among specific subgroups.

These insights argue for adaptive, context-aware, and customizable assistant behaviors, echoing the need for affordances in HCI and basic human-AI interaction guidelines. The conceptual model serves as a foundation for future research on inclusive AI design, pointing toward individualized interaction strategies, task/context adaptation, and the identification of cognitive diversity-aware interaction affordances.

Future Directions

Potential avenues include controlled experimental testing of causal relationships, extending interaction mode analysis to real-world multi-developer settings, dynamic mode switching, and task lifecycle coverage (e.g., planning, design, not just implementation). External validation and broader studies encompassing greater gender and identity diversity are warranted.

Additionally, cross-tool comparisons (e.g., inline vs. conversational assistants), investigation of LLM affordances, and exploration of temporal dynamics in interaction behavior may deliver actionable guidelines for more inclusive, effective developer-AI collaboration.

Conclusion

This paper presents a robust empirical and conceptual analysis of how problem-solving styles and experience profiles shape developer needs and interaction modes with conversational LLM-based assistants. The proposed model highlights interdependent needs and demonstrates the substantial variation in preferred interaction styles, task strategies, and perceived outcomes. These results call for adaptive, diversity-aware design of programming assistants and inform both future research agendas and practical tool-building strategies for effective human-AI collaboration.