- The paper uses semi-structured interviews with 35 developers, risk-ranking activities, and bow-tie analyses to examine how teams perceive, prioritize, and mitigate risks in user-facing agentic AI products.
- Developers prioritized risks based on task quality, failure severity, user trust, compliance, and business goals, while engineering complexity, time, cost, and unclear accountability often limited mitigation efforts.
- The study finds that most safeguards constrain autonomy, adaptability, tool access, or goal complexity, creating a central challenge: controlling agentic risks without undermining the capabilities that make agents useful.
This paper presents a qualitative interview study of 35 industry developers building user-facing agentic AI products, examining how they perceive, prioritize, and mitigate the risks that agentic capabilities introduce. Organized around the Security and Privacy Acceptance Framework (SPAF)—with its three barriers of awareness, motivation, and ability—the study documents how developers' risk practices are shaped by product-proximate concerns and reveals a central structural tension: the controls available for mitigating agentic risks largely operate by constraining the same characteristics (autonomy, adaptability, tool use, goal complexity) that make agents valuable (2606.15485).
Study design and participants
The authors conducted semi-structured interviews (~60 minutes each) with 35 developers at an international software company, all of whom had direct experience addressing risks in user-facing agentic AI products spanning domains such as customer support (n=7), information technology (n=7), corporate services (n=6), finance/banking, legal/compliance, healthcare, and government. The study was approved by an internal review committee, which restricted collection of demographic data.
Three instruments grounded each interview in real product contexts. First, a pre-study questionnaire asked participants to describe their product, select three human-centered AI (HAI) principles of concern from an adapted list, and rate the relevance of agentic AI risks drawn from the AI Risk Atlas (Bagehorn et al., 26 Feb 2025) to each principle. Second, a Mural-based risk-ranking activity presented five product-relevant risks for prioritization and justification. Third, a bow-tie analysis—adapted from safety-critical industries via Koessler and Schuett (Koessler et al., 2023)—had participants map causes, preventive controls, consequences, and protective controls for one risk their team had attempted to mitigate. Analysis followed iterative open coding with a shared codebook developed across multiple coders.
Notably, three risks discussed by participants were not fully captured by the AI Risk Atlas baseline: agent policy violation, compounding errors, and compounding latency—suggesting practitioner experience is already outpacing existing taxonomies.
Risk awareness: agency creates and reshapes risk
For RQ1, developers' risk perceptions clustered around two dimensions: risks newly created or instantiated by agentic characteristics, and risks exacerbated by deployment requirements.
Agentic characteristics as risk sources. Agent autonomy enabled agent policy violation (8/35): prompt-based "agent policies" proved unreliable constraints, with one participant noting "the constraints are not really constraints." Environment complexity exposed connected resources to attack (10/35), including an IT agent flagged by cybersecurity for attempting to delete a root folder. Goal complexity exacerbated model inaccuracy (19/35), the most frequently cited risk, because errors compound across multi-turn autonomous execution—one participant set an 80% minimum performance threshold below which "the agent is actively harming your system." Adaptability produced new instantiations of reproducibility risk (8/35), as agents select different tools and plans across similar situations.
Deployment requirements. Layered multi-agent architectures intensified compounding errors (7/35) and compounding latency (4/35). Security threats included unauthorized use (3/35) and IP/PI information leakage via tool use (14/35). Deployment also surfaced unclear action accountability (8/35)—particularly acute in multi-agent systems assembled across teams—and unexplainable, untraceable agent actions (16/35), which participants viewed as a critical barrier to user trust and to deployment in regulated environments.
Awareness barriers. Awareness was uneven: nine participants reported no organizational guidance ("zero resources available"), forcing reliance on individual judgment extrapolated from non-agentic AI experience; eight described difficulty distinguishing hype from genuine capability when scoping both product value and risk. The implication is that downstream societal harms—job displacement, end-user privacy—are systematically under-anticipated not because developers reject them, but because available risk-identification structures are organized around product utility and deployment.
Prioritization: business success as implicit risk model
For RQ2, the ranking activity revealed that developers prioritized risks through five consequence-oriented lenses: agent task quality (14/35), production failure severity (12/35), user adoption and trust (13/35), regulatory compliance (11/35), and alignment with client/business objectives (9/35). Risks were also weighted by mitigation readiness: perceived robustness of the model/tool stack (14/35) led to de-prioritization—for instance, one team dismissed function-calling hallucination after adopting ReAct (Yao et al., 2022)—while poorly understood mitigations paradoxically elevated risk priority (9/35).
These priorities came with substantial inhibitors. Most prominently, mitigation carried opportunity costs: trade-offs against agent performance (12/35), cost-efficiency (6/35), business priorities and timelines (10/35, 9/35), and engineering complexity (22/35—the most cited barrier, since controls add guardrails, policies, and maintenance burden to already complex systems). Responsibility externalization compounded this: 15 participants bounded their accountability to components they owned, with one dismissing regulatory concerns as "not really my concern," echoing prior findings on distributed ethical responsibility in AI practice. The paper's claim here is direct: unless a risk could be tied to business success, developers encountered more inhibitors than motivators—a mechanism that helps explain the persistent principle-practice gap documented in responsible AI literature.
Mitigation: controls that constrain agency
For RQ3, bow-tie analyses surfaced five mitigation strategy families:
| Strategy |
Representative controls |
Prevalence examples |
| Constrain agentic characteristics |
Human-in-the-loop approval, tool specification, limiting problem space, predefined workflows/output formats |
Autonomy (7/35), environment (8/35), goals (5/35), adaptability (15/35) |
| Reduce/recover from errors |
Action timeouts, validator agents / LLM-as-a-judge, retry loops |
Validation (14/35) |
| Strengthen agent-mediated security |
Agentic access control aligned to user permissions, tool-combination restrictions, input/output sanitization |
Access control (9/35), sanitization (11/35) |
| Prevent human-agent interaction errors |
Traces, citations, source references, query rewriting |
Visibility (13/35), intent clarification (6/35) |
| Apply LLM/software best practices |
Model selection, fine-tuning on task data, redundancy, monitoring |
LLM practices (9/35), SE practices (14/35) |
The recurring pattern is the paper's core finding: nearly every control works by reducing autonomy, adaptability, tool access, or goal scope. One participant replaced an adaptable agentic retrieval solution with a "predefined workflow"; another framed the tension as "how far do we swing to give the user the ability to do things but also not shoot themselves in the foot?"
Ability barriers compounded the tension. Ten participants lacked reliable risk assessment resources—ground truth data, domain-specific benchmarks, and realism relative to live environments. Thirteen found controls immature and case-specific; six acknowledged controls only reduced rather than eliminated risk (one called disclaimers "a crappy solution. It's just giving up"). Most tellingly, 17 participants worried that controls themselves inherit nondeterminism: validator agents, LLM-as-a-judge mechanisms, and policy-following agents are stochastic systems introduced to contain stochastic systems—"you are introducing another point of failure" (P25).
Discussion and proposed directions
The authors synthesize SPAF-aligned responses. For awareness, they propose adapting generative-AI risk-scanning tools (e.g., Farsight-style approaches) to foreground agentic data flows, and scaling LLM-powered red teaming across tool-use and prompt-injection scenarios. For motivation, they suggest pro-social design repositories providing social proof and empathy-based approaches translating abstract harms into relatable consequences—while cautioning these work only when leadership signals that mitigation carries equal weight to commercial goals. For ability, they argue the gap is less about absent resources than about support for navigating trade-offs, and propose agentic-specific design artifacts (checklists, impact assessments) that help teams articulate which agentic characteristics their product depends on and what each control would cost.
Limitations
The paper concedes three limitations plainly. Findings derive from self-reported experiences at a single international software company, limiting generalizability across industry contexts. The protocol did not prime participants on what counts as an agentic AI risk, so results may differ under more scaffolded elicitation. And recruitment targeted developers already engaged in risk work, likely skewing the sample toward higher-than-average risk awareness—which means the awareness gaps reported here may understate those in the broader developer population.
Conclusion
Through 35 practitioner interviews structured by SPAF, this study provides an empirical account of agentic AI risk practice: developers perceive risks generated by autonomy, environment and goal complexity, and adaptability; prioritize them through an implicit model of business success; and mitigate them with controls that largely operate by making products less agentic. The capability vs. risk control tension the paper identifies—developers must manage risks created by agentic capabilities using tools, incentives, and organizational structures that offer little support short of constraining those capabilities—frames a concrete open question for the field: what mitigation techniques can bound agentic risk without proportionally sacrificing the autonomy and adaptability that constitute the product's value?
