Conflicts of Interest and Safeguards for AI Agents

Investigate and resolve conflicts of interest between AI Agent providers and users—such as priority of platform or developer instructions over user instructions—and develop improved safeguards to prevent or mitigate undesired AI Agent actions.

Background

Building on a discussion of loyalty in legal agency and the "chain of command" model specification that may prioritize platform or developer instructions over user instructions, the authors highlight potential conflicts of interest in agent behavior. They suggest that value‑alignment should incorporate conflict‑avoidance and transparency, yet acknowledge open questions.

They also emphasize the need for stronger safeguards to reduce undesired agent actions, noting that current technical and socio‑legal mechanisms are powerful but incomplete, and calling attention to gaps that warrant further work.

References

Relying on current approaches is powerful, but open questions around conflicts of interest, and improving safeguards around undesired AI Agent actions remain.

— Responsible AI Agents  (2502.18359 - Desai et al., 25 Feb 2025) in Conclusion

Our experiments examine distinct conflicts across five model families and show directional gains in the tested alternative code-agent settings; applicability to a wider range of settings remains to be established.

— ConflictGuide: AutoResearch Improves When Competing Behaviors Are Made Visible  (2609.39933 - Xu et al., 30 Sep 2026) in Appendix K, Limitations and Future Work