Capabilities Needed for Deploying Agents in Extended Real-World Workflows

Identify the specific capabilities required for deploying AI agent systems in real-world settings such that they can complete tasks independently or effectively support humans across extended workflows.

Background

The paper distinguishes traditional single-step benchmarks from agentic, multi-step evaluations and argues that agent benchmarks better probe deployment-relevant capabilities.

Despite this, the authors explicitly acknowledge that clarifying which capabilities are necessary for agents to operate autonomously or in augmentation roles across extended workflows remains an open question for real-world use.

References

Agent benchmarks therefore more directly probe the capabilities needed for deploying AI systems that can complete tasks independently or support humans across extended workflows—an important open question for real-world use.

— How Well Does Agent Development Reflect Real-World Work?  (2603.01203 - Wang et al., 1 Mar 2026) in Appendix B, Agentic Benchmark Selection

Furthermore, as the capabilities of the ChatInvent agent are still growing, the scope of questions needs to be grow as well, and it remains to be seen what the most prominent issues with the agent will be in the future.

— Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment  (2608.21057 - Granqvist et al., 21 Aug 2026) in Section 7, Discussion, p. 17

Research systems are already testing this frontier, but reliability is not solved.

— Beyond Code Generation: Reliability, Verification, and Cost Economics in the Agentic Software Development Lifecycle  (2609.04681 - Bhati, 4 Sep 2026) in Section 7.3, “H2: multi-agent and long-horizon execution”

We pose two research questions along these lines which need to be answered before deployment:-

— Deploying Foundation Models for Embodied Navigation  (2609.25666 - Dorbala et al., 22 Sep 2026) in Section “Conclusion {content} Future Directions,” subsection “On Day-One Unboxing”

Yet building agents that are both capable and deployable remains an open problem.

— Crossing the Cyber Divide: Sim-to-Sim and Sim-to-Real Transfer for RL Agents  (2610.00759 - Saika et al., 30 Sep 2026) in Section 1, Introduction