Prioritize Programmatic Methods or UI Workflow Emulation for Half-Programmable Tasks

Determine whether research and development for AI computer-use agents should prioritize advancing programmatic methods or emulating human interface–based workflows to solve tasks that are theoretically programmable but lack clear, direct programmatic paths in the tools used by humans, exemplified by tasks such as logo design.

Background

The paper finds that current AI agents overwhelmingly favor programmatic approaches, even for visually intensive tasks, whereas human workers rely more on UI-oriented tools. To guide effective human–agent teaming, the authors propose three levels of task programmability: readily programmable, half programmable, and less programmable.

For half-programmable tasks—those theoretically programmable but lacking direct programmatic paths in human-used tools—the authors explicitly state uncertainty about whether progress should focus on building stronger programmatic methods for agents or on better emulating human UI workflows. Resolving this question would clarify development priorities and collaboration interfaces for such tasks.