LLM capability for naturalistic asynchronous planning

Determine whether large language models can correctly solve naturalistic asynchronous planning tasks by computing the shortest possible completion time for an optimal plan given a set of compulsory steps, their time durations, and step ordering constraints, under the assumption of infinite resources enabling maximal parallelism.

Background

The paper defines naturalistic asynchronous planning as complex tasks that involve both sequential and parallel actions, where the objective is to compute the shortest possible time to complete all required steps subject to ordering constraints. This setting assumes infinite resources, allowing optimal parallelization, and can be cast as finding the longest path in a directed acyclic graph.

The authors note that solving such tasks requires the composition of multiple skills—time summation, time comparison, and constrained optimization—whose joint handling may challenge current LLMs. They introduce the AsyncHow benchmark and the Plan Like a Graph (PLaG) prompting technique to investigate LLM performance on these tasks, motivating the explicit question of whether LLMs are capable of solving them.