Practical viability and trade-offs of parallel execution for LLM-generated code

Determine whether parallel execution of large language model (LLM)-generated code—dispatching executable statements to an interpreter as they are produced rather than waiting for the full program—is practically viable, and characterize its benefits and costs relative to the conventional serial generate-then-execute paradigm.

Background

LLM-based coding agents commonly follow a serial workflow: the model generates a complete code block and then executes it, causing the generator to be idle during execution and the executor to be idle during generation. The paper proposes a parallel execution paradigm in which executable code statements are identified and run as soon as they are produced, potentially overlapping generation and execution to reduce end-to-end latency.

While prior work explores incremental execution to provide feedback that improves code quality, those approaches remain serial with respect to latency because generation pauses while awaiting execution results. This gap motivates an explicit question about whether overlapping generation and execution is feasible in practice and what the performance trade-offs would be.