Execution Gap: Bridging Intention and Action

• Execution Gap (EG) is defined as the measurable difference between high-level abstract plans and their effective, parameter-accurate execution across diverse domains.
• It arises from issues such as parameter drift, mapping failures, and cascading errors, which impede the translation of intention into successful action.
• Mitigation strategies include grounded feedback, reinforced learning, and rigorous evaluation protocols to align abstract reasoning with concrete execution.

The term "Execution Gap" (EG) is used across diverse domains—including GUI agents, program synthesis, LLM-driven scientific workflows, robotics, agent safety, and computational social science—to formalize the discrepancy between a system’s abstract intentions, surface-level competence, or high-level reasoning and the fidelity or effectiveness of its realized, concrete actions. EG frameworks precisely quantify where and why systems that appear to "know what to do" systematically fail, stumble, or distort outcomes in the process of actual execution. This article provides a comprehensive, technical survey of the formal definitions, recurring sources, evaluation methodologies, mitigation strategies, and empirical manifestations of EG in contemporary research settings.

1. Formal Definitions and Domain Taxonomy

Distinct classes of Execution Gap arise depending on the domain, but each is mathematically instantiated as the difference between two success measures: the ability to specify or select a desirable plan versus the realized success of the corresponding execution.

a. GUI Agents and Vision-LLMs

• Semantic-Execution Gap: $EG = AA - TSR$, where $AA$ is action-type accuracy and $TSR$ is strict task success rate under point-level geometric tolerances. EG thus measures the frequency with which an agent can select correct abstract actions but fails to execute them with parameter precision necessary for downstream validity (Wei et al., 15 May 2026).
• Reasoning-Execution Gap in Mobile GUI Agents: EG is the rate at which an agent’s chain-of-thought (CoT) reasoning is correct (ground-truth aligned), but actual execution fails:

$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$

where $GTA_n$ and $EM_n$ denote ground-truth alignment and exact match at step $n$ (Dong et al., 2 Oct 2025).

b. Automated AI Research and Scientific Discovery

• Ideation–Execution Gap: The mean per-idea drop in expert review scores from ideation to execution, $$\Delta_m = \mathrm{Score}_{\mathrm{execution},m} - \mathrm{Score}_{\mathrm{ideation},m}$$, reflects the degree to which promising research ideas (especially from LLMs) see their assessed quality diminish upon empirical realization (Si et al., 25 Jun 2025).
• Execution Gap in Automated AI Research: For tasks measured by accuracy to be maximized, $EG(N) = \max_{i \in I_N} R(i) - B$, where $R(i)$ is empirical performance of idea $AA$0, $AA$1 is the baseline, and $AA$2 is the top-N set. For efficiency-minimization tasks, $AA$3 (Si et al., 20 Jan 2026).

c. Code Synthesis and Program Repair

• Execution Gap in Code Generation/Repair: $AA$4, the resolve-rate improvement attributed to permitted code execution in agents (Lin et al., 25 Jun 2026); or as the drop from code-level correctness (e.g., compilability) to end-to-end executable success (e.g., full test or validator pass) (Yang et al., 10 Jan 2026).

d. Robotics

• Motion Execution Gap: Failure of the mapping $AA$5 from symbolic/semantic task specifications to executable, constraint-preserving control objectives (Stelter et al., 12 May 2026).
• Mental-Reality Gap in Code Synthesis: Decomposed into Specification Gap (missed edge cases) and Verification Gap (hallucinated passes vs. failed executions), where both must be closed to eliminate divergence between "mental" code simulation and actual run-time behavior (Lee et al., 20 Apr 2026).

e. Trust and Safety in Open-World Agents

• Authorization–Execution Gap (AEG): Abstractly, divergence between interpreted agent authorization scope $AA$6 and realized execution trace $AA$7, modeled as $AA$8 for some notion of divergence $AA$9 (Wu et al., 10 May 2026).

f. Political Science (for completeness)

• Efficiency Gap: In legislative redistricting, $TSR$0, where $TSR$1 is per-district vote share and $TSR$2 is seat indicator, is the canonical formalization as a per-district weighting function (Wallin, 2018).

2. Root-Cause Analysis and Propagation Mechanisms

Empirical and analytical studies across domains consistently identify a robust typology of root causes for the Execution Gap.

a. Parameter Drift & Geometric Cascades

• In continuous-control and GUI settings, minute parameter deviations at early stages (e.g., sub-pixel errors) can be linearly or even nonlinearly amplified through dependency chains or geometric transformations, resulting in catastrophic downstream failures (Wei et al., 15 May 2026, Wang et al., 26 Jan 2026).
• These are rigorously modeled via local Jacobians: $TSR$3, with compounded amplification in dependency-structured primitives.

b. Specification/Reasoning Bottlenecks

• Agents may reason or plan correctly in symbolic space but fail to ground those plans in executable, parameter-complete, and environment-constrained actions (e.g., missing swipe duration or coordinate precision, or incomplete edge-case enumeration) (Lee et al., 20 Apr 2026, Wang et al., 26 Jan 2026).

c. Semantic–Symbolic Mapping Failures

• Symbolic task planners in robotics fail to guarantee preservation of intended high-level constraints when interfaced with continuous controllers, due to lack of an explicit, sound mapping $TSR$4 (Stelter et al., 12 May 2026).
• In agent safety, incomplete, corrupted, or fragmented delegation and authorization is not recoverable by downstream execution policies (delegation-level incompleteness, channel-level corruption, composition-level fragmentation) (Wu et al., 10 May 2026).

d. Mode Collapse and Search Exhaustion

• In automated research, optimization methods (RL, evolutionary search) can prematurely converge on high-reward but low-diversity idea clusters, raising average scores but failing to discover any dramatically improved or novel solution—EG remains open (Si et al., 20 Jan 2026).

3. Quantitative Measurement and Benchmarking Protocols

Across all domains, EG is empirically quantified using protocolized evaluation schemes that rigorously distinguish abstract competence from execution fidelity.

a. GUI and Control Benchmarks

• Metrics delineate stepwise type selection (Action Accuracy, AA), argument or parameter accuracy (PA), step-level joint correctness (SSR), and full-task completion under strict tolerances (TSR); EG is then $TSR$5 (Wei et al., 15 May 2026).
• In SwipeGen, $TSR$6 is the expected parameter deviation across real-valued and categorical gesture dimensions; execution accuracy is $TSR$7 (Wang et al., 26 Jan 2026).

b. Scientific Workflows and Automated Research

• EG is reported as the best-of-N performance gain or reduction versus baseline, contextualized as the ability to realize putatively high-potential ideas sampled or optimized via LLMs (Si et al., 20 Jan 2026).
• Ideation-execution gap is computed as the empirical decrease from "promising" review at idea stage to achieved score post-execution (Si et al., 25 Jun 2025).

c. Code and Transaction Generation

• EVM-QuestBench computes $TSR$8, with CodeAcc as surface-level (e.g., compile success) and ExecAcc as actual on-chain behavioral correctness; step-efficiency decay scores (SES) provide graded credit for multi-step workflows (Yang et al., 10 Jan 2026).

d. Agent Safety

• AEG detection protocols emphasize process-level reporting: integrity check coverage, source attribution on divergences, and incidence of undetected deviations, rather than outcome-only task scores (Wu et al., 10 May 2026).

e. Program Repair

• In LLM-based repair, $TSR$9 is the resolve-rate benefit from allowed code execution ($$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$0), with paired and equivalence statistical testing to assess practical significance (Lin et al., 25 Jun 2026).

4. Systematic Mitigation and Architecture

Successful approaches to reducing Execution Gap are characterized by architecture and training methods that enforce tight coupling between planning/abstraction and execution/concretization.

a. Supervised and Reinforcement Learning with Grounded Feedback

• Pixel-grounded supervised tuning establishes parameter and grammar priors; precision-aligned reinforcement learning rewards stepwise action-type and parameter accuracy, penalizing exposure bias in rollout (Wei et al., 15 May 2026).
• Reinforcement Learning from Execution Reward must avoid mode collapse—otherwise, only average score increases, not best-case or diversity of solutions (Si et al., 20 Jan 2026).

b. Intermediate Planning Representations and Statecharts

• In robotics, Motion Statecharts formalize hierarchical composition of semantic tasks, integrated with a differentiable kinematic world model and smooth lMPC-based control for jerk-bounded, constraint-respecting task switches (Stelter et al., 12 May 2026).

c. Oracle-Guided and Live Execution Validation

• Addressing the Mental-Reality Gap, architectural patterns require explicit edge-case enumeration and live, sandboxed execution across a property-based oracle set, both to close the specification gap (plan robustness) and the verification gap (absence of hallucinated success) (Lee et al., 20 Apr 2026).

d. Explicit Cost-Benefit Execution Policies

• In program repair and agentic computing, optimal execution policy triggers are derived from cost-benefit modeling: execution is invoked iff $$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$1 exceeds a threshold, controlling token and wall-clock resource expenditure (Lin et al., 25 Jun 2026).

e. Process-level Safety Checks and Attribution

• Agent safety frameworks call for fine-grained, in-flight integrity checks at all boundary transitions (delegation assessment, authority labeling, provenance preservation, recomposition checks), with explicit process-divergence reporting (Wu et al., 10 May 2026).

5. Empirical Manifestations Across Systems

Empirical studies consistently demonstrate the stubbornness and impact of EG even as abstract models improve.

a. GUI Agents and Construction

• General vision-LLMs attain AA > 88% but TSR < 6%, yielding EG > 80 points; topology- and pixel-grounded training in PAGER reduces EG by 25 points, but does not eliminate it (Wei et al., 15 May 2026).
• In swipe execution, baseline VLMs achieve only 32% accuracy, while GUISwiper, trained with parameter-rich data, attains 69%, closing a substantial part of the parameter gap (Wang et al., 26 Jan 2026).

b. Scientific and Research Workflows

• Execution-guided search in automated AI research drives post-training accuracy +21.4 points above baseline (48.0%→69.4%) and reduces pre-training time by 45% (35.9→19.7 min), while RL-only approaches induce mode collapse and stagnate EG (Si et al., 20 Jan 2026).
• In blinded RCTs, AI-generated research ideas initially rate 0.8–1.2 points higher on novelty/excitement but drop by –1.8 points post-execution, with human ideas overtaking AI on all major metrics (Si et al., 25 Jun 2025).

c. Program Synthesis and Repair

• On EVM-QuestBench, atomic code correctness >90% falls to <50% on five-step workflows, showing a 40–50 point EG despite models’ high surface skill (Yang et al., 10 Jan 2026).
• Program repair agents' resolve-rate gaps from execution permissions are ~1–3 percentage points, at a cost of 48–62% increases in computational overhead, with most execution benefit concentrated in late-stage revisions or a minority of tasks (Lin et al., 25 Jun 2026).

d. Trust, Safety, and Multi-Agent Systems

• AEG failures can cause irrevocable harm (fund transfers, memory corruption, policy violation), cannot be reliably detected by up-front or post-hoc filters, and propagate through multi-stage agent handoffs (Wu et al., 10 May 2026).

6. Implications, Limitations, and Future Directions

The persistence of Execution Gaps underlines foundational constraints in current AI, agent, and automation methods.

• EG reveals a distinct bottleneck: surface-level competence does not guarantee reliability in open-ended, parametric, or multi-step tasks.
• Progress on EG requires hybrid architectures that marry symbolic reasoning with fully parameterized, execution-grounded learning and verification.
• Evaluation protocols must consistently report EG metrics—not just outcome scores—for meaningful progress assessment.
• Closing EG is critical for safety, trustworthiness, and cost-effective deployment in autonomous systems, especially in domains with irreversible side effects or financial risks.
• Methodological limitations include the scalability of dynamic execution, reliance on human-in-the-loop baselines, and incomplete domain coverage by current parameterized benchmarks.
• Anticipated research threads include generalized oracle/invariant generation, difficulty-aware and selective execution strategies, and deeper integration of symbolic structure with continuous execution feedback in both agent training and deployment pipelines.

7. Summary Table: Canonical Execution Gap Instantiations

Domain/Task	EG Formalization	Primary Manifestation
Geometric GUI Control (Wei et al., 15 May 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$2	High AA, low TSR due to pixel imprecision
Mobile GUI Agents (Dong et al., 2 Oct 2025)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$3 correct reasoning, failed execution	Dominant over "reasoning gap"
Automated AI Research (Si et al., 20 Jan 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$4	Ideation samples deliver weak executions
Code Synthesis (EVM) (Yang et al., 10 Jan 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$5	Compiles but fails on strict execution
Robotics Motion Planning (Stelter et al., 12 May 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$6 fails to preserve semantics	Symbolic plan not guaranteed feasible
Program Repair (Lin et al., 25 Jun 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$7	Marginal gains, high cost
Agent Authorization Safety (Wu et al., 10 May 2026)	$$EG = \frac{1}{N} \sum_{n=1}^N 1_{ \{GTA_n=1 \wedge EM_n=0\} }$$8	Unchecked/fragmented delegated authority

Execution Gap is thus a central analytical and practical construct in aligning intent and execution in large-scale, complex, and autonomous systems. Its rigorous diagnosis and closure remain open research challenges with wide-reaching implications.