Survival of LLM repairs across the mobile runtime boundary

Determine whether repairs produced by repository-level LLM agents survive the complete mobile build–install–launch–test boundary, rather than being misclassified because missing SDKs, offline devices, or pre-assertion application crashes prevent the intended behavior test from executing.

Background

Repository-level LLM agents are commonly evaluated with tests that execute on the build host. Mobile application repairs face an additional execution boundary: the repaired application must be built with the required SDK, installed on a device or simulator, launched successfully, and exercised by a runtime test. Failures before the behavioral assertion—such as unavailable SDKs, offline devices, or application crashes—can be mistaken for failures of the generated patch. AppEval is introduced to evaluate this issue through a native-toolchain workflow, but the abstract explicitly identifies whether repairs survive this boundary as unclear.