LLMVue: LLM-Assisted GUI Testing for Vue.js

Updated 5 July 2026

LLMVue is a framework that leverages LLM-based semantic refinement to extract and normalize executable page transition graphs from Vue.js source code.
It employs a three-stage pipeline—static extraction, LLM refinement, and deterministic normalization—to produce precise, selector-annotated testing models.
Empirical evaluations demonstrate high precision, recall, and improved action, page, and statement coverage in automated GUI testing of Vue.js applications.

Searching arXiv for the exact LLMVue paper and closely related work on visualization/LLM evaluation to ground the article. LLMVue is a source-level, LLM-assisted framework for model-based GUI testing of Vue.js single-page applications. Its central idea is to automatically reconstruct an executable page transition graph (PTG) from Vue source code and then use that graph to guide Playwright-based exploration. In the published literature, LLMVue is introduced as a framework that uses a LLM to generate a PTG from Vue.js source code, infers component hierarchies and route transitions, and merges them into a unified PTG across multiple single-file components (SFCs) (Li et al., 26 Jun 2026). It is presented as the first use of LLMs for model-based GUI testing of Vue.js applications using source-level PTG extraction (Li et al., 26 Jun 2026).

1. Problem setting and research motivation

LLMVue addresses a specific difficulty in automated GUI testing for Vue.js web applications: transition candidates are spread across router configurations and SFCs, and they must be concretized and normalized into an executable PTG for testing (Li et al., 26 Jun 2026). The motivating observation is that Vue applications encode navigation in a distributed and framework-specific way. Route definitions live in router configuration, but actual navigation actions may be triggered in SFCs, inside event handlers, inside shared layouts, or through UI-library abstractions such as Element UI/Plus menus and dropdowns (Li et al., 26 Jun 2026).

The framework is situated within model-based testing. The paper argues that classic crawler-based or URL-only modeling is too weak for Vue.js single-page applications, because the navigation structure is fragmented across many files and runtime conditions (Li et al., 26 Jun 2026). In this setting, a usable navigation model must be both complete enough to avoid missed routes and precise enough to drive executable actions. A representative difficulty arises when two different menu items both lead to /book but are guarded by mutually exclusive role checks; if these are collapsed into a single edge, testing may repeatedly try to click a hidden element and waste exploration budget (Li et al., 26 Jun 2026).

Three technical difficulties are emphasized. First, navigation is split across router configs and SFCs. Second, UI-library abstractions hide real clickable elements, so source-level extraction must be refined into Playwright-ready locators. Third, runtime conditions create multiple versions of the same page transition, meaning that a graph that ignores those conditions becomes misleading for testing (Li et al., 26 Jun 2026). In this sense, LLMVue is not merely a route extractor; it is a framework for producing an executable, selector-annotated PTG that preserves conditional distinctions relevant to GUI testing.

2. Formal model and end-to-end architecture

LLMVue constructs a final PTG $G_3$ in three stages, starting from source code $C$ of a Vue SPA. The PTG is defined as a directed labeled graph $G=(V,E)$ , where $V$ is the set of route nodes and $E$ is the set of navigation edges; each node is a route path from the router, such as /home or /user/add (Li et al., 26 Jun 2026).

Each edge includes source and target routes, an executable selector, an event type such as click or redirect, and an optional visibility or guard condition (Li et al., 26 Jun 2026). The extracted graph is denoted $G_{ext}$ , and the manually constructed ground-truth graph is denoted $G_{gt}$ (Li et al., 26 Jun 2026). The staged pipeline produces three intermediate graph forms:

$G_1 = (V_1, E_1)$ : a recall-oriented candidate graph
$G_2 = (V_2, E_2)$ : an LLM-refined graph
$G_3 = (V_3, E_3)$ : a normalized final PTG (Li et al., 26 Jun 2026)

This architecture is explicitly hybrid. The paper attributes its effectiveness to the combination of high-recall static extraction for broad coverage, LLM reasoning for selector enrichment and condition interpretation, and deterministic normalization for precision and usability (Li et al., 26 Jun 2026). A plausible implication is that LLMVue treats the LLM as a semantic refiner embedded within a larger static-analysis and normalization workflow, rather than as a stand-alone extractor.

3. Three-stage PTG construction pipeline

Stage I is static graph extraction and is designed for high recall rather than precision. It parses the Vue router configuration, all Vue SFCs, and ASTs of templates and scripts (Li et al., 26 Jun 2026). From the router configuration it builds a route table containing valid route paths and mapped components, explicit redirects, and parent-child relations for nested routes (Li et al., 26 Jun 2026). It also computes how many times each parent route appears in the route config, to avoid incorrectly propagating child routes to the wrong parent layout (Li et al., 26 Jun 2026).

From SFC templates and scripts, the framework extracts navigation facts including router-link targets, menu-item to or index values, event bindings, router API calls such as $C$ 0router.replace, and other navigation-like patterns such as window.open and router.resolve (Li et al., 26 Jun 2026). A key auxiliary structure is the component usage index, which maps each component file to all route paths under which it may be rendered, including direct route bindings and transitive usage inferred from component imports and template references (Li et al., 26 Jun 2026). This is essential because a layout component may render children from nested routes even though the clickable menu lives in the shared parent layout (Li et al., 26 Jun 2026).

Stage I produces two major edge types: router-level edges, where each explicit redirect becomes an edge with selector="-" and event=redirect, and component-level edges, where extracted navigation facts are converted into candidate edges for each associated component-route pair (Li et al., 26 Jun 2026). Nested routing is handled by propagating layout-derived menu edges to child routes, while ambiguous widgets and dynamic rendering are deliberately not filtered out at this stage (Li et al., 26 Jun 2026). The result is $C$ 1, a high-recall but noisy candidate PTG (Li et al., 26 Jun 2026).

Stage II is LLM-based semantic refinement and uses GPT-4o as an enhancement-only semantic refiner (Li et al., 26 Jun 2026). It does not delete edges; it preserves the endpoints and event type, and only improves the selector and condition fields (Li et al., 26 Jun 2026). Given source file context, the LLM augments selectors with text cues, normalizes framework-specific rendering, extracts conditions and guards from v-if, v-show, and conditional menu construction in scripts, and preserves recall through validation and fallback (Li et al., 26 Jun 2026). The paper states that the LLM must return the same number of edges as it received, that output is validated against edge count, required fields, and route validity after normalization, and that the system retries up to three times with reflection prompts if parsing or validation fails (Li et al., 26 Jun 2026). If all retries fail, the original Stage I edges are kept (Li et al., 26 Jun 2026).

Stage III is graph normalization and turns $C$ 2 into the final executable PTG $C$ 3 through deterministic cleanup (Li et al., 26 Jun 2026). The normalization operations include post-processing selectors and conditions, selector-level deduplication, route-pair preference filtering, filtering route-table structural edges and wildcard auto-redirect edges, deduplicating submenu-prefix variants for Element UI/Plus, filtering cross-group edges whose branch labels disagree, and filtering most self-loops while retaining some UI-structure self-loops needed for submenu or dropdown expansion (Li et al., 26 Jun 2026). Two edges are considered duplicates if they share the same from, to, event, and selector; the condition field is not part of the deduplication key (Li et al., 26 Jun 2026).

4. Implementation details and executable testing workflow

The LLM backbone used in LLMVue is GPT-4o, with temperature 0 for deterministic extraction (Li et al., 26 Jun 2026). The PTG extraction module is implemented in Java JDK 17, while graph and runtime data collection use Node.js v18.20.8 and npm 10.8.2; the browser is Google Chrome 143.0.7499.194, and the exploration engine is Playwright (Li et al., 26 Jun 2026). Because of token-budget constraints, Stage II groups edges by source file and splits them into batches of 30, with a system prompt enforcing enhancement-only behavior and a user prompt containing source code and JSON edge lists (Li et al., 26 Jun 2026).

The framework is evaluated on ten open-source Vue.js projects from GitHub: Vue-Springboot-Library, questionnaire, dormitory_springboot, music-website, Blog, COVID, VehicleMangement-Web, UniTakeout, stu_exam_sys, and house_rental_management (Li et al., 26 Jun 2026). Their sizes range roughly from 10 to 29 SFCs, with route counts from 7 to 24, and frontend LOC from about 2.2k to 10.2k (Li et al., 26 Jun 2026). These projects were selected after querying GitHub for Vue.js SPA repositories and then manually filtering for systems that are Vue.js SPAs with Vue Router, have non-trivial route-level navigation, and are buildable and runnable end-to-end under version-locked snapshots (Li et al., 26 Jun 2026).

Ground-truth PTGs were built manually using a deterministic, version-locked expert protocol co-developed with Vue/Element UI practitioners (Li et al., 26 Jun 2026). The labeling rules include script-based routing, router-enabled menu navigation, post-submit routing, and submenu expand or collapse as explicit self-loops; they exclude actions that do not change routes; they use stable rendered DOM classes with :has-text constraints; and they prioritize script routing over inherent menu navigation when both exist (Li et al., 26 Jun 2026). To reduce annotator variance, the study used calibration examples, centralized arbitration, and automated sanity checks (Li et al., 26 Jun 2026).

For testing effectiveness, the only baseline is random exploration under the same action and time budget, used as a lower-bound comparator rather than against other GUI-testing tools (Li et al., 26 Jun 2026). The testing budgets are 60s, 300s, and 600s, and the experiment is repeated 10 times per configuration (Li et al., 26 Jun 2026). The testing metrics are Action Number (AN), Statement Coverage (SC), and Page Coverage (PC) (Li et al., 26 Jun 2026). The graph-quality metrics are precision, recall, F1-score, and Graph Edit Distance (GED) (Li et al., 26 Jun 2026).

5. Empirical performance and observed error patterns

Across the ten projects, LLMVue achieved average precision of 94.57%, average recall of 88.79%, average F1 of 91.25%, and average GED of 16 (Li et al., 26 Jun 2026). Per-project F1 ranged from 86.51% to 97.33%, with 5 of 10 projects achieving F1 greater than or equal to 90% and 3 of 10 exceeding 95% F1 (Li et al., 26 Jun 2026). Precision was usually very high, often perfect, while recall varied more strongly due to conditional navigation and callback-driven routes (Li et al., 26 Jun 2026). Reported examples include P1 with 98.65% precision, 96.05% recall, 97.33% F1, and GED 4; P2 with 100% precision, 93.33% recall, 96.55% F1, and GED 2; P3 with 100% precision, 94.04% recall, 96.93% F1, and GED 13; and P10 with 76.60% precision, 100% recall, 86.75% F1, and GED 22 (Li et al., 26 Jun 2026).

For GUI exploration, the testing study focused on P1, which had the highest extraction quality with F1 = 97.33%, and P3, which had the largest PTG and was structurally most complex (Li et al., 26 Jun 2026). LLMVue-guided exploration executed substantially more actions than random exploration under the same time budget. On P1, the action counts were 95.8 versus 60.1 at 60s, 518.0 versus 310.6 at 300s, and 1038.3 versus 658.8 at 600s; on P3, they were 1011.8 versus 606.5 at 300s and 1911.1 versus 1158.7 at 600s (Li et al., 26 Jun 2026). The paper also reports lower variance, indicating more stable throughput (Li et al., 26 Jun 2026).

Page coverage improved earlier under PTG guidance. On P1, PC rises from 54% to 67% at 60s and reaches 70% quickly, while random exploration reaches that level later (Li et al., 26 Jun 2026). On P3, PTG guidance yields sustained page-coverage gains across budgets (Li et al., 26 Jun 2026). Statement coverage also improves with longer budgets: on P1, SC is consistently better and the gap grows with budget, while on P3, SC can lag slightly at short budgets but becomes better at longer budgets (Li et al., 26 Jun 2026). The stated explanation is that PTG guidance improves route discovery efficiency by avoiding redundant interactions, reaching new pages earlier, and reducing time wasted on ineffective actions (Li et al., 26 Jun 2026). Page coverage improves first because PTGs are naturally about navigation, whereas statement coverage improves later because many statements are executed through intra-page widget interactions that do not change routes (Li et al., 26 Jun 2026).

The mismatch analysis identifies two main error families. False negatives mainly come from conditional and callback-driven routing, including navigation only after successful validation, routing inside asynchronous callbacks or confirmation dialogs, and missing links when the route call is not adjacent to the triggering widget (Li et al., 26 Jun 2026). False positives mainly come from selector ambiguity and cross-role noise, including generic menu selectors without enough text constraints and role-dependent menus where some clickable entry points are excluded from the ground truth because they are not stable under a fixed role context (Li et al., 26 Jun 2026).

6. Limitations, future directions, and terminological context

The paper discusses several threats to validity. Ground truth remains ambiguous in some corner cases, especially condition-controlled widgets and callback-driven routing (Li et al., 26 Jun 2026). Edge-level metrics do not directly capture UI robustness or intra-page state diversity (Li et al., 26 Jun 2026). Browser execution and asynchronous rendering can vary run to run, although extraction uses deterministic decoding and testing was repeated multiple times (Li et al., 26 Jun 2026). Generalizability is limited to Vue.js SPAs with Vue Router and may not directly transfer to other frameworks or architectural styles (Li et al., 26 Jun 2026).

The proposed future directions are closed-loop extraction and testing, role-aware and state-aware PTGs, coverage-aware intra-page exploration, and better selector disambiguation using richer DOM context such as ancestor chains, ARIA tags, or stable component IDs (Li et al., 26 Jun 2026). This suggests that LLMVue is best understood as a foundation for more context-aware testing of modern frontend frameworks rather than as a complete solution to GUI exploration.

The term “LLMVue” can be confused with LLM-based work on visualization or visual artifact generation, but the named framework in the literature refers specifically to Vue.js GUI testing (Li et al., 26 Jun 2026). Related but distinct lines of research include VISTA, a multi-agent framework for generating mathematically correct visual aids and aligned math problems for geometry and function tasks (Lee et al., 2024); SimVecVis, a dataset and structured representation for enhancing MLLMs in chart understanding (Liu et al., 26 Jun 2025); LSLE, a literacy-stratified framework for using LLMs as simulated evaluators of visualizations (Panda, 8 Jun 2026); and later experiments arguing that frontier LLMs possess visualization literacy but still struggle with instruction following and graphical integrity in visualization evaluation (Seto et al., 13 Jun 2026). A common misconception is therefore to treat “LLMVue” as a generic label for LLM-based visualization systems. The published evidence instead distinguishes a specific Vue.js testing framework from several separate research programs on visualization generation, visualization understanding, and visualization evaluation.