---
title: 'LLMVue: LLM-Assisted GUI Testing for Vue.js'
url: https://www.emergentmind.com/topics/llmvue
type: topic
---

# LLMVue: LLM-Assisted GUI Testing for Vue.js

Searching arXiv for the exact LLMVue paper and closely related work on visualization/LLM evaluation to ground the article.
LLMVue is a source-level, LLM-assisted framework for model-based GUI testing of Vue.js single-page applications. Its central idea is to automatically reconstruct an executable page transition graph (PTG) from Vue source code and then use that graph to guide Playwright-based exploration. In the published literature, LLMVue is introduced as a framework that uses a large language model to generate a PTG from Vue.js source code, infers component hierarchies and route transitions, and merges them into a unified PTG across multiple single-file components (SFCs) [2606.27665]. It is presented as the first use of LLMs for model-based GUI testing of Vue.js applications using source-level PTG extraction [2606.27665].

## 1. Problem setting and research motivation

LLMVue addresses a specific difficulty in automated GUI testing for Vue.js web applications: transition candidates are spread across router configurations and SFCs, and they must be concretized and normalized into an executable PTG for testing [2606.27665]. The motivating observation is that Vue applications encode navigation in a distributed and framework-specific way. Route definitions live in router configuration, but actual navigation actions may be triggered in SFCs, inside event handlers, inside shared layouts, or through UI-library abstractions such as Element UI/Plus menus and dropdowns [2606.27665].

The framework is situated within model-based testing. The paper argues that classic crawler-based or URL-only modeling is too weak for Vue.js single-page applications, because the navigation structure is fragmented across many files and runtime conditions [2606.27665]. In this setting, a usable navigation model must be both complete enough to avoid missed routes and precise enough to drive executable actions. A representative difficulty arises when two different menu items both lead to `/book` but are guarded by mutually exclusive role checks; if these are collapsed into a single edge, testing may repeatedly try to click a hidden element and waste exploration budget [2606.27665].

Three technical difficulties are emphasized. First, navigation is split across router configs and SFCs. Second, UI-library abstractions hide real clickable elements, so source-level extraction must be refined into Playwright-ready locators. Third, runtime conditions create multiple versions of the same page transition, meaning that a graph that ignores those conditions becomes misleading for testing [2606.27665]. In this sense, LLMVue is not merely a route extractor; it is a framework for producing an executable, selector-annotated PTG that preserves conditional distinctions relevant to GUI testing.

## 2. Formal model and end-to-end architecture

LLMVue constructs a final PTG $G_3$ in three stages, starting from source code $C$ of a Vue SPA. The PTG is defined as a directed labeled graph $G=(V,E)$, where $V$ is the set of route nodes and $E$ is the set of navigation edges; each node is a route path from the router, such as `/home` or `/user/add` [2606.27665].

Each edge includes source and target routes, an executable selector, an event type such as `click` or `redirect`, and an optional visibility or guard condition [2606.27665]. The extracted graph is denoted $G_{ext}$, and the manually constructed ground-truth graph is denoted $G_{gt}$ [2606.27665]. The staged pipeline produces three intermediate graph forms:

- $G_1 = (V_1, E_1)$: a recall-oriented candidate graph  
- $G_2 = (V_2, E_2)$: an LLM-refined graph  
- $G_3 = (V_3, E_3)$: a normalized final PTG [2606.27665]

This architecture is explicitly hybrid. The paper attributes its effectiveness to the combination of high-recall static extraction for broad coverage, LLM reasoning for selector enrichment and condition interpretation, and deterministic normalization for precision and usability [2606.27665]. A plausible implication is that LLMVue treats the LLM as a semantic refiner embedded within a larger static-analysis and normalization workflow, rather than as a stand-alone extractor.

## 3. Three-stage PTG construction pipeline

Stage I is static graph extraction and is designed for high recall rather than precision. It parses the Vue router configuration, all Vue SFCs, and ASTs of templates and scripts [2606.27665]. From the router configuration it builds a route table containing valid route paths and mapped components, explicit redirects, and parent-child relations for nested routes [2606.27665]. It also computes how many times each parent route appears in the route config, to avoid incorrectly propagating child routes to the wrong parent layout [2606.27665].

From SFC templates and scripts, the framework extracts navigation facts including `router-link` targets, menu-item `to` or `index` values, event bindings, router API calls such as `$router.push` and `$router.replace`, and other navigation-like patterns such as `window.open` and `router.resolve` [2606.27665]. A key auxiliary structure is the component usage index, which maps each component file to all route paths under which it may be rendered, including direct route bindings and transitive usage inferred from component imports and template references [2606.27665]. This is essential because a layout component may render children from nested routes even though the clickable menu lives in the shared parent layout [2606.27665].

Stage I produces two major edge types: router-level edges, where each explicit redirect becomes an edge with `selector="-"` and `event=redirect`, and component-level edges, where extracted navigation facts are converted into candidate edges for each associated component-route pair [2606.27665]. Nested routing is handled by propagating layout-derived menu edges to child routes, while ambiguous widgets and dynamic rendering are deliberately not filtered out at this stage [2606.27665]. The result is $G_1$, a high-recall but noisy candidate PTG [2606.27665].

Stage II is LLM-based semantic refinement and uses GPT-4o as an enhancement-only semantic refiner [2606.27665]. It does not delete edges; it preserves the endpoints and event type, and only improves the selector and condition fields [2606.27665]. Given source file context, the LLM augments selectors with text cues, normalizes framework-specific rendering, extracts conditions and guards from `v-if`, `v-show`, and conditional menu construction in scripts, and preserves recall through validation and fallback [2606.27665]. The paper states that the LLM must return the same number of edges as it received, that output is validated against edge count, required fields, and route validity after normalization, and that the system retries up to three times with reflection prompts if parsing or validation fails [2606.27665]. If all retries fail, the original Stage I edges are kept [2606.27665].

Stage III is graph normalization and turns $G_2$ into the final executable PTG $G_3$ through deterministic cleanup [2606.27665]. The normalization operations include post-processing selectors and conditions, selector-level deduplication, route-pair preference filtering, filtering route-table structural edges and wildcard auto-redirect edges, deduplicating submenu-prefix variants for Element UI/Plus, filtering cross-group edges whose branch labels disagree, and filtering most self-loops while retaining some UI-structure self-loops needed for submenu or dropdown expansion [2606.27665]. Two edges are considered duplicates if they share the same `from`, `to`, `event`, and `selector`; the condition field is not part of the deduplication key [2606.27665].

## 4. Implementation details and executable testing workflow

The LLM backbone used in LLMVue is GPT-4o, with temperature 0 for deterministic extraction [2606.27665]. The PTG extraction module is implemented in Java JDK 17, while graph and runtime data collection use Node.js v18.20.8 and npm 10.8.2; the browser is Google Chrome 143.0.7499.194, and the exploration engine is Playwright [2606.27665]. Because of token-budget constraints, Stage II groups edges by source file and splits them into batches of 30, with a system prompt enforcing enhancement-only behavior and a user prompt containing source code and JSON edge lists [2606.27665].

The framework is evaluated on ten open-source Vue.js projects from GitHub: Vue-Springboot-Library, questionnaire, dormitory_springboot, music-website, Blog, COVID, VehicleMangement-Web, UniTakeout, stu_exam_sys, and house_rental_management [2606.27665]. Their sizes range roughly from 10 to 29 SFCs, with route counts from 7 to 24, and frontend LOC from about 2.2k to 10.2k [2606.27665]. These projects were selected after querying GitHub for Vue.js SPA repositories and then manually filtering for systems that are Vue.js SPAs with Vue Router, have non-trivial route-level navigation, and are buildable and runnable end-to-end under version-locked snapshots [2606.27665].

Ground-truth PTGs were built manually using a deterministic, version-locked expert protocol co-developed with Vue/Element UI practitioners [2606.27665]. The labeling rules include script-based routing, router-enabled menu navigation, post-submit routing, and submenu expand or collapse as explicit self-loops; they exclude actions that do not change routes; they use stable rendered DOM classes with `:has-text` constraints; and they prioritize script routing over inherent menu navigation when both exist [2606.27665]. To reduce annotator variance, the study used calibration examples, centralized arbitration, and automated sanity checks [2606.27665].

For testing effectiveness, the only baseline is random exploration under the same action and time budget, used as a lower-bound comparator rather than against other GUI-testing tools [2606.27665]. The testing budgets are 60s, 300s, and 600s, and the experiment is repeated 10 times per configuration [2606.27665]. The testing metrics are Action Number (AN), Statement Coverage (SC), and Page Coverage (PC) [2606.27665]. The graph-quality metrics are precision, recall, F1-score, and Graph Edit Distance (GED) [2606.27665].

## 5. Empirical performance and observed error patterns

Across the ten projects, LLMVue achieved average precision of 94.57%, average recall of 88.79%, average F1 of 91.25%, and average GED of 16 [2606.27665]. Per-project F1 ranged from 86.51% to 97.33%, with 5 of 10 projects achieving F1 greater than or equal to 90% and 3 of 10 exceeding 95% F1 [2606.27665]. Precision was usually very high, often perfect, while recall varied more strongly due to conditional navigation and callback-driven routes [2606.27665]. Reported examples include P1 with 98.65% precision, 96.05% recall, 97.33% F1, and GED 4; P2 with 100% precision, 93.33% recall, 96.55% F1, and GED 2; P3 with 100% precision, 94.04% recall, 96.93% F1, and GED 13; and P10 with 76.60% precision, 100% recall, 86.75% F1, and GED 22 [2606.27665].

For GUI exploration, the testing study focused on P1, which had the highest extraction quality with F1 = 97.33%, and P3, which had the largest PTG and was structurally most complex [2606.27665]. LLMVue-guided exploration executed substantially more actions than random exploration under the same time budget. On P1, the action counts were 95.8 versus 60.1 at 60s, 518.0 versus 310.6 at 300s, and 1038.3 versus 658.8 at 600s; on P3, they were 1011.8 versus 606.5 at 300s and 1911.1 versus 1158.7 at 600s [2606.27665]. The paper also reports lower variance, indicating more stable throughput [2606.27665].

Page coverage improved earlier under PTG guidance. On P1, PC rises from 54% to 67% at 60s and reaches 70% quickly, while random exploration reaches that level later [2606.27665]. On P3, PTG guidance yields sustained page-coverage gains across budgets [2606.27665]. Statement coverage also improves with longer budgets: on P1, SC is consistently better and the gap grows with budget, while on P3, SC can lag slightly at short budgets but becomes better at longer budgets [2606.27665]. The stated explanation is that PTG guidance improves route discovery efficiency by avoiding redundant interactions, reaching new pages earlier, and reducing time wasted on ineffective actions [2606.27665]. Page coverage improves first because PTGs are naturally about navigation, whereas statement coverage improves later because many statements are executed through intra-page widget interactions that do not change routes [2606.27665].

The mismatch analysis identifies two main error families. False negatives mainly come from conditional and callback-driven routing, including navigation only after successful validation, routing inside asynchronous callbacks or confirmation dialogs, and missing links when the route call is not adjacent to the triggering widget [2606.27665]. False positives mainly come from selector ambiguity and cross-role noise, including generic menu selectors without enough text constraints and role-dependent menus where some clickable entry points are excluded from the ground truth because they are not stable under a fixed role context [2606.27665].

## 6. Limitations, future directions, and terminological context

The paper discusses several threats to validity. Ground truth remains ambiguous in some corner cases, especially condition-controlled widgets and callback-driven routing [2606.27665]. Edge-level metrics do not directly capture UI robustness or intra-page state diversity [2606.27665]. Browser execution and asynchronous rendering can vary run to run, although extraction uses deterministic decoding and testing was repeated multiple times [2606.27665]. Generalizability is limited to Vue.js SPAs with Vue Router and may not directly transfer to other frameworks or architectural styles [2606.27665].

The proposed future directions are closed-loop extraction and testing, role-aware and state-aware PTGs, coverage-aware intra-page exploration, and better selector disambiguation using richer DOM context such as ancestor chains, ARIA tags, or stable component IDs [2606.27665]. This suggests that LLMVue is best understood as a foundation for more context-aware testing of modern frontend frameworks rather than as a complete solution to GUI exploration.

The term “LLMVue” can be confused with LLM-based work on visualization or visual artifact generation, but the named framework in the literature refers specifically to Vue.js GUI testing [2606.27665]. Related but distinct lines of research include VISTA, a multi-agent framework for generating mathematically correct visual aids and aligned math problems for geometry and function tasks [2411.05423]; SimVecVis, a dataset and structured representation for enhancing MLLMs in chart understanding [2506.21319]; LSLE, a literacy-stratified framework for using LLMs as simulated evaluators of visualizations [2606.10095]; and later experiments arguing that frontier LLMs possess visualization literacy but still struggle with instruction following and graphical integrity in visualization evaluation [2606.15136]. A common misconception is therefore to treat “LLMVue” as a generic label for LLM-based visualization systems. The published evidence instead distinguishes a specific Vue.js testing framework from several separate research programs on visualization generation, visualization understanding, and visualization evaluation.

Source: https://www.emergentmind.com/topics/llmvue