LLM-Assisted Model-Based GUI Testing for Vue.js Web Applications

Abstract: Vue.js is a popular framework for building modern web applications. As Vue.js functionality and tooling support grow, ensuring its reliability (through automated testing) is becoming increasingly important. Although model-based testing has been successfully used to automate graphical user interface (GUI) testing on other platforms, its application to Vue.js remains challenging: Transition candidates, which are spread across router configurations and single-file components (SFCs), must be concretized and normalized into an executable page transition graph (PTG) for testing. To address this, we propose the LLMVue framework, which uses a LLM to generate a PTG from Vue.js source code. LLMVue infers component hierarchies and route transitions, merging them into a unified PTG across multiple SFCs. We evaluated LLMVue on a collection of ten open-source Vue.js projects from GitHub, using GPT-4o as the LLM backbone. The constructed graphs demonstrate high precision and recall, with low graph edit distance. LLMVue -guided testing also significantly improves the coverage and exploration efficiency, compared to a random exploration baseline (with the same time constraints). To the best of our knowledge, this is the first use of LLMs for model-based GUI testing of Vue.js applications using source-level PTG extraction.

• The paper demonstrates a novel framework, LLMVue, that leverages LLMs to automatically extract source-level page transition graphs for Vue.js applications.
• LLMVue integrates static graph extraction, semantic refinement with LLMs, and normalization to deliver actionable selectors and improve testing coverage.
• Experimental results show up to 67% increases in action throughput and significant gains in page and statement coverage over random exploration.

LLM-Assisted Model-Based GUI Testing for Vue.js Applications: The LLMVue Framework

Introduction

Model-based testing (MBT) enables systematic test generation for GUI-based software by constructing explicit navigation models such as Page Transition Graphs (PTGs). Vue.js, as a widely adopted reactive frontend framework, presents nuanced challenges for GUI test modeling due to its dynamically routed single-page architecture, use of single-file components (SFCs), and integration with UI libraries. Traditional MBT approaches, while effective on classical web applications, have limited applicability for Vue.js because actionable GUI transitions are fragmented and heavily context-dependent. The paper "LLM-Assisted Model-Based GUI Testing for Vue.js Web Applications" (2606.27665) introduces LLMVue, an end-to-end framework employing LLMs, specifically GPT-4o, to automatically extract source-level PTGs from Vue.js code bases, thus facilitating automated GUI testing with high efficacy.

Methodology

Pipeline Overview

LLMVue employs a structured three-stage pipeline:

1. Static Graph Extraction: Parses Vue.js SFCs and router configurations into a conservative graph over-approximating all plausible navigation transitions, prioritizing recall. Cross-component and nested routing relationships are uncovered, including those encapsulated within UI libraries.
2. LLM-Based Semantic Refinement: An LLM enhances candidate edges, synthesizing precise executable selectors and inferring guarding conditions from static and dynamic template/script analysis. This transformation yields selectors compatible with Playwright for automation and annotates runtime-dependent navigability (e.g., role-based menu entries).
3. Graph Normalization: Deduplicates redundant transitions, removes structural or global redirect artifacts, and filters out spurious or ambiguous edges, producing an actionable PTG for MBT.

   Figure 1: High-level view of the LLMVue pipeline, integrating static analysis with LLM-based enrichment and normalization for Vue.js PTGs.

Handling Vue-Specific Navigation Semantics

LLMVue's extraction phase traverses router and component ASTs, reconciles direct and transitive component-route associations, and manages ambiguities arising from role-guarded UI elements, dynamic menu rendering, and nested route layouts. The LLM augmentation stage plays a critical role in refining these edges, moving selectors from syntactic imprecision (such as .el-menu-item) to actionable locators with conditional annotations (e.g., :has-text('Book Management') conditioned on user.role == 1).

Edge normalization employs heuristics to condense occurrences of functionally equivalent selectors, discards implementation artifacts, and prunes self-loops unless structurally meaningful for GUI expansion sequences.

Experimental Evaluation

Datasets and Ground Truth

Ten open-source Vue.js applications with non-trivial navigation logic were selected, with ground-truth PTGs annotated via a deterministic protocol by Vue/Element UI practitioners. This base ensured comprehensive coverage of observable navigation behaviors and enabled precise measurement of PTG extraction accuracy.

Metrics

• Structural Metrics: Precision, recall, F1, and Graph Edit Distance (GED), measured at the edge level for PTG quality.
• Testing Metrics: Action Number (AN), Statement Coverage (SC), and Page Coverage (PC), assessing GUI testing effectiveness when guided by LLMVue-constructed PTGs versus random exploration under fixed time budgets.

Results

PTG Extraction Quality

LLMVue achieves strong average results across all projects: precision of 94.57%, recall of 88.79%, and F1-score of 91.25%. The GED analysis indicates that generated graphs are structurally close to expert-annotated ground truths for the majority of cases, with significant deviations largely attributed to highly conditional or callback-driven navigation logic.

Key findings:

• Highest F1-scores (≥95%) are observed when transitions are statically analyzable and role/state conditions are expressible as template-level constraints.
• Lower recall in some cases is traced to deep callback chains and menu items only accessible under specific runtime states, revealing opportunities to enhance control/data-flow integration in extraction.
• Precision remains high across all projects, suggesting effective spurious-edge suppression by the LLM and normalization pipeline.

GUI Testing Effectiveness

LLMVue-guided testing consistently outperforms random exploration:

• Action throughput (AN): Substantial increases (e.g., +59% to +67% under 5–10 minute budgets) due to structurally prioritized navigation.
• Page Coverage (PC): Rapid saturation to maximum reachable pages in low to medium budgets, with gains up to +13 percentage points in short runs.
• Statement Coverage (SC): Pronounced advantages with increasing testing budgets, particularly in projects where navigation breadth correlates with deeper intra-page execution.

Figure 2: Relative gains in action number (AN) for LLMVue-guided over random exploration, illustrating consistent improvement in exploration efficiency.

Notably, coverage improvements are more pronounced in projects where route transitions closely map to substantive functionality distribution. In projects with substantial intra-page, non-route-triggered logic, the marginal effect on SC is delayed until PTG guidance increases diversity in page visits.

Discussion and Implications

LLM-Augmented Extraction

The hybrid approach leverages LLMs' capacity for cross-context reasoning and transformation, overcoming limitations of rule-based static analysis in capturing context-dependent, dynamic Vue.js navigation. High recall in extraction ensures exploration is not bottlenecked by missed transitions, while LLM-augmented selectors and conditions mitigate the risk of unexecutable tests.

Practical Applicability

The PTGs generated by LLMVue are directly usable in GUI test engines, yielding high initial coverage and action efficiency—an advantage in regression test automation and CI/CD validation for rapidly evolving Vue.js codebases. The approach is robust in handling multi-role, multi-condition navigation, a common pattern in business-critical web applications.

Limitations and Future Directions

• Conditional Path Under-Approximation: Missed transitions tied to runtime data or deep async/callbacks highlight the need for further integration of dynamic analysis or refinement via execution traces.
• Role/State Context Modeling: Current PTGs annotate but do not fully partition transitions by user roles or authentication states. Role-aware subgraph modeling and adaptive exploration policies could enhance both coverage and bug-finding potential.
• Intra-Page Widget Exploration: While LLMVue expedites path diversity at page granularity, full statement coverage in widget-rich pages requires integration with systematic in-page action selection and coverage feedback.

Conclusion

LLMVue demonstrates that LLM-assisted model-based GUI testing is both feasible and effective for complex, real-world Vue.js applications. By unifying static analysis and deep semantic enrichment, it achieves high PTG fidelity and practical gains in automated GUI coverage and test efficiency. The architectural generality of the framework suggests similar methodologies can be extended to other reactive frontend systems. Addressing identified limitations—particularly regarding conditional navigation, context partitioning, and intra-page action selection—will further enhance the theoretical rigor and applied practicality of LLM-assisted MBT in modern web application testing.