VISTA: Visual Spec-to-App Benchmark

• VISTA is a benchmark that evaluates agents generating runnable multi-page web apps from mixed natural-language and visual specifications.
• It integrates Figma-derived designs, human annotations, and DOM-grounded as well as CLIP-based evaluations for both functional and visual fidelity.
• Empirical results highlight trade-offs between input richness, stack freedom, localization accuracy, and behavioral performance in UI generation.

Searching arXiv for the benchmark paper and a few related benchmarks mentioned in the provided material. {"query":"(Guo et al., 22 May 2026) VISTA benchmark visual spec to web app coding agents"} VISTA, short for VIsual Spec-To-App Benchmark, is an end-to-end benchmark and evaluation framework for “visual spec-to-web-app” coding agents: LLM-based systems tasked with building runnable multi-page web applications from mixed natural-language and visual specifications rather than isolated coding puzzles. It targets realistic UI-centric development in which agents must infer underspecified behavior, implement interactivity, survive build and runtime errors, and produce applications that are simultaneously functional and visually coherent. Its design combines ten Figma-derived web applications, five prompt-information conditions, human annotations, DOM-grounded evaluation, behavior-specific browser checks, and CLIP-based visual similarity, with the explicit goal of separating structural alignment, behavioral completeness, and visual fidelity (Guo et al., 22 May 2026).

1. Benchmark scope and task formulation

VISTA was introduced to address limitations in existing code benchmarks that focus on algorithmic or back-end tasks, evaluate agents in fixed stacks or single input modalities, rely heavily on LLM-as-judge or brittle browser scripts, and treat the model separately from the harness even though the harness significantly affects performance. In contrast, VISTA evaluates full coding agents operating in realistic development environments, where they inspect files, edit code, run builds or tests, and repair failures across complete web applications rather than code snippets (Guo et al., 22 May 2026).

The benchmark is organized around ten multi-page web applications with realistic UI and interaction patterns, derived from Figma designs. The task is to generate a runnable application from a product target in natural language, optionally augmented by reference screenshots and a pruned Figma JSON structure, under either free stack choice or a specified scaffold. The resulting application is expected to match intended UI structure and layout, implement behaviors such as navigation, inputs, toggles, and popouts, and remain executable after iterative repair. This makes VISTA substantially closer to front-end software engineering than to conventional program-synthesis tasks (Guo et al., 22 May 2026).

A central design choice is the use of five prompt-information conditions, defined over two axes: visual or structural fidelity, and stack constraint. The conditions are $C_0$ text only with free stack, $C_1$ text plus screenshots with three specified stacks, $C_2$ text plus screenshots with free stack, $C_3$ text plus screenshots plus pruned Figma JSON with a single specified stack, and $C_4$ text plus screenshots plus pruned Figma JSON with free stack. This allows independent study of how visual grounding, structural grounding, and implementation freedom interact (Guo et al., 22 May 2026).

The primary task metric is the combined structure-function score

$S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$

where $L_i \in [0,1]$ is a localization score for the matched DOM element corresponding to annotated interaction $i$, $B_i \in [0,1]$ is the behavior score for that interaction, and $N$ is the number of critical annotated interactions. This formulation makes correctness explicitly multiplicative: a well-localized but non-functional control, or a functional behavior assigned to the wrong element, both depress the score (Guo et al., 22 May 2026).

2. Dataset composition, annotations, and interfaces

VISTA includes 10 application categories: Newsletter, Real estate, Job board, Forum, Travel booking, Chat, Cloud storage, E-commerce, Project management, and Music streaming. Across these categories, the benchmark contains 128 pages, 3,253 interactive annotations, and 458 visual anchors. The applications cover dashboards, navigation menus, filters, tabs, lists and item detail views, search, forms, authentication-style flows, and e-commerce carts and checkouts, thereby stressing multi-page navigation, application state, and interaction logic rather than static page reconstruction alone (Guo et al., 22 May 2026).

Each page originates from a Figma design. The benchmark exports PNG screenshots for each page and a pruned Figma JSON structure with layout and hierarchy information. Raw Figma JSON is explicitly reduced by removing rendering-only metadata while retaining positions, frames, component-tree hierarchy, text labels, and interaction targets where possible. In $C_1$0 and $C_1$1, this pruned JSON functions as the structural input intended to help agents infer component boundaries, repeated patterns, and nesting (Guo et al., 22 May 2026).

Human annotation is foundational. Every page is annotated with controls that “should trigger a visible state change, navigation, or request,” including buttons, links, inputs, textareas, selects, menus, tabs, filters, toggles, and popouts. Each annotation includes a bounding box and an interaction type drawn from navigation, input, toggle, external link, popout, or generic click. In addition, each page has 2–5 visual anchors, labeled with stable IDs such as <search> or <checkout>, which are used for page-to-URL mapping, coordinate alignment, and robustness to layout shifts (Guo et al., 22 May 2026).

The benchmark also standardizes scaffolding and execution interfaces. Instead of large GitHub templates, it uses lightweight scaffolds such as create-next-app with TypeScript, Tailwind, ESLint, and modern routing. For each task, an LLM proposes three stacks, labeled pick A, B, and C; these support $C_1$2, and pick A is reused as the fixed template in $C_1$3. Agents run in harnesses comparable to a patch-oriented Codex harness or the Claude Code environment with Write, Edit, and MultiEdit, and VISTA logs file mutations, commands, tool usage, build or test attempts, and timestamps for later workflow analysis (Guo et al., 22 May 2026).

3. Evaluation methodology

VISTA’s evaluator combines three components: DOM-grounded reference matching and behavior tests as the primary measure, behavior-specific browser automation, and CLIP-based visual similarity. The central claim of the benchmark is that naive script-based testing is inadequate in open-ended UI generation, because small layout changes, selector differences, and alternate HTML structures can invalidate brittle scripts without reflecting true functional failure. VISTA addresses this by grounding evaluation in human annotations, spatial alignment, and semantic matching (Guo et al., 22 May 2026).

The first step is page-to-URL mapping. Generated applications may not preserve Figma page names or route structures, so the evaluator maps reference pages to app URLs by combining declared route mappings, crawling internal links, source route patterns, DOM signatures, visual anchors, and text cues. It then performs coordinate alignment between Figma-space annotations and rendered pages via a per-axis affine transform

$C_1$4

which tolerates global rescaling and shifting while preserving expectations about local relative placement (Guo et al., 22 May 2026).

After alignment, the evaluator extracts visible interactive DOM elements such as <a>, <button>, <input>, <select>, <textarea>, checkboxes, radio controls, and ARIA-role interactive elements, ignoring hidden or zero-size elements. Localization is computed through tiered matching. Exact IoU thresholds or center-distance thresholds yield partial credit, and a text-similarity fallback yields a small residual score. The tiers are: IoU $C_1$5 gives $C_1$6; IoU $C_1$7 gives $C_1$8; center distance $C_1$9 px gives $C_2$0; center distance $C_2$1 px gives $C_2$2; text similarity fallback gives $C_2$3; and a miss gives $C_2$4 (Guo et al., 22 May 2026).

Behavior scoring then triggers interactions on matched elements and checks expected outcomes. Navigation requires a URL change or meaningful page transition; input requires accepting text and firing input or change events; toggle requires state change such as checked or ARIA expanded or collapsed; popout requires opening a dialog or overlay; and external link requires exposing an external URL. VISTA also reports mean localization $C_2$5 and mean behavior $C_2$6, which makes it possible to separate structural from behavioral failures (Guo et al., 22 May 2026).

CLIP-based visual similarity is computed between screenshots of rendered generated pages and ground-truth Figma screenshots, using a CLIP model to embed both images and aggregate similarity over pages. A common misconception in visual UI evaluation is that screenshot similarity is sufficient. VISTA explicitly rejects that premise: CLIP is treated as complementary rather than primary, since high CLIP with low behavior can correspond to a visually plausible but non-functional UI, while high behavior with lower CLIP can indicate a functionally correct but visually inaccurate implementation (Guo et al., 22 May 2026).

4. Agents, harnesses, and experimental protocol

The benchmark evaluates four deployed agents drawn from two model families and two harnesses. The GPT family includes GPT-5.4 and GPT-5.5, both operating in a Codex-like harness characterized by patch-oriented editing, shell commands, and batch file changes. The Claude family includes Claude Sonnet and Claude Opus, both operating in the Claude Code harness with Write, Edit, and MultiEdit tools. VISTA evaluates these systems as full agents rather than raw LLMs, meaning they can inspect files, modify files, run builds or tests, and iterate on failures (Guo et al., 22 May 2026).

All ten tasks are run under all five prompt conditions $C_2$7 through $C_2$8. In $C_2$9, each task is solved once per pick A, B, and C stack, and results are averaged, with per-pick results reported separately. For every task, condition, and model combination, the agent is allowed a full end-to-end development cycle. Reported metrics include mean localization, mean behavior, combined $C_3$0, median $C_3$1, and CLIP similarity (Guo et al., 22 May 2026).

VISTA also instruments the process of software generation itself. It logs reads, writes, edits, deletes, commands, and temporal action sequences, enabling an analysis of editing style. The benchmark introduces the Surgical Diff Score: $C_3$2 where $C_3$3 is edit-event share, $C_3$4 is diff-byte share, and $C_3$5 is targetedness. It also defines a stricter variant,

$C_3$6

which penalizes agents that rarely use edit operations. These metrics are not quality metrics for the task itself; they quantify process style, especially the degree to which an agent rewrites files versus making localized edits (Guo et al., 22 May 2026).

Workflow trajectories show a recurrent inspect–write–verify rhythm, with family-level differences. Claude models exhibit stronger inspect-to-inspect transitions and high failure-to-inspect transitions, while GPT models show more diverse transitions after failures and verification. The benchmark’s interpretation is cautious: these workflow grammars are descriptive of process, not direct indicators of task success (Guo et al., 22 May 2026).

5. Empirical results and performance patterns

Aggregate results across prompt conditions reveal a consistent but nontrivial interaction between input richness and stack freedom. $C_3$7 yields localization $C_3$8, behavior $C_3$9, combined $C_4$0, and median $C_4$1. Adding screenshots while pinning stacks in $C_4$2 raises localization to $C_4$3 but lowers combined $C_4$4 to $C_4$5. Freeing the stack in $C_4$6 raises combined $C_4$7 to $C_4$8, the best mean combined score. The richest input with free stack, $C_4$9, gives localization $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$0, behavior $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$1, combined $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$2, median $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$3, and the best CLIP score, $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$4 (Guo et al., 22 May 2026).

These results motivate several conclusions. Screenshots improve localization: $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$5 raises localization by 18%, from $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$6 to $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$7. Free stack choice significantly boosts combined performance: $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$8 changes combined $S = \frac{1}{N} \sum_{i=1}^{N} L_i \cdot B_i,$9 from $L_i \in [0,1]$0 to $L_i \in [0,1]$1, and $L_i \in [0,1]$2 changes it from $L_i \in [0,1]$3 to $L_i \in [0,1]$4. Figma structure yields the highest localization values, with $L_i \in [0,1]$5 and $L_i \in [0,1]$6. However, richer structural input does not automatically improve functional performance, since $L_i \in [0,1]$7 remains below $L_i \in [0,1]$8 on combined $L_i \in [0,1]$9 despite additional Figma JSON (Guo et al., 22 May 2026).

Per-model results are even more revealing. GPT-5.5 has the best localization, $i$0, and best CLIP, $i$1, but the worst behavior, $i$2, and the lowest combined $i$3, $i$4. Claude Opus has the best combined $i$5, $i$6, the best median $i$7, $i$8, and the best behavior, $i$9, while its CLIP score is $B_i \in [0,1]$0, not the best. GPT-5.4 and Sonnet occupy intermediate positions. This establishes a central empirical point of the benchmark: visual fidelity and functional correctness are partially decoupled across both models and prompt conditions (Guo et al., 22 May 2026).

Stack choice within $B_i \in [0,1]$1 is not neutral. Pick A is consistently best, with localization $B_i \in [0,1]$2, behavior $B_i \in [0,1]$3, combined $B_i \in [0,1]$4, median $B_i \in [0,1]$5, and CLIP $B_i \in [0,1]$6, outperforming picks B and C. This justifies the reuse of pick A as the fixed stack scaffold in $B_i \in [0,1]$7 (Guo et al., 22 May 2026).

Editing-style analysis shows that high quality does not imply surgical editing. Opus, which has the best combined score, has the lowest Surgical Diff score, $B_i \in [0,1]$8, and rewrite share $B_i \in [0,1]$9. GPT-5.5, which has the lowest combined score, has the highest Surgical Diff score, $N$0, and Strict Diff score, $N$1. The correlation between Surgical Diff and combined $N$2 is low and slightly negative, $N$3, and the Strict Diff correlation is $N$4. A plausible implication is that localized editing is not a reliable proxy for end-task quality in agentic web-app generation (Guo et al., 22 May 2026).

6. Interpretation, relation to prior benchmarks, and significance

VISTA positions itself between design-to-code and agentic software-engineering benchmarks. Relative to SWE-bench, SWE-agent, and OpenHands, it shifts attention from real GitHub issues and general software engineering toward visual specs, layout, multi-page UI, and design fidelity. Relative to WebArena, VisualWebArena, and OSWorld, it changes the task from acting in existing web or operating-system environments to building new applications from scratch. Relative to Design2Code, it extends static design reconstruction into multi-page, interactive, end-to-end application generation (Guo et al., 22 May 2026).

A recurrent misconception in evaluation of coding agents is that app quality can be read off from a single scalar such as screenshot similarity, build success, or edit efficiency. VISTA systematically argues against this reduction. Its results show that structural fidelity, functional behavior, and overall visual similarity can move independently; that rich structural input can improve localization without improving behavior; that stack familiarity can dominate the effect of better specifications; and that process-level measures such as rewrite share or targeted edits are largely orthogonal to outcome quality (Guo et al., 22 May 2026).

The benchmark’s larger significance lies in reproducibility and extensibility. It fixes designs, annotations, and scaffolds; formalizes a structure-function metric $N$5; augments it with CLIP, localization, behavior, and editing-style analyses; and supports future addition of new stacks, more complex applications, and extra evaluation dimensions such as accessibility, performance, and robustness. In that sense, VISTA establishes a rigorous and reproducible foundation for studying agent-based front-end software engineering under mixed visual, structural, and textual specifications (Guo et al., 22 May 2026).