WebIGBench: Interactive Webpage Code Benchmark

• WebIGBench is a benchmark that assesses multimodal language models on generating executable, interactive webpage code from visual and textual inputs.
• It measures both functional fidelity and visual alignment using metrics like SR, ACR, and ASTC to ensure that generated webpages reproduce intended interaction logic.
• The benchmark features a detailed execution model with human-validated action sequences across five types of interactions, enabling reproducible and comprehensive performance evaluation.

WebIGBench is a benchmark for evaluating multimodal LLMs on code generation for complex interactive webpages, introduced in "Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages" (Wu et al., 29 May 2026). It is designed to address a gap in webpage-to-code evaluation: modern web applications are dynamic, stateful, and interaction-driven, whereas earlier benchmarks largely emphasized static layout reconstruction, visual fidelity, or code structure. WebIGBench therefore centers evaluation on whether generated code reproduces the intended interaction logic of a reference webpage across multi-step trajectories, rather than merely producing visually similar outputs (Wu et al., 29 May 2026).

1. Conceptual scope and problem formulation

WebIGBench targets end-to-end generation of interactive webpages from multimodal inputs. The paper models an interactive webpage as a sequence of states $S_0 \ldots S_n$ induced by an action sequence $a_0 \ldots a_n$ and observed through screenshots $I_0 \ldots I_n$ (Wu et al., 29 May 2026). In its basic formulation, the generation task is

$$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$

To reduce ambiguity for visually implicit actions such as key presses and scrolling, the benchmark also considers an augmented formulation:

$$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$

The central evaluative concept is interaction consistency. In the benchmark’s definition, interaction consistency means that a generated webpage exhibits the same interaction-driven behavior as the reference, step by step, rather than only resembling it visually. The paper distinguishes two facets. Behavioral alignment requires that the generated action trajectory match the reference sequence semantically and functionally, such as a click revealing content or an input triggering search. State alignment requires that each reference state transition be realized at the correct step in the generated page (Wu et al., 29 May 2026).

This framing makes WebIGBench a benchmark of executable behavioral fidelity. A common misconception in webpage generation is that high visual similarity implies a successful reconstruction; WebIGBench is explicitly built around the contrary observation that many models can produce pages that “look right” while failing to bind events, manage state, or manipulate the DOM correctly (Wu et al., 29 May 2026).

2. Dataset composition and interaction coverage

WebIGBench contains 103 unique interactive webpages curated from popular real-world domains sampled via Cloudflare Radar, spanning 15 categories. The paper lists examples including Technology at 15.5%, E-commerce at 13.6%, Video Streaming, AI, Social Media, Jobs, Tools, and Blogs. Across these webpages, the benchmark includes 871 annotated interactive actions, averaging about 8.5 actions per page, which the paper contrasts with an interaction density of approximately 2.9 in Interaction2Code (Wu et al., 29 May 2026).

Interaction paths are human-authored and goal-oriented. Each task begins with a precise online location anchor, namely a URL used for reproducibility. The steps are written as clear natural-language instructions specifying the action and target element, and the paths are intended to capture multi-step, state-dependent behaviors typical of modern web applications (Wu et al., 29 May 2026).

The five official action types covered by the reported statistics are summarized below.

Action type	Count or status	Example from the benchmark description
Click	671	Clicking a tab to reveal a panel; clicking a filter chip to apply a constraint
Input	76	Typing a query in a search box
Scroll	56	Scrolling to reveal a footer region or lazy-loaded content
Key press	24	Pressing Enter to execute a search; sending keys while holding a modifier key
Switch-tab	Present; count not explicitly stated	Selecting a different tabbed section in a multi-view UI

The action list used by agents includes primitives such as click_element_by_index, input_text, send_keys, scroll, and done, which is used for task termination. Hover and drag are not part of the five official types covered by WebIGBench’s reported statistics, although the prompts mention general event-driven changes (Wu et al., 29 May 2026).

Validation is performed by a UI agent that executes the human-authored steps end to end. The collection pipeline records structured behavioral logs, synchronized screenshots after each step, and a full-session video. The agent-extracted actions are validated through human agreement, with Cohen’s $\kappa = 0.853$, and through a cross-LLM robustness study; the paper later reports cross-LLM agreement of approximately 90.4% as a mitigation against evaluator noise (Wu et al., 29 May 2026).

3. Data collection, execution model, and automated evaluation

The benchmark’s executor is built on the open-source browser-use tool. The agent operates in a closed loop comprising perception and decision, execution, and capture. First, an MLLM-based agent reads the current browser state and the next instruction and decides which action to take. Second, the tool translates abstract actions such as click_element_by_index into concrete browser operations. Third, hooks trigger a screenshot after each action so that behavioral and visual data, including timestamps, are aligned step by step (Wu et al., 29 May 2026).

The public release includes interaction traces consisting of URLs, action sequences, and metadata, together with the evaluation pipeline, but it does not re-host third-party HTML or media assets. This design is tied to the benchmark’s use of anchors: generated HTML is served through a local HTTP server so that the UI agent can access reference and generated pages in the same way (Wu et al., 29 May 2026).

The evaluation pipeline has four stages. First, anchor replacement via a local HTTP server exposes generated HTML as a URL. Second, UI-agent execution on the generated page produces a generated action and screenshot sequence. Third, action semantic matching aligns the reference action list with the generated one using a deterministic LLM matcher with temperature set to 0; each reference step $i$ is assigned either to a generated step $\pi_t(i)$ or to an unmatched symbol $\bot$. Fourth, the benchmark computes functional, visual, and efficiency metrics from the resulting match sequence (Wu et al., 29 May 2026).

The functional metrics are the benchmark’s distinguishing feature:

$$\mathrm{SR} = \frac{1}{|\mathcal{T}|}\sum_{t\in\mathcal{T}} \mathbb{1}[s_t=\text{success}],$$

$a_0 \ldots a_n$0

$a_0 \ldots a_n$1

Here, SR measures task-level success, ACR measures per-step completion, and ASTC measures agent effort relative to the reference path length, clipped to the interval $a_0 \ldots a_n$2. The hard step budget is $a_0 \ldots a_n$3; if a run exceeds it, the evaluation returns a Step Limit Exceeded error (Wu et al., 29 May 2026).

WebIGBench also reports visual metrics, but these are treated as orthogonal to functional correctness. For $a_0 \ldots a_n$4, the benchmark defines Match and Full aggregations. Match aggregation averages over aligned steps only, whereas Full aggregation preserves original order and charges failures to the score by using $a_0 \ldots a_n$5 (Wu et al., 29 May 2026). CLIP is cosine similarity between OpenAI CLIP ViT-B/32 image embeddings; SSIM is structural similarity on greyscale images resized to a common resolution; BLEU is sentence-level BLEU-4 computed on text extracted with EasyOCR (Wu et al., 29 May 2026).

Efficiency metrics are reported as ATTokC and ATTimC. ATTokC is provided for prompt, completion, and total tokens, and ATTimC is reported separately for generation and evaluation phases. There is no code structure metric such as DOM tree edit distance in this release; the benchmark emphasizes functional behavior and visual alignment instead (Wu et al., 29 May 2026).

4. Experimental configuration and empirical findings

The benchmark evaluates several representative MLLMs. The closed-source set consists of GPT-5, Gemini-2.5-Pro, and Grok-4. The open-weight set consists of InternVL-3.5-241B, Qwen-VL-Max, and GLM-4.5V. Prompting strategies include Direct, which uses screenshots plus textual instructions, and Action, which adds a structured action list. An ablation on GPT-5 also evaluates Visual and Action+Visual prompting (Wu et al., 29 May 2026).

The output constraint is deliberately narrow. Models must generate self-contained single-file HTML/CSS/JavaScript running in a single rendering process. Modern component frameworks such as React and Vue, multi-file projects, server back-ends, and multi-page navigation are explicitly out of scope by design (Wu et al., 29 May 2026). The evaluation environment uses deterministic decoding with temperature $a_0 \ldots a_n$6 for all MLLM calls, maximum context window, default API parameters otherwise, EasyOCR, CLIP ViT-B/32, and a browser-use UI agent with local HTTP anchors (Wu et al., 29 May 2026).

The results show that WebIGBench is difficult. The strongest reported configuration, Gemini-2.5-Pro with Action prompting, reaches $a_0 \ldots a_n$7, $a_0 \ldots a_n$8, and $a_0 \ldots a_n$9, together with Match-CLIP of approximately $I_0 \ldots I_n$0 and Match-SSIM of approximately $I_0 \ldots I_n$1 (Wu et al., 29 May 2026). Under Direct prompting, Gemini-2.5-Pro falls to $I_0 \ldots I_n$2 and $I_0 \ldots I_n$3, indicating that explicit action guidance materially improves functional performance (Wu et al., 29 May 2026).

That trend is consistent across models. For GPT-5, Action prompting increases SR from $I_0 \ldots I_n$4 to $I_0 \ldots I_n$5, increases ACR from $I_0 \ldots I_n$6 to $I_0 \ldots I_n$7, and reduces ASTC from $I_0 \ldots I_n$8 to $I_0 \ldots I_n$9 (Wu et al., 29 May 2026). The paper therefore treats structured action sequences as a practical means of reducing ambiguity in interactive code generation.

The modality ablation on GPT-5 further sharpens this conclusion. The reported sequence is:

• Direct: $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$0, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$1, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$2
• Action: $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$3, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$4, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$5
• Visual: $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$6, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$7, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$8
• Action+Visual: $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, P_{\mathrm{dyn}}).$$9, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$0, $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$1

Action performs best overall, while Visual alone improves Match-level visual scores, including Match-CLIP of approximately $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$2, without producing the highest functional success (Wu et al., 29 May 2026).

A central empirical finding is the decoupling of visual fidelity and behavioral correctness. Under Direct prompting, top models can obtain Match-CLIP in the $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$3 to $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$4 range while achieving SR only in the $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$5 to $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$6 range. Full visual metrics are harsher than Match metrics because they preserve failed or missing steps rather than averaging only over successful alignments (Wu et al., 29 May 2026). This is the benchmark’s strongest evidence that visual similarity is not a sufficient proxy for interactive equivalence.

5. Error taxonomy, failure modes, and interpretive significance

WebIGBench uses deterministic rules to categorize failures. The dominant outcome categories are Step Limit Exceeded and Functional Deviation. A fine-grained root-cause classifier identifies Missing Element / Selector as the largest source of failed runs, accounting for 45.4% of failures overall (Wu et al., 29 May 2026). This points directly to difficulties in wiring dynamic widgets to the correct handlers or identifiers.

The paper also reports model-specific pathologies. For GLM-4.5V, Page Blank / Not Loaded constitutes approximately 21–23% of failures, which the authors associate with runtime initialization issues (Wu et al., 29 May 2026). Action prompting most directly reduces Missing-Element failures on strong models; for GPT-5, the number of such cases drops from 46 to 15 when moving from Direct to Action prompting (Wu et al., 29 May 2026). This suggests that action lists help models produce more operational event-binding and selector logic even when aggregate visual fidelity does not increase proportionally.

Failure analysis by first interaction shows that clicks dominate numerically, reflecting their frequency in the benchmark, but non-click actions such as input, scroll, and key press also constitute a non-negligible share of initial failures (Wu et al., 29 May 2026). This is important because many earlier evaluations of interactive webpage generation reduced interactivity to single-step or click-dominant events; WebIGBench’s broader action coverage exposes a wider class of implementation failures.

The benchmark also analyzes interaction step length. Longer paths degrade visual fidelity, with mean Spearman $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$7 for Match-CLIP across configurations, but have no consistent effect on ACR, for which mean $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$8 (Wu et al., 29 May 2026). A plausible implication is that path length alone does not explain functional difficulty; the paper correspondingly identifies element density, action heterogeneity, and semantic dependencies as possible future difficulty descriptors.

The paper’s illustrative example.com trajectory makes the scoring semantics concrete: open URL, click “Insights,” click the search box, input “AI,” send Enter, deny cookies, click “Contact Us,” scroll, click “Region,” click “Europe,” click “Send,” and then done (Wu et al., 29 May 2026). In this framing, a page may render the search box correctly but fail to bind the Enter key to a search handler; the relevant step is then unmatched or misaligned, decreasing ACR and often preventing SR from succeeding. Likewise, a cookie modal that cannot be dismissed can drive the agent into repeated retries until the step budget is exceeded, in which case ASTC saturates at 2 and SR becomes 0 (Wu et al., 29 May 2026).

6. Relation to prior benchmarks, reproducibility, and limitations

WebIGBench is positioned against two adjacent lines of work. The first consists of static webpage benchmarks such as Design2Code, Web2Code, IW-Bench, WebCode2M, and WebSight, which the paper characterizes as emphasizing image-to-code fidelity and DOM structure without multi-step functional assessment. The second consists of interactive benchmarks such as Interaction2Code and MRWeb, which begin to address interactivity but remain limited to single-step events or simpler dynamics and do not perform state-aligned, automated functional evaluation across diverse action types (Wu et al., 29 May 2026).

The benchmark also differs from agent environments such as MiniWoB++, Mind2Web, and WebArena. Those environments target autonomous web task completion, whereas WebIGBench evaluates code generation fidelity for interactive webpages from visual references and asks whether generated code reproduces a prescribed, human-annotated interaction trajectory and its associated state transitions (Wu et al., 29 May 2026).

Its novelty is therefore threefold: real-world multi-step interaction continuity with high action density across five action types; automated functional evaluation beyond visual similarity through SR, ACR, and ASTC with state-level alignment; and UI-agent execution with consistent anchors for comparable access to reference and generated pages (Wu et al., 29 May 2026).

The released resources are hosted at https://github.com/anoa12159-hue/WebIGBench_eval under the CC BY-NC-SA 4.0 license. The repository includes interaction traces, prompts for Direct, Action, and Visual settings, the action matcher, scripts for automated checks, and evaluation code. The expected inputs are generated single-file HTML outputs served through a local HTTP server; the expected outputs are per-task metrics reports containing per-action alignment, per-page scores, and aggregate corpus-level scores (Wu et al., 29 May 2026).

Reproducibility is a major design goal. Deterministic execution with temperature $$C_{\mathrm{dyn}} = \mathrm{MLLM}(L_{\mathrm{image}}, L_{\mathrm{action}}, P_{\mathrm{dyn}}).$$9 ensures identical SR, ACR, and ASTC on re-runs, and the paper reports bootstrap 95% confidence intervals based on 10,000 resamples over pages. These intervals are described as narrow, with typical SR half-width at most $\kappa = 0.853$0 and ACR half-width at most $\kappa = 0.853$1, and the differences across model and prompt configurations exceed these intervals (Wu et al., 29 May 2026).

The stated limitations are equally important to the benchmark’s interpretation. Its scope is restricted to single-file HTML/CSS/JS in a single rendering process, excluding React and Vue, multi-file projects, back-end state, and multi-page navigation. The dataset size, while denser than prior interactive benchmarks, remains limited at 103 webpages and 871 actions because collection requires human-in-the-loop curation. The evaluator may still introduce residual noise through the UI agent and LLM-based matcher, although the paper mitigates this with reference-agent filtering, deterministic decoding, and human and multi-LLM validation. Finally, the current evaluator models a single linear sequence rather than a DAG of equivalent interaction paths (Wu et al., 29 May 2026).

Taken together, these design choices make WebIGBench a benchmark of interaction-level code generation fidelity rather than a general benchmark for full-stack web application synthesis. Its principal contribution is to formalize and operationalize the distinction between looking correct and behaving correctly in multimodal webpage generation (Wu et al., 29 May 2026).