WebGen-V: Advanced HTML Generation Benchmark

• WebGen-V is a structured benchmark for instruction‑to‑HTML generation that employs agentic crawling and detailed section-wise data representation.
• It decomposes webpages into multimodal JSON sections capturing text, images, metadata, and layout to enable precise evaluation using advanced LLMs.
• By integrating dynamic data augmentation with granular metric assessment, WebGen-V enhances both model training and automated web synthesis evaluation.

WebGen-V is a benchmark and structured framework for high-granularity instruction‑to‑HTML generation, designed to advance both the realism and assessment precision of LLM-based web page generation. It integrates agentic crawling of real-world webpages, a section-wise multimodal data representation, and a detailed evaluation protocol that explicitly aligns text, layout, and visual components for each section. By enabling continuous data augmentation and granular supervision, WebGen-V establishes a unified pipeline that supports both enhanced model training and rigorous evaluation for the automated synthesis of visually coherent, functionally correct HTML designs.

1. Core Innovations and Benchmark Contributions

WebGen-V advances previous benchmarks through three principal innovations:

• Agentic Crawling Framework: This module continuously harvests real-world webpages using keyword-driven seed queries and a hybrid rendering strategy (static HTTP and dynamic Playwright), facilitating unbounded expansion and compatibility with legacy benchmarks.
• Structured Section-Wise Representation: Each page is decomposed into discrete sections (e.g., hero, footer), with each section recorded as JSON objects containing text (T), localized screenshots (I), metadata (M), color and font information, and bounding boxes (B) for spatial layout. The structured representation is formalized as $\mathcal{Z}(W) = \{S, T, I, M, B\}$.
• Section-Level Multimodal Evaluation Protocol: Evaluation of generated webpages is performed per section using a sophisticated LLM evaluator (e.g., GPT‑5), which scores nine metrics (textual accuracy, layout, media positioning, spacing, and more) on discrete scales and provides both quantitative and qualitative feedback.

Collectively, these innovations shift the paradigm from monolithic webpage evaluation toward fine-grained, multimodal assessment compatible with real-world design complexity (Wang et al., 17 Oct 2025).

2. Agentic Data Collection Framework

The agentic crawling module in WebGen-V supports scalable, extensible, and continuously updatable dataset construction:

• Unbounded Harvesting: The crawler uses curated keyword lists (e.g., “crm”, “portfolio”, “contact us”) to generate search queries that discover high-intent web domains. Both static and dynamic pages are captured using HTTP requests and Playwright-driven browser automation, respectively.
• Backward Compatibility: The framework is engineered to transform and enrich legacy web datasets such as WebMMU into the section-wise structured format.
• Dynamic Realism: Over 3,000 new webpages from diverse application domains are continually ingested, ensuring benchmarks reflect evolving web design standards and practices.

This agentic approach establishes a data foundation for multimodal web generation tasks requiring adaptive, up-to-date sources (Wang et al., 17 Oct 2025).

3. Section-Wise Structured Data Representation

WebGen-V’s representation model enables precise supervision by aligning textual, spatial, and visual modalities at the section level:

• Decomposition Algorithm: Algorithm 1 identifies candidate containers (e.g., “section”, “div”) based on DOM features such as rendered height (>50px) and proximity, applying an intersection-over-union (IoU) criterion to merge overlapping regions.
• Rich Modal Structure: For each section $s_i$, the Processor extracts structured text (T), metadata (M) including style and color palettes, cropped UI screenshots (I), and semantic image classifications generated by an LLM. Bounding boxes (B) encode visual layout dimensions.
• Multimodal Alignment: This unified JSON structure enables targeted inspection and correction of localized design flaws, as the representation explicitly couples each web section’s content, style, and spatial attributes.

The structured format supports detailed supervision and is well-suited for generation, evaluation, and model refinement cycles (Wang et al., 17 Oct 2025).

4. Granular Multimodal Evaluation Protocol

Evaluation in WebGen-V achieves unprecedented granularity and actionable feedback by operating at the section level:

• Per-Section Metric Scoring: Each generated section $\hat{s}_i$ is assessed for nine metrics, including textual accuracy (TA), text placement (TP), media positional accuracy (MP), alignment consistency (ALN), and spacing consistency (SPC). Discrete scores (1–5) are governed by visible flaw constraints (e.g., design imperfections cap maximum at 3).
• Quantitative and Qualitative Feedback: Each metric evaluation yields structured tuples $$\mathcal{F} = \{ (\hat{s}_i, m_k, \text{score}, \text{reason}, \text{feedback}) \}$$, supporting both direct scoring and context-sensitive diagnostic insight.
• Iterative Refinement Cycle: Algorithm 2 formalizes a Generation–Evaluation–Refinement loop: if a section’s score falls below a defined threshold $\tau$, its feedback prompts targeted model revision and re-generation for improved output.

This multimodal protocol directly supports iterative model improvement and enables high-fidelity diagnosis of localized webpage synthesis errors (Wang et al., 17 Oct 2025).

5. Experimental Validation and Ablation Analysis

WebGen-V’s impact is substantiated by experiments with state-of-the-art multimodal LLMs and targeted ablation studies:

• Model Comparisons: Models such as GPT‑5, Gemini‑2.5-Pro, and Claude‑Opus‑4.1 were assessed in both zero-shot and one-turn refinement scenarios. Quantitative improvements are noted in metrics like spacing consistency, media placement, and alignment.
• Ablation Studies: Removal of structured text or section-wise screenshots results in marked performance degradation, demonstrating the necessity of fine-grained cues for high-quality generation. For example, structured evaluation detects human-injected design degradations with F1 scores increasing from 0.44 (non-structured) to 0.75.
• Legacy Dataset Transformation: Conversion of existing datasets (e.g., WebMMU) into comprehensive section-wise representations highlights the adaptability and extensibility of the framework for other benchmarks.

These results confirm the substantial contribution of the structured data model and evaluation protocol to generation accuracy, granularity, and model robustness (Wang et al., 17 Oct 2025).

6. Implications, Novelty, and Context

WebGen-V sets a new standard for automated web synthesis and evaluation:

• High-Granularity Agentic Benchmarking: WebGen-V is the first framework to unify agentic crawling, section-wise decomposition, and multimodal assessment for instruction-to-HTML tasks, continuously integrating evolving web designs and supporting legacy data transformation.
• Unified Training and Assessment Pipeline: By establishing granular data structures and rule-based evaluation, the framework enables both efficient training and reliable, targeted evaluation cycles for web-focused LLM models.
• Research Directions: The section-wise structured approach is instrumental in reducing token cost, improving performance consistency across diverse models, and supporting iterative design refinement. This suggests further research opportunities in dynamic data sampling, multimodal model supervision, and real-world asset augmentation.
• Broader Significance: By extending multimodal reasoning to arbitrary web sections, WebGen-V facilitates targeted defect diagnosis, improved supervision, and future advances in autonomous web page generation.

In summation, WebGen-V’s systematic approach to acquisition, representation, and granular evaluation provides a robust, extensible foundation for LLM-driven web design research, marking a significant evolution in the precision and adaptability of automated webpage synthesis and validation (Wang et al., 17 Oct 2025).