EfficientUICoder: Optimized UI Code Generation

• EfficientUICoder is a UI code generation framework that compresses both image and code tokens, optimizing the processing of high-resolution UI screenshots with numerous elements.
• It employs a three-stage approach—structured token selection, attention-driven refinement, and adaptive duplicate suppression—to eliminate redundant tokens and reduce computational overhead.
• Empirical evaluations demonstrate nearly 50% runtime reductions while maintaining or improving code fidelity on benchmark UI2Code datasets.

EfficientUICoder is a UI code generation framework designed for Multimodal LLMs (MLLMs) that optimizes computational efficiency by compressing both input (image) and output (code) token streams. It targets the UI2Code scenario, where models must process high-resolution screenshots with many UI elements and generate extensive, structured HTML + CSS code, often resulting in prohibitively long token sequences and elevated runtime costs (Xiao et al., 15 Sep 2025). EfficientUICoder systematically addresses the core inefficiencies inherent to prior UI2Code pipelines, offering a three-part solution that combines structured visual token selection, attention-driven refinement, and dynamic suppression of repetitive output generation.

1. Architectural Overview and Compression Motivation

UI2Code tasks differ fundamentally from canonical text-to-code generation: typical web page screenshots result in a large number of image tokens (due to fine-grained layouts) and necessitate long HTML/CSS outputs. In a standard Transformer-based VLM pipeline, self-attention computational overhead scales as $O(n^2d)$ where $n$ is the combined input sequence length; decoder search and storage costs also grow linearly with output tokens. Empirical analysis identified two primary inefficiency sources:

• Redundant visual tokens: Many image patches handled by the Vision Transformer (e.g., CLIP-based encoders) contribute negligible information, particularly in backgrounds or UI chrome, yet still drive up computational complexity.
• Redundant code tokens: Large models tend to overproduce repeated HTML containers, CSS selectors, or text snippets as a byproduct of autoregressive decoding, frequently resulting in inflated code length and occasional non-terminating generations.

2. Framework Components: Methods of Compression and Suppression

EfficientUICoder employs a three-stage compression strategy:

2.1 Element and Layout-aware Token Compression (ELTC)

• Detection and graph construction: The framework applies a GUI element detector (e.g., UIED) to extract bounding boxes $\{B_i\}$ for text, buttons, images, and similar page primitives from the input screenshot. Overlapping or fragmented detections are merged based on pairwise box-center proximity.
• UI element graph/minimum spanning tree: Nodes represent element bounding boxes; edge weights $$w_{ij} = \min_{p\in \partial B_i, q\in \partial B_j}\|p-q\|_2$$ encode spatial proximity. A minimum spanning tree $T^*$ is constructed to preserve visual layout.
• Token selection: Only those image tokens (patches) mapped to the bounding boxes of tree nodes are retained for the encoder input, while all background or non-element tokens are dropped. This eliminates extraneous patches not responsible for UI semantics.
• Compression ratio: Defined as $$R = \frac{|\text{original tokens}| - |\text{compressed tokens}|}{|\text{original tokens}|}$$, with typical $R\approx0.55\text{–}0.60$.

2.2 Region-aware Token Refinement (RTR)

Following ELTC, the remaining image tokens can still include low-saliency regions, while some high-information patches (such as background color cues) may be lost. RTR operates by:

• Attention score computation: Using the output of the last CLIP layer, average cross-token attention $A_{\rm avg}\in \mathbb{R}^{N\times N}$ is computed. For each token, its relevance score $I_i = A_{\rm avg}[\mathrm{CLS},i]$ is used.
• Refinement process: The selected token set $S$ from ELTC and the dropped set $n$0 are cross-compared. RTR removes the bottom $n$1 tokens (least attention) and introduces the top $n$2 tokens (highest attention), controlling $n$3 (found optimal in $n$4–$n$5 range) to preserve overall length and maximize semantic coverage.

2.3 Adaptive Duplicate Token Suppression (ADTS)

Redundancy in code generation is addressed through online tracking and softmax logit penalization:

• Repetition tracking: The decoding process maintains counters for CSS selectors, HTML tag-attribute pairs, and substrings within textual content. Each time a candidate token $n$6 is generated, an associated count $n$7 tracks its frequency in the output so far.
• Exponential penalty: For the next $n$8 decoding steps, logits for over-represented tokens are weighted by $n$9 (typically $\{B_i\}$0, $\{B_i\}$1), effectively reducing their sampling probabilities. This mechanism dynamically suppresses repeated structures, mitigating output bloating and potential infinite loops.

3. Empirical Performance and Evaluation

EfficientUICoder’s effectiveness has been established on large-scale UI2Code benchmarks and compared to vanilla MLLM pipelines:

• Datasets: Design2Code (484 webpages) and WebCode2M (multiple splits, 300 samples).
• Backbone models: Llava-v1.6-7B and Llava-v1.6-34B (temperature 0, max generation 4096).
• Metrics: BLEU for code quality, CLIP-Score for rendered-image similarity, block-match and structural similarity for code fidelity, and efficiency statistics (FLOPs, token count, prefill/inference time).

Quantitative results:

	Compression $\{B_i\}$2	FLOPs ↓	Tokens ↓	Prefill ↓	Inference ↓
Llava-34B	60.36%	44.9%	41.4%	46.6%	48.8%
WebCode2M	55.86%	42.8%	36.7%	45.9%	45.1%

Quality on BLEU/CLIP/other metrics is maintained or slightly improved compared to non-compressed baselines, with human judges preferring EfficientUICoder outputs in pairwise evaluation (Xiao et al., 15 Sep 2025).

4. Algorithmic Complexity and Implementation Considerations

• Self-attention FLOPs: Reduced from $\{B_i\}$3 to $\{B_i\}$4, owing to aggressive input image token pruning.
• Decoder runtime: Linearly reduced proportional to the output token decrease contributed by ADTS.
• Trade-offs: ELTC offers maximum token reduction at the risk of losing layout cues; RTR serves as a correction. ADTS correct tuning of $\{B_i\}$5 is critical to avoid penalizing necessary repetitions (excessive strength may reduce BLEU); optimal found at $\{B_i\}$6, $\{B_i\}$7.
• Ablation and sensitivity: Full system (ELTC + RTR + ADTS) outperforms any ablated configuration, with $\{B_i\}$8–$\{B_i\}$9 and proper $$w_{ij} = \min_{p\in \partial B_i, q\in \partial B_j}\|p-q\|_2$$0 as the best parameter choices.

5. Extensions, Applicability, and Comparative Systems

EfficientUICoder’s principles generalize to broader multimodal sequence tasks with large visual input bottlenecks, such as slide-to-text or poster-generation. Possible avenues include:

• Learnable or RL-guided compression: Extension of the static RTR drop/add rates and ADTS penalty parameters to reinforcement-learned or adaptive policies.
• Enhanced output handling: Trainable repeat-bias layers for more intricate pattern suppression, possibly extending beyond HTML/CSS sequence to other target languages or templates.
• Cross-model relevance: The structured, element-aware selection in EfficientUICoder is related to optical (encoder-side learned) compression as in UIPress (Dai et al., 10 Apr 2026), which uses a learned compressive module between a ViT encoder and decoder, achieving even higher token compression (256 tokens from large input) and showing further runtime and fidelity improvements.

6. Limitations and Future Research Directions

While EfficientUICoder achieves substantial compute gains without code/visual quality regression, its architecture does have constraints:

• Dependence on UI Element Detection: ELTC’s efficacy relies on the UI detector’s recall and segmentation quality; errors propagate to subsequent compression stages.
• Heuristic-based refinement: RTR and ADTS are currently rule-based with hyperparameter dependence; learnable alternatives may further optimize compression-quality trade-offs.
• Non-adaptive compression ratios: Fixed proportion token selection may be suboptimal for UIs with especially sparse or dense layouts.
• Extension to mobile and vector UI2Code: Models and benchmarks are primarily web-centric; applicability to app UIs or design-tool artifacts would require retraining detectors and possible architecture tuning.

Potential trajectories include integration of reinforcement learning for dynamic compression, per-instance token budgets, and incorporation of auxiliary rewards for structure or semantic similarity (Xiao et al., 15 Sep 2025, Dai et al., 10 Apr 2026).

7. Relationship to Broader UI2Code Systems

EfficientUICoder is part of a broader trend towards task-adaptive compression and output management in UI2Code MLLMs:

• Contrast with learned compressors (UIPress): While EfficientUICoder employs explicit detection- and attention-based compression, systems like UIPress apply fully-learned, domain-adapted optical compression modules to the encoder output, combined with LoRA-adapted decoders, attaining up to 25.5× true token-length reduction and $$w_{ij} = \min_{p\in \partial B_i, q\in \partial B_j}\|p-q\|_2$$17.5% CLIP score gains (Dai et al., 10 Apr 2026). This suggests that hybrid strategies—melding element-aware token selection, attention refinement, and learned compressive mappings—are likely to define future EfficientUICoder derivatives.
• Comparison with data feedback-driven methods (UICoder): Orthogonally, compiler- and CLIP-based filtering (as in UICoder) provide sample-level efficiency for model training rather than sequence-level runtime speedups, underlining that EfficientUICoder’s methods target a different regime of efficiency (Wu et al., 2024).

EfficientUICoder defines a practical state-of-the-art in efficient, scalable MLLM UI2Code by fusing layout-aware token pruning, attention-driven refinement, and on-the-fly output suppression, yielding nearly 50% speedups and meaningful compute reductions without sacrificing output validity or visual congruence (Xiao et al., 15 Sep 2025).