G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness (2505.05026v2)

Abstract: Evaluating user interface (UI) design effectiveness extends beyond aesthetics to influencing user behavior, a principle central to Design Persuasiveness. A/B testing is the predominant method for determining which UI variations drive higher user engagement, but it is costly and time-consuming. While recent Vision-LLMs (VLMs) can process automated UI analysis, current approaches focus on isolated design attributes rather than comparative persuasiveness-the key factor in optimizing user interactions. To address this, we introduce WiserUI-Bench, a benchmark designed for Pairwise UI Design Persuasiveness Assessment task, featuring 300 real-world UI image pairs labeled with A/B test results and expert rationales. Additionally, we propose G-FOCUS, a novel inference-time reasoning strategy that enhances VLM-based persuasiveness assessment by reducing position bias and improving evaluation accuracy. Experimental results show that G-FOCUS surpasses existing inference strategies in consistency and accuracy for pairwise UI evaluation. Through promoting VLM-driven evaluation of UI persuasiveness, our work offers an approach to complement A/B testing, propelling progress in scalable UI preference modeling and design optimization. Code and data will be released publicly.

An Overview of "G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness"

The paper "G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness" addresses a significant challenge in user interface (UI) design evaluation: the effective assessment of design persuasiveness. The paper identifies a critical gap in current methods whereby traditional A/B testing, albeit effective, is resource-intensive and time-consuming. This paper introduces a novel approach using Vision-LLMs (VLMs) to automate and enhance the persuasiveness evaluation of UI designs, proposing a method named G-FOCUS alongside a benchmark named WiserUI-Bench.

WiserUI-Bench constitutes a crucial component of the research, featuring 300 real-world UI image pairs, annotated with A/B test outcomes and expert rationales. This assembly serves a dual purpose: it allows for systematic evaluation of UI design effectiveness and provides a repository that reflects real-world scenarios validated by extensive user interactions. By facilitating pairwise comparisons instead of isolated assessments, this benchmark aids VLMs in discerning the hierarchical effectiveness of UI elements, closely aligning with human-centered design principles. The benchmark's credibility stems from empirical data, offering a reliable foundation for the automation of design evaluations.

G-FOCUS, the core methodology proposed, provides an inference-time reasoning strategy for VLMs tailored for pairwise UI evaluation tasks. This strategy encompasses four phases: persuasion goal extraction, UI difference localization, contrastive reasoning, and rationale-based evaluation. It aims to tackle prevalent issues such as position biases and inconsistent rationales, enhancing the model's capacity for reliable UI design assessments. G-FOCUS demonstrates superior consistency and accuracy compared to existing strategies, as shown in controlled experiments using leading VLMs such as GPT-4o, Claude 3.5 Sonnet, and Llama-3.2-90B-Vision.

The implications of G-FOCUS extend beyond mere UI evaluation. By aligning closely with human preferences and enhancing the understanding of persuasive design principles, this method paves the way for scalable preference modeling in AI. The application of G-FOCUS in automated UI preference verification suggests significant potential for advancements in design automation, reinforcing the bridge between traditional empirical methods and AI-driven assessments.

The methodology holds promise for evolving AI's role in UX design, emphasizing scalable, data-driven, and reliable evaluation techniques. Future research may explore the integration of G-FOCUS in dynamic or interactive UI contexts, as well as its application across varying cultural and user-centric perspectives. The work not only contributes to methodological advancements in design evaluation but also sets the stage for further AI-centric developments in user experience optimization, poised to align digital interfaces more closely with human behavior and expectations.