Visual Evaluator Extension Overview

• Visual Evaluator Extension is a class of systems that inspect, modify, or score visual content through techniques like color adjustment, rubric-conditioned feedback, and interactive visualization.
• Browser-based implementations use SVG filters and matrix transformations to provide real-time color correction and selective zoom for improved readability and accessibility.
• Automated evaluators employ rubric-based scoring and interactive, visual debugging to deliver structured feedback in vision-language tasks and program execution tracing.

Visual Evaluator Extension denotes a class of extension-based systems that inspect, transform, or score visual content within a host environment. In browser accessibility research, SightGlow is presented as a web extension that functions as a “visual enhancement and evaluation tool” for people with red–green color vision deficiency and low vision, combining color transformation, selective zoom, and user-controlled contrast and saturation adjustment (Paudel, 2024). In adjacent literatures, closely related evaluator extensions appear as rubric-conditioned vision-language judges, dynamic interactive assessment frameworks for text-to-image systems, and structural program visualizers that expose evaluation steps through animated morphing (Lee et al., 2024, Mi et al., 2024, Godek, 6 Aug 2025).

1. Terminological scope and research usages

The literature uses the notion of a visual evaluator in several technically distinct ways. One usage is assistive and browser-native: an extension modifies the visual representation of web content so that users can inspect readability and color discriminability directly during browsing. A second usage is model-evaluative: a vision-LLM scores captions, generated images, edits, reviews, or answers under explicit criteria. A third usage is execution-visualizing: an extension renders intermediate states of a computation so that evaluation itself becomes a visual object.

Setting	Representative work	Core role
Web accessibility	SightGlow (Paudel, 2024)	Browser extension for color perception, readability, and personalization
Vision-language evaluation	GPT-4V evaluator, Prometheus-Vision, SFVE (Zhang et al., 2023, Lee et al., 2024, Ji et al., 2024)	Scalar scoring, pairwise judgment, rubric-conditioned feedback
Structural program visualization	GRASP visual stepper (Godek, 6 Aug 2025)	Visualizes substitution-based evaluation as animated expression-tree morphing

This diversity matters because the phrase does not refer to a single algorithmic family. In SightGlow, evaluation is user-centered and perceptual. In GPT-4V-based and rubric-conditioned systems, evaluation is automated and model-centered. In GRASP, evaluation is pedagogical and semantic, exposing reduction structure rather than rating outputs.

2. Browser-based accessibility realization in SightGlow

SightGlow is a browser extension designed for red–green color vision deficiency and low vision, with three explicit functional targets: color perception, readability, and personalization. It injects a UI overlay into the current web page, applies SVG-based color filters and matrix transformations to images, videos, and page graphics, and provides a selective zoom feature that enlarges selected text in a popup (Paudel, 2024).

Its color-processing pipeline is described conceptually as: input RGB content, conversion from RGB to LMS, simulation of color deficiency in LMS, daltonization to shift chromatic information from defective cones to functioning cones, and conversion back to RGB. The paper formalizes the core filter as a $4 \times 5$ matrix acting on RGBA:

$$\mathbf{c}_{out} = \begin{bmatrix} R'\ G'\ B'\ A' \end{bmatrix} = \begin{bmatrix} r_1 & r_2 & r_3 & r_4 & r_5 \ g_1 & g_2 & g_3 & g_4 & g_5 \ b_1 & b_2 & b_3 & b_4 & b_5 \ a_1 & a_2 & a_3 & a_4 & a_5 \end{bmatrix} \begin{bmatrix} R\ G\ B\ A\ 1 \end{bmatrix}.$$

For the specific red–green daltonization filter used in SightGlow, the instantiated matrix is:

$$\begin{bmatrix} 0.5 & 0.2 & 0.3 & 0 & 0 \ 0.0 & 1.0 & 0 & 0 & 0 \ 0.2 & 0.3 & 0.5 & 0 & 0 \ 0.2 & 0 & 0.0 & 1 & 0 \end{bmatrix}.$$

This yields:

$R' = 0.5R + 0.2G + 0.3B$

$G' = 1.0G$

$B' = 0.2R + 0.3G + 0.5B$

$A' = 0.2R + 1.0A.$

The paper notes that the alpha row has a small inconsistency, while the stated intent is to keep opacity unchanged. Implementation details are partly inferential rather than fully specified: the extension likely wraps the document in an SVG element with a <filter> definition or applies equivalent feColorMatrix-style transforms per image or video element. The system appears entirely client-side, since no server-side component is described.

3. Interaction model, personalization, and selective evaluation

SightGlow’s interaction model is explicitly user-tuned rather than automatically calibrated. Dynamic color control is provided through contrast and saturation sliders for images and videos, alongside enable/disable controls for the SVG color filter and selective zoom. The paper describes this as “real-time adjustment,” allowing users to alter color balance for “any images or video graphic content” according to need (Paudel, 2024).

Selective zoom is the system’s most distinctive local readability mechanism. Users select any text on a web page, and the extension returns a popup card with enlarged text content. The popup uses a color scheme chosen for color-perception accuracy among color-blind users; the paper specifically notes a yellow background with dark text. This is not global page zoom: it is a local, content-aware enlargement of the selected text segment.

The UI was iteratively redesigned after a pilot survey. The design uses blue and orange for interface elements because those colors are more CVD-friendly for red–green impairment, and it uses larger toggles and sliders to reduce motor effort. The redesign also increased font sizes and adjusted layout to better match low-vision needs. The paper explicitly states that the system is a user-controlled tuning tool rather than an automatically calibrated one; there is no multi-step diagnostic calibration and no explicit modeling of degree of impairment.

A broader implication is that this kind of visual evaluator extension couples inspection and adaptation in the same interface. Users do not merely receive an accessibility transformation; they actively explore alternative visual states and settle on settings that maximize subjective comfort and discriminability.

4. Evaluation paradigms in multimodal visual evaluators

In model-centered work, the visual evaluator is no longer a browser add-on but an evaluation engine. GPT-4V is studied as a generalist evaluator across image-to-text, text-to-image, text-guided image editing, and multi-image-to-text alignment, using two core modes: single-answer grading and pairwise comparison. The framework uses explicit scoring scales, task-specific rubrics, and a dual-sided comparison protocol that swaps candidate order to mitigate position bias (Zhang et al., 2023).

Several later systems specialize this pattern. VisCE$^2$ replaces reference captions with a structured “visual context” extracted from the image itself—objects, attributes, and relationships—and then asks a VLM to score a candidate caption from 0 to 100 using image, context, and caption jointly (Maeda et al., 2024). SFVE treats generative VQA evaluation as question-aware semantic similarity: it encodes Question + Answer and Question + Response, then scores them by cosine similarity, with training centered on VQA-specific contrastive signals such as candidate answers, synonyms, antonyms, and generated descriptions (Ji et al., 2024). Prometheus-Vision instead conditions on a user-defined 1–5 score rubric, a candidate response, an image, and a reference answer, and is trained to generate both detailed feedback and a numeric score (Lee et al., 2024).

Defeasible Visual Entailment introduces a different evaluator objective. Rather than scoring static correctness, it models how an update $U$ changes the plausibility of a hypothesis $H$ given an image $$\mathbf{c}_{out} = \begin{bmatrix} R'\ G'\ B'\ A' \end{bmatrix} = \begin{bmatrix} r_1 & r_2 & r_3 & r_4 & r_5 \ g_1 & g_2 & g_3 & g_4 & g_5 \ b_1 & b_2 & b_3 & b_4 & b_5 \ a_1 & a_2 & a_3 & a_4 & a_5 \end{bmatrix} \begin{bmatrix} R\ G\ B\ A\ 1 \end{bmatrix}.$$0. Its scalar entailment-strength head is trained with pairwise contrastive learning relative to a baseline caption and combined with categorical supervision over weakener versus strengthener labels (Zhang et al., 2024).

These systems share a common architectural move: the evaluator is given more structure than traditional automatic metrics. That structure may be a rubric, a visual-context decomposition, a question-conditioned pair encoding, or an inference-aware ranking objective. This suggests that a visual evaluator extension becomes substantially more reliable when it is criterion-explicit rather than purely similarity-based.

5. Interactive and structural extensions beyond scoring

Not all visual evaluators are scalar judges. DyEval frames evaluation of text-to-image systems as an adaptive exploration process over a test tree of topics and prompts. It combines LLM-based topic generation, prompt generation, human assessment, and a contextual reflection module that mines failure triggers. The framework reports metrics such as number of bugs, average pass rate, and average fail rate, and it is designed to uncover rare or emerging failure patterns through iterative user–LLM collaboration (Mi et al., 2024).

BYO-Eval shifts the problem again: instead of evaluating with static benchmarks, it uses procedurally generated synthetic images to produce controlled diagnostic tests for multimodal models. Difficulty can be varied systematically through object count, blur, overlap, camera distance, and related scene parameters, while metrics include accuracy, MAE, MSE, NMAE, and a grid-based localization loss. The methodological emphasis is on interpretable performance-versus-difficulty curves rather than a single benchmark score (Arnould et al., 5 Jun 2025).

In programming environments, the phrase becomes even more literal. The GRASP visual stepper—explicitly also called a “visual evaluator extension”—evaluates a purely functional subset of Scheme via substitution-based small-step semantics and visualizes each reduction step as animated morphing between expression trees. Provenance is tracked through origin and progeny mappings, and the interface exposes player-style controls such as rewind, step backward, play, and step forward (Godek, 6 Aug 2025). Eye, although not using the same term, similarly treats program execution as a visual interpretation problem: code, stack frames, scopes, and data structures are rendered as an interactive execution trace for learning and debugging (Bansal et al., 2021).

Taken together, these systems broaden the concept of a visual evaluator extension from “a component that assigns scores” to “a component that externalizes evaluation as an inspectable visual process.” In SightGlow the inspected object is perceptual comfort; in DyEval it is model failure structure; in GRASP it is operational semantics.

6. Limitations, reliability issues, and future directions

SightGlow’s current evidence base is explicitly limited. Its pilot evaluation involved nine participants, all with normal vision, and no participants with diagnosed color blindness or low vision. The study is formative, uses seven Likert-scale questions plus one open-ended question, and reports no inferential statistics. The system currently focuses on red–green color blindness, provides only selective text zoom as its low-vision mechanism, does not support PDF documents, and requires manual slider adjustment rather than automatic calibration (Paudel, 2024).

Automated visual evaluators exhibit a different limitation profile. GPT-4V shows strong agreement with humans in captioning and multi-image review alignment, but weaker performance on text-to-image pairwise judgment, especially for visual clarity and complex prompt satisfaction (Zhang et al., 2023). Prometheus-Vision inherits limitations from its LLaVA-1.5 backbone, particularly on text-rich images such as charts and diagrams (Lee et al., 2024). VIABLE shows that VLM-as-a-judge is especially fragile in visually impaired assistance settings: even the strongest judge reaches only 52.6% single-failure diagnostic accuracy, while self-preference and adversarial fragility remain pronounced; VIA-Judge-Agent improves this by adding visual evidence extraction and a taxonomy-guided workflow (Zhao et al., 29 May 2026). Multimodal Evaluator Preference Collapse extends the caution further, showing that evaluator-induced strategy collapse is amplified in multimodal settings and that cross-modal contagion can distort strategy selection across text and visual tasks (Liu, 15 Jun 2026).

A plausible synthesis is that future visual evaluator extensions will increasingly combine three elements. First, explicit structure, such as rubrics, taxonomies, or visual-context decompositions, to constrain what is being evaluated. Second, interactive inspection, so that evaluation is observable and revisable rather than opaque. Third, modality-aware safeguards, including dual-sided comparison, evidence extraction, and evaluator-diversity checks, to reduce bias, instability, and collapse. In the browser-accessibility lineage represented by SightGlow, the corresponding future directions are broader impairment support, richer content coverage such as PDFs and canvas elements, automated detection of low-contrast or problematic color combinations, and empirical testing with the intended user populations (Paudel, 2024).