VIP-Sim: Participatory Vision Simulation

• VIP-Sim is a real-time, symptom-based vision simulator that models 21 visual impairments using custom shaders and gaze-contingent effects.
• It employs a participatory co-design process via the MonoWitness Protocol, directly involving people with vision impairments to iteratively refine simulation fidelity.
• Designed as an early-stage design aid, VIP-Sim offers actionable insights for interface accessibility while emphasizing the need for direct feedback from VIPs.

Searching arXiv for the specified VIP-Sim paper and closely related context. VIP-Sim is a symptom-based vision simulator for accessible design that was developed through a participatory design process involving people with vision impairments (VIPs). It is intended to let interface designers inspect desktop applications through overlays that model 21 symptoms, including field loss, light sensitivity, blur, contrast reduction, and color-vision changes. The system combines a lightweight desktop overlay, a GPU shader pipeline, and optional gaze-contingent rendering, while its methodological contribution is the “MonoWitness Protocol” for involving VIPs directly in the design and refinement of vision simulators (Rädler et al., 14 Jul 2025).

1. Participatory design basis

VIP-Sim was created to address a gap identified by its authors: previous vision impairment simulators had been introduced, but none had been developed with the direct involvement of VIPs or thoroughly evaluated from their perspective. The project therefore positioned VIPs as co-designers rather than passive evaluators. Seven participants took part in iterative sessions, and the simulator was refined after each session through a “one-participant, one-iteration” cycle (Rädler et al., 14 Jul 2025).

The co-design process was organized through the MonoWitness Protocol. Each session contained five stages: demographics and medical history; vision assessment, including an Amsler Grid for central field loss and Pelli-Robson/Ishihara tests for contrast and color sensitivity; qualitative description of real-world perception using two standard photographs; hands-on tuning of simulated symptoms while covering one eye at a time; and semi-structured interviews about symptom fidelity and simulator use. In Stage D, participants adjusted shader severities directly where possible, and when a shader did not match remembered perception, the research team modified parameters or added new effects. A cited example is the addition of a pixelation-style “Detail Loss” effect for participants who reported “seeing blocks rather than blur” (Rädler et al., 14 Jul 2025).

This process converged on symptom definitions that five of seven VIPs agreed closely matched their individual experiences. At the same time, the study documented an important boundary condition: participants whose impairments involved unusual combinations, such as congenital optic atrophy with nystagmus, stressed that simulation should not substitute for direct VIP involvement. That concern is integral to the project’s framing rather than external criticism.

2. Symptom taxonomy and rendering models

VIP-Sim offers 21 independently togglable symptom shaders with dedicated parameters. These are Central Field Loss, Peripheral Field Loss, Foveal Darkness, Hyperopic Blur, Detail Loss, Contrast Sensitivity Reduction, Protan, Deutan, Tritan, Monochrome, Metamorphopsia (Pointwise), Metamorphopsia Overlay, Glare / Photophobia, Retinopathy / Floaters, Teichopsia, Double Vision, In-Filling, Distortion (Vortex), Flickering Stars, Nystagmus, and Cataracts (Rädler et al., 14 Jul 2025).

Several of these effects are gaze-contingent. Central Field Loss, for example, is modeled around the current point of gaze by

$$g(x,y)=\sum_i I(x_i,y_i)\cdot w(\|(x,y)-(x_0,y_0)\|),$$

where $I$ is the source image, $(x_0,y_0)$ is the gaze point, and $w(r)$ is a grayscale, radially symmetric low-pass weighting that drops to zero at radius $R$. Peripheral Field Loss is the inverted form of this masking logic, while Foveal Darkness uses an opaque gray disc with optional feathered alpha falloff

$A(r)=\mathrm{clamp}(1-r/R,0,1).$

The implementation uses single-pass foveated sampling for field-loss effects (Rädler et al., 14 Jul 2025).

Blur, contrast, and color impairments are modeled through explicit image-processing operators. Hyperopic Blur uses a Gaussian blur kernel with $\sigma \propto 1/\mathrm{CPD}$ and maps visual acuity to

$\sigma = K/V,$

with $V=30\ \mathrm{cpd}$ corresponding to approximately no blur. Contrast Sensitivity Reduction applies

$I' = ((I-0.5)\cdot(1-C) + 0.5 + B)^{1/\Gamma},$

while color-vision deficiencies use precomputed $I$0 LMS-to-RGB transformation matrices blended by severity:

$I$1

Detail Loss differs from blur by quantizing coordinates into blocks of size $I$2 and then softening block edges by interpolation (Rädler et al., 14 Jul 2025).

Other effects model distortions, luminous artifacts, and motion instability. Pointwise Metamorphopsia jitters coordinates according to

$I$3

where $I$4 is a 2D noise field, whereas Metamorphopsia Overlay uses a screen-wide sine-wave displacement. Glare / Photophobia is implemented as bloom on bright areas:

$I$5

Distortion (Vortex) applies a polar warp with

$I$6

and Nystagmus is modeled as full-screen translation:

$I$7

Cataracts are implemented in two stages: first contrast and brightness reduction, then a frosting displacement via a Perlin-noise offset distribution (Rädler et al., 14 Jul 2025).

The simulator’s modeling choice is symptom-focused rather than disease-preset-focused. This is significant because the paper explicitly recommends symptom-focused sliders over pre-baked disease presets in order to better represent comorbidities and mixed symptomologies.

3. Software architecture and execution model

VIP-Sim is implemented in Unity 2022.3 as a transparent, click-through overlay on Windows and macOS. On Windows it uses UWindowCapture for window capture; on macOS it uses mcDesktopCapture. The underlying application frame is copied into an off-screen render texture and then processed by a custom Unity URP/HDRP multi-shader pass (Rädler et al., 14 Jul 2025).

Gaze-contingent effects are driven by webcam-based eye tracking through the UnitEye plugin, which provides screen-space gaze coordinates and a covariance estimate smoothed with a Kalman filter. If tracking fails, including in cases such as nystagmus, the system falls back to mouse-position mapping. Each symptom is implemented as an independent full-screen fragment shader controlled by uniform parameters and, when required, gaze input (Rädler et al., 14 Jul 2025).

The rendering path is explicitly GPU-centered. Central and peripheral field-loss effects use mip-mapped foveated sampling; shaders are grouped into a single draw call where possible; inter-symptom blending is performed via additive alpha in a final composite pass. On a midrange GPU, the reported performance is 60 fps at 1440p while eye tracking and all 21 symptoms are enabled, with dynamic culling of inactive symptoms. That performance claim is important because the system is meant to remain usable while attached to ordinary design workflows rather than in isolated laboratory demonstrations (Rädler et al., 14 Jul 2025).

4. Interaction design and design-tool workflow

The user interface is a thin, semi-transparent toolbar that floats above any application. Its controls include a “Capture Application” dropdown that lists detected top-level windows such as Adobe XD, Figma, and Unreal Engine; an Enable/Disable all toggle; a symptom list with checkboxes for the 21 shaders; a context-sensitive parameter pane; an Eye/MOUSE toggle for gaze control; and a camera selector. The toolbar can be docked, minimized, or collapsed to a thin header, and a global hotkey toggles simulation visibility (Rädler et al., 14 Jul 2025).

The paper describes a typical workflow in operational terms. A designer launches VIP-Sim, selects a target application such as Adobe XD, enables Central Vision Loss, sets its size to match a target scotoma radius, activates Contrast Sensitivity with a specified severity, and then inspects interface elements such as buttons or icons while the underlying design tool remains visible. The same workflow supports exporting a screenshot for documentation or reverting all symptoms with the Enable toggle (Rädler et al., 14 Jul 2025).

This workflow clarifies the intended use case: VIP-Sim is positioned for early wireframe and mid-fidelity prototyping, where changes to font size, contrast, spacing, and visual hierarchy remain inexpensive. The recommendation is not to treat the simulator as a certification device, but as a design aid for exposing likely accessibility issues earlier in the pipeline.

5. Evaluation results and interpretive limits

The central evaluation question was how well VIP-Sim replicates real impairments. The study did not attempt statistical generalization from $I$8; instead, each session fed back directly into shader refinements. For each shader, the researchers recorded a 5-point match rating, time to reach “just right,” and free-form feedback (Rädler et al., 14 Jul 2025).

The reported outcomes are specific. Of the evaluated shaders, 83% (20/24) were rated “about right” or “slightly mild/strong” by at least 5 of 7 VIPs without major revisions. Central Field Loss, Glare, Hyperopia, Contrast Sensitivity, Retinopathy, and Cataracts were unanimously rated as realistic after iterative tuning. Color Vision Deficiency matrices were left unmodified because participants with CVD could not discern changes at full severity. Nystagmus and one Metamorphopsia variant were ultimately deprecated because no participant felt they matched lived experience (Rädler et al., 14 Jul 2025).

Participants also commented directly on gaze contingency. Gaze-contingent effects were valued because they moved with sight, which participants described as essential, although some noted that mouse-pointer fallback is not equivalent. In a final semi-structured survey, five of seven participants agreed that VIP-Sim could help designers empathize and catch minor design issues early. Four expressed slight concern about overreliance on such tools if direct VIP testing is omitted, and two emphasized the need for personalization because impairments are heterogeneous (Rädler et al., 14 Jul 2025).

These findings constrain how the system should be interpreted. The data support the claim that most participants felt VIP-Sim could replicate their symptoms, but they do not support a claim of universal fidelity across all impairments. The project explicitly presents comprehensiveness and exclusion as unresolved issues.

6. Recommendations, limitations, and scope

The authors distill several design recommendations from the co-design process. Simulation parameters should be calibrated to the individual’s scotoma size, blur level, and contrast sensitivity using simple tests such as the Amsler grid and Pelli-Robson chart. Symptom-focused controls are preferred over disease presets. Gaze-contingent field-loss effects should be embedded whenever possible. VIP-Sim should be used during early and mid-fidelity design stages. Most importantly, simulation should be complemented with direct VIP feedback because simulations cannot capture cognitive strategies, hallucinations such as Charles Bonnet syndrome, or individual internal compensations (Rädler et al., 14 Jul 2025).

The system also has explicit technical and methodological limitations. It currently runs on desktop tools rather than web or mobile platforms. Eye-tracking accuracy of approximately 0.5 cm can introduce small scotoma offsets, motivating more robust mapping of gaze to screen, including possible snapping to UI elements. Scaling beyond $I$9, especially toward more congenital cases, may require different elicitation methods because some participants lack an unimpaired reference eye. The paper further proposes quantitative A/B testing of shader variants, human-in-the-loop parameter optimization, and longitudinal studies of workflow integration as future directions (Rädler et al., 14 Jul 2025).

Within those bounds, VIP-Sim occupies a specific position in accessibility research: it is a user-centered, symptom-focused, real-time simulator intended to bring perceptual impairments into everyday interface design practice without claiming to replace direct participatory testing. The methodological emphasis on co-creation is as central as the rendering engine itself.