VeasyGuide: Enhanced Video Accessibility
- VeasyGuide is an accessibility system that aids low-vision learners by detecting subtle instructor actions in presentation videos using image processing and graph-based grouping.
- It implements real-time, box-shaped visual highlights with optional magnification to reduce search effort and support personalized visual preferences.
- Empirical evaluations demonstrate improved detection success and reduced workload, effectively narrowing the performance gap between low-vision and sighted users.
Searching arXiv for the named paper and closely related guidance/accessibility work to ground the article and citations. VeasyGuide is an accessibility system for low-vision learners watching presentation videos, especially videos in which an instructor speaks over slides while visually interacting with them using a mouse, digital pen, or annotation tool. It automatically detects instructor actions such as pointing, marking, and sketching, then augments playback with personalized visual highlights and optional magnification. Its stated objective is to reduce visual search effort by clarifying what to look for and where to look, while supporting the use of residual vision rather than replacing vision with audio description (Sechayk et al., 29 Jul 2025).
1. Problem domain and accessibility rationale
VeasyGuide addresses a specific failure mode of educational video accessibility: important instructional content is often conveyed through subtle visual actions that are not verbally described. In presentation videos, instructors may point to a term, underline or circle a phrase, sketch a supplementary diagram, or draw an arrow. For a low-vision viewer, such actions can be fleeting, small, low-contrast, or outside the current locus of attention. The resulting difficulty is not only missed content but also sustained visual-search burden.
The system is motivated by the observation that many low-vision viewers prefer to use residual vision rather than substitute the visual channel entirely with audio description. On that view, the central accessibility problem is not merely missing narration, but uncertainty about whether something changed, where it changed, and what kind of visual event is worth tracking. VeasyGuide therefore treats presentation-video accessibility as a visual-search problem rather than as a purely linguistic description problem.
The paper supports the prevalence of this problem through an empirical scan of 300 YouTube presentation videos across five academic domains. Visual activities appeared in all domains: 20.6% of Applied Science videos, 43.3% of Formal Science videos, 24.6% of Humanities videos, 50.0% of Natural Science videos, and 34.4% of Social Science videos contained at least one such activity. This situates instructor gestures and marks as recurrent pedagogical signals rather than rare exceptions (Sechayk et al., 29 Jul 2025).
2. Participatory design and design implications
VeasyGuide emerged from a co-design study with three low-vision participants. The participants were C1, a 36-year-old man with albinism, astigmatism, photophobia, and nystagmus who used a screen magnifier; C2, a 30-year-old woman with retinitis pigmentosa, narrow field of view, and limited night vision; and C3, a 33-year-old woman with Duane syndrome. All conditions were congenital. The study was conducted remotely over Zoom as five one-hour sessions per participant across five weeks.
The study found that missing visual information in presentation videos was routine. Participants reported difficulty with small pointers, low-contrast colors, thin digital pen lines, and overlapping sketches. Marking actions were described as especially difficult because they were both brief and subtle. Participants also differed in visual preferences: one preferred darker content, while the other two preferred bright backgrounds. This heterogeneity led directly to a design emphasis on personalization rather than a single fixed rendering style.
Participants described existing coping strategies such as pausing the video, using a screen magnifier, or taking screenshots and enlarging them separately. These strategies were effective only at the cost of flow disruption and multiple manual steps. The study distilled four final design implications: DI1 Visual Access, DI2 Familiarity, DI3 Predictability, and DI4 Personalization.
Prototype iteration closely tracked these findings. An initial design used a red semi-transparent circle with adjustable diameter to highlight notable visual change and offered keyboard-triggered zoom for entire “visual activities,” defined there as sequences of consecutive visual changes lasting at least 5 seconds. Participants valued highlighting but found the circle too rigid, the red transparency difficult on dark backgrounds, and the behavior too noisy when every local change was emphasized. The final design therefore shifted to box-shaped highlights, optional pointer icons, optional animation, integrated zoom with smooth panning between activities, optional auto-pause during zoom, adjustable animation speed, post-processing filters, and a 1.5-second pre-activity trigger (Sechayk et al., 29 Jul 2025).
3. Recognition pipeline and activity model
VeasyGuide is implemented as a web-based system with two modules: an “Activity Recognition Pipeline” and an “Activity Visualization” module. The recognition pipeline is written in Python using OpenCV for image processing and NetworkX for graph operations; the frontend is built in React and the backend in Flask.
The first recognition stage is activity detection. To avoid mistaking slide transitions for instructor actions, the video is first split into shots using PySceneDetect. Each shot is divided into non-overlapping segments, each containing up to one-third of a second’s worth of frames, based on the video’s frame rate. For each segment, the system computes the difference between the first and last frame, extracts contours using grayscale thresholding, and filters out contours smaller than of the frame area. The remaining contours are called Regions of Change, or RoC.
Each RoC becomes a graph node represented by a tuple containing the segment timestamp, the top-left coordinates of the RoC’s bounding box, and the bounding box width and height. Nodes are linked by edges when their RoCs are close in time and space: timestamps must differ by less than , and the minimum distance between the two bounding boxes must be less than of the frame diagonal. Edge weights are computed from Hu moments. To suppress transient artifacts such as pointer trails, VeasyGuide merges RoC nodes if three conditions hold: the timestamps are temporally adjacent based on the segment duration, the regions occupy the same position, and the visual difference between them is below a Hu-moment threshold of $0.5$.
The second stage is activity grouping. Connected components of the graph are treated as candidate activities. For each connected component, the system computes a minimal bounding box covering all component nodes and the time span from earliest to latest node. That tuple—connected component, overall bounding box, and start/end timestamps—defines an activity. The authors note that they considered rule-based classification into pointing, marking, or sketching, but did not expose activity-type-specific visual encodings because co-design participants preferred consistent highlight styles. VeasyGuide therefore localizes activities in space and time, but the released system does not surface semantic activity labels to viewers (Sechayk et al., 29 Jul 2025).
4. Visualization architecture and personalization controls
The visualization module uses activity bounds and timestamps to guide attention during playback. VeasyGuide overlays visual highlights in real time and provides optional magnification. If multiple activities overlap at a given moment, the system first filters activities to those whose time span fully contains the current playback time; if several remain, it selects the one whose start time is nearest to the current time.
The highlight is box-shaped by default because participants judged a box to preserve spatial context better than a circle. The system can also add a familiar pointer icon, either a cursor or hand icon, because participants found it easier to attend when they knew what kind of visual object to expect. A 1.5-second pre-activity trigger is used so that users can prepare their gaze before the action starts. Optional animation can vary highlight size to increase noticeability, although it is off by default because some participants found it distracting or uncomfortable. The system also includes post-processing filters, specifically color inversion and line thickening.
Magnification is toggled with the key. When enabled, VeasyGuide zooms into the most recent instructor activity and centers that region in the viewport. While magnified, the highlight is hidden to reduce clutter. As new activities are detected, the zoomed view smoothly repositions to follow them rather than snapping or zooming out. The interface is explicitly WYSIWYG: highlight and zoom settings apply immediately while the video is playing, enabling in-context adjustment.
The default settings reported in the paper are as follows (Sechayk et al., 29 Jul 2025):
| Module | Setting | Default |
|---|---|---|
| Highlight | Fill Color | Yellow |
| Highlight | Fill Opacity | 15% |
| Highlight | Border Color | Red |
| Highlight | Border Width | 4px |
| Highlight | Shape Style | Box |
| Highlight | Scale | 100% |
| Highlight | Animation Style | None |
| Highlight | Animation Speed | 1.0x |
| Highlight | Pointer Style | Hand |
| Highlight | Pointer Scale | 100% |
| Highlight | Filter | None |
| Zoom | Strength | 50% |
| Zoom | Speed | 1.0x |
| Zoom | Pause on Zoom | False |
| Zoom | Sharpness | 1.0x |
| Zoom | Filter | None |
This control surface is central to the system’s accessibility model. Fill and border controls address contrast needs, pointer and border scaling strengthen saliency, animation can support users with narrow field of view, inversion can reduce eye strain for some users, and “Pause on Zoom” gives additional inspection time when needed.
5. Empirical evaluation
The main evaluation involved 8 low-vision participants and 8 sighted participants. Low-vision sessions were 1.5-hour remote Zoom studies conducted on participants’ own computers with their usual assistive tools. The study used six real educational videos. Four short videos, V1–V4, were used for a localization task; two longer videos, V5–V6, were used for a viewing task. The baseline player matched VeasyGuide except that highlights and magnification were disabled.
The localization task tested whether VeasyGuide helps viewers notice instructor actions faster and more reliably. Participants watched V1–V4 and pressed Enter when they visually identified a cued activity. In this task, VeasyGuide used highlights only, with default settings, and magnification was disabled. VeasyGuide significantly improved detection success: mean success rose from $0.61$ in baseline to $0.88$ with VeasyGuide, with Welch’s , , and . The mean rate of improvement in success rates between participants was 0. Detection speed improved numerically but not significantly: median search time changed from 1 (2) to 3 (4), and mean search time changed from 5 (6) to 7 (8) (Sechayk et al., 29 Jul 2025).
Behavioral observations in the localization task are notable. In the baseline condition, several low-vision participants repeatedly paused playback, and several used screen magnification, sometimes in combination with pausing. In the VeasyGuide condition, no participants paused playback during localization, and magnification use decreased substantially. Only two users used magnification there, mainly as a complement to highlights rather than as a primary search strategy. This suggests that the system altered the mechanics of search, not only final success rates.
The viewing task tested broader viewing experience. Participants watched V5 and V6 for comprehension, then completed a six-question quiz after each video. In this task VeasyGuide included both highlighting and automatic zoom, and participants could personalize settings. VeasyGuide did not significantly change quiz performance: mean quiz scores for low-vision participants were 9 in baseline and 0 with VeasyGuide, with mean completion times 1 and 2 minutes, respectively.
Its strongest effect in the viewing task was workload reduction. A paired Wilcoxon Signed-Rank Test on NASA-TLX dimensions found statistically significant reductions across all assessed dimensions, each with large effect size (3): Mental demand, median difference 4, 5, 6, 7, 8; Physical demand, median difference 9, $0.5$0, $0.5$1, $0.5$2, $0.5$3; Temporal demand, median difference $0.5$4, $0.5$5, $0.5$6, $0.5$7, $0.5$8; Effort, median difference $0.5$9, 0, 1, 2, 3; and Frustration, median difference 4, 5, 6, 7, 8.
Personalization was used actively. In the viewing task, most low-vision users changed settings, especially fill and border styles; fill style was the most changed setting, followed by border style and filters. Participants with limited field of view enabled animation. Most found the optional pointer useful; only two disabled it. Several used inversion filters to reduce eye strain or improve contrast.
The sighted-user study produced near-ceiling objective results in the localization task, with no significant differences in success or speed. Subjectively, however, sighted participants reported that highlights improved focus and clarified what the instructor considered important, while zoom reduced clutter by narrowing attention to the active region. The paper treats this as suggestive evidence that the system may have universal-design value, although not as strong evidence of generalized objective performance gains (Sechayk et al., 29 Jul 2025).
6. Interpretation, limitations, and research context
The paper’s broader interpretation is that VeasyGuide contributes a design framework for low-vision visual-search support built on four principles: familiarity with target visuals, predictability of target spatial context, personalization with instant visual feedback, and support for user agency. In that framing, the box highlight and pointer icon supply familiarity, the pre-activity cue and spatially extended box supply predictability, the settings panels and WYSIWYG updates supply personalization, and the emphasis on residual vision preserves agency. The authors further report that the detection success gap between low-vision and sighted participants was reduced by 9 and the detection speed gap by $0.61$0. This suggests a narrowing of performance disparity rather than a replacement of one perceptual mode by another.
Several misconceptions are explicitly ruled out by the system design and results. VeasyGuide is not an audio-description system; it is an augmentation system for the visual channel. It does not provide semantic grounding of slide referents; its current detection is context-free and motion/change-based. It also does not demonstrate significant quiz-score improvement on the reported viewing-task measure. Its strongest empirical contributions are improved detection of instructor actions, reduced reliance on disruptive search tactics, and significantly reduced workload.
The paper also identifies substantive limitations. The sample sizes are small: 3 co-design participants and 8 low-vision participants in the main evaluation. The localization task used fixed default highlight settings rather than personalized ones. Longer-term effects, including possible visual fatigue, were not established. Technically, the current activity detector uses motion/change heuristics rather than contextual understanding of slide content. Proposed future directions include context-based cues, highlighting the referenced slide element rather than the pointer itself, previewing sketch strokes before they fully appear, activity-based video navigation, stronger user control over magnification, support for live lectures, and less intrusive highlighting strategies for domains such as movies (Sechayk et al., 29 Jul 2025).
Within the broader literature, VeasyGuide occupies a specific position. “Virtual Guidance as a Mid-level Representation for Navigation with Augmented Reality” (Yang et al., 2023) converts non-visual guidance into AR-like overlays for autonomous navigation policies, whereas VeasyGuide applies overlay-based guidance to recorded educational video for low-vision learners. “From Struggle to Success: Context-Aware Guidance for Screen Reader Users in Computer Use” (Chen et al., 26 Jan 2026) shows a parallel accessibility pattern in which context-aware assistance reduces workload by aligning guidance with users’ perceptual and interaction constraints. “GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance” (Kim et al., 17 Mar 2025) similarly emphasizes accessibility-aware guidance that combines scene context, hazard information, and concise action-oriented output. “Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models” (Ling et al., 13 May 2026) extends the idea of lightweight visual intervention into embodied robot control through points, boxes, and traces. A plausible implication is that VeasyGuide belongs to a broader class of systems in which low-friction visual cues are used to reshape attention and reduce search cost, but its contribution is specialized to presentation-video accessibility rather than navigation or manipulation.