Word-Level American Sign Language (WLASL)

• WLASL is a large-scale ASL dataset featuring 21,083 clips, 2,006 glosses, and diverse signers to benchmark automatic recognition of isolated signs.
• It supports multi-modal research by providing both RGB videos and detailed pose keypoints, enabling spatiotemporal and kinematic analysis.
• Advancements using WLASL have driven improvements in deep learning architectures, including attention-based and graph models, to enhance sign recognition accuracy.

Word-Level American Sign Language (WLASL) is a large-scale video dataset and an associated research paradigm supporting the development, benchmarking, and analysis of deep learning methods for automatic recognition of isolated American Sign Language (ASL) signs. Characterized by its extensive vocabulary, signer diversity, and challenging real-world variability in signing conditions, WLASL has become the principal platform for evaluating word-level ASL recognition models using both RGB visual and pose-based inputs. This article surveys the structure, evolution, and methodological advances enabled by WLASL, contextualizing its impact and current state in computational sign language research.

1. Dataset Composition and Annotation Protocols

WLASL comprises 21,083 video clips representing 2,006 ASL glosses, produced by 119 unique signers across diverse recording environments. Glosses are single-word ASL units, mined and selected from educational resources and verified to ensure semantic distinctness. The per-gloss sample count ranges from 4 to 83, with a mean of approximately 10.5, yielding substantial class imbalance and a long-tailed distribution. Video properties are highly variable, spanning spatial resolutions (320×240 up to 1920×1080), durations (mode ≈2.8 s, σ = 0.9 s), and frame rates (typically 20–30 fps). Signers represent a broad but uncategorized demographic spectrum, and many videos were crowd-sourced from public forums and vlogs, lacking standardized camera setups or backgrounds (Rahman et al., 9 Jul 2025).

Annotation is performed at the gloss level, with start and end times for the sign within each video, though temporal granularity varies. No demographic or handedness meta-data are systematically encoded. WLASL provides standard nested evaluation splits: WLASL100 (100 glosses), WLASL300, WLASL1000, and WLASL2000 (full set), each stratified to maintain class representation across train/validation/test partitions. The splits enable benchmarking at increasing levels of vocabulary size and recognition difficulty (Rahman et al., 9 Jul 2025).

2. Modalities and Preprocessing: RGB, Skeletal, and 3D Keypoints

WLASL supports both holistic visual (RGB) and pose-based (skeleton, 2D/3D keypoints) modalities, facilitating methodological diversity. RGB-based pipelines exploit spatio-temporal cues from raw frames, requiring preprocessing such as signer detection, cropping, and temporal alignment. In contrast, pose-based approaches extract per-frame 2D or 3D joint locations using keypoint detectors (e.g., OpenPose, MediaPipe Holistic, MMPose HRNet) followed by normalization (wrist centering, z-score, or [−1, 1] scaling), dynamic time warping, and temporal resampling (often to 50–120 frames per clip) (Alishzade et al., 17 Nov 2025, Song et al., 2022).

Extended datasets (e.g., Sign3D-WLASL) produce 3D keypoint trajectories via fusion of multiple pose-estimation pipelines, resulting in a tensor $X\in\mathbb{R}^{T\times3K}$ with $T$ frames and $K$ keypoints. Missing or occluded keypoints are handled via interpolation and quality filtering, enforcing a minimum detection rate per clip (Rahman et al., 9 Jul 2025).

3. Baseline and State-of-the-Art Recognition Architectures

3.1 Appearance-Based and Hybrid Models

Baseline models on WLASL include Inflated 3D ConvNets (I3D), which learn spatiotemporal representations from RGB sequences; top-1 accuracy on WLASL100 ranges from 65.89 % (I3D) to 80.72 % (Full Transformer using RGB inputs) (Brettmann et al., 10 Apr 2025). Multi-Stream Neural Networks (MSNN) integrate global signer appearance, cropped hand and face substreams, and pose-based streams via late score-level fusion, achieving up to 81.38 % top-1 on WLASL100 (and 47.26 % on WLASL2000), outpacing single-stream baselines (Maruyama et al., 2021). Ablation studies indicate that local/regional substreams focusing on handshape and facial articulators yield substantial gains, especially for fine-grained lexical distinctions.

3.2 Pose-Based Methods and Graph Models

Pose-based pipelines leverage explicit body and hand kinematics. Early architectures include Pose-TGCN (Temporal Graph Convolutional Network), ST-GCN, and GRU-based models, with Pose-TGCN attaining ~55.43 % top-1 on WLASL100 (Laines et al., 2023). Subsequent advances utilize spatial-temporal decomposition: GCN-BERT models perform spatial GCN encoding per frame, followed by temporal BERT-style self-attention, achieving 60.15 % on WLASL100 and a ~5 pp improvement over prior graph-based networks (Tunga et al., 2020). SLGTformer adopts a fully attention-based backbone, using Learnable Graph Relative Positional Encoding and Temporal Twin Self-Attention blocks, establishing a keypoint-only, non-ensemble state-of-the-art of 47.42 % top-1 on WLASL2000 (Song et al., 2022).

Table: WLASL100 Top-1 Accuracy (Representative Models)

Model	Input	Top-1 (%)
I3D (Li et al. 2020)	RGB	65.89
Full Transformer (Du et al.)	RGB	80.72
MSNN (all streams)	RGB+pose	81.38
Pose-TGCN	Skeleton	55.43
GCN-BERT	Skeleton	60.15
SL-TSSI-DenseNet + DA	Skeleton	81.47

3.3 Vision Transformers and Self-Attention

Video Vision Transformers (ViViTs), such as TimeSformer and VideoMAE, surpass CNNs by exploiting global spatiotemporal self-attention. VideoMAE achieves 75.58 % top-1 on WLASL100, outperforming both CNN-based I3D and pure RGB-based ResNet/Transformer baselines (Brettmann et al., 10 Apr 2025). Ablations confirm that tube-based masking and even frame sampling are critical for robust feature learning, and that self-attention maps focus adaptively on hands and upper body regions.

4. Phonology-Aware and Multi-Task Recognition

Approaches integrating ASL phonological structure into recognition networks have been shown to enhance accuracy and interpretability. By augmenting pose-based models (e.g., SL-GCN, pose-only Transformer) with auxiliary heads predicting up to 16 phonological features—such as handshape, minor location, and movement—joint optimization yields up to 8.7 percentage point absolute gains on WLASL2000 (from 29.4 % to 38.1 % top-1 on SL-GCN), even when only 48 % of signs are annotated for phonology. Notably, handshape and minor location contribute the single largest performance jumps, indicating that modeling ASL’s sublexical structure regularizes the latent space and aids recognition of visually similar signs (Kezar et al., 2023).

A plausible implication is that joint phoneme-gloss multitask learning can serve both as an inductive bias for neural models and as a tool for linguistic analysis within sign language datasets.

5. Methodological Trends and Comparative Analysis

Comparison across recurrent (ConvLSTM), attention-based (Vanilla Transformer), and classical CNN or GCN models on WLASL2000 confirms a consistent accuracy advantage for attention architectures (88.3 % top-1 for Transformer vs. 85.3 % for ConvLSTM), with additional benefits in signer-independence and lower cross-validation variance. However, ConvLSTM models remain more efficient in inference latency (~25 ms per sample vs. 60 ms) and FLOPs. Trade-offs depend on deployment context: low-latency applications favor ConvLSTM, while highest-accuracy or batch-mode scenarios benefit from Transformers (Alishzade et al., 17 Nov 2025).

Experimental ablations reinforce several consistent findings: temporal augmentation (speed jittering or resampling) is crucial for pose-based methods; local subregions and explicit hand/facial cues systematically yield gains for RGB and multimodal pipelines; and performance degrades as vocabulary size increases (from ~81 % top-1 at 100 classes to ~47 % at 2,000).

6. Limitations, Biases, and Future Directions

The WLASL dataset’s size and diversity expose several open challenges. Gloss frequency is strongly long-tailed, so rare-words remain underrepresented, hindering model generalization on low-sample classes. Signer demographic information (age, gender, handedness) is not provided, complicating assessment of dataset bias. Variations in video quality, signer perspective, and keypoint extraction robustness (especially under occlusion, loss of depth cues, or unconventional handshapes) introduce further noise. Multi-stream and keypoint-based pipelines are sensitive to errors in pose estimation, particularly for hands and fingers (Rahman et al., 9 Jul 2025, Song et al., 2022).

Potential remedies include expanded phonological annotation coverage, semi- or weakly-supervised pretraining, viewpoint-invariant augmentation, adaptive domain normalization, and tighter integration of multimodal cues (hand, face, mouth region). Future research aims to extend WLASL-style benchmarking to continuous and sentence-level SLR, integrate 3D kinematic signals (e.g., via Sign3D-WLASL), and develop robust signer-adaptive models using the full multimodal spectrum.

7. Practical Impact and Applications

WLASL’s design has enabled the creation, benchmarking, and cross-comparison of most prominent isolated sign language recognition pipelines published since 2020. Emerging downstream applications include speech-to-sign animation systems (e.g., Speak2Sign3D) that support spoken English to ASL gloss translation and realistic 3D human motion synthesis, using WLASL-derived glosses and pose trajectories as supervision. The dataset is publicly available for non-commercial research under CC-BY-4.0 annotation licenses, though video redistribution remains bound by original source terms. WLASL’s influence extends to allied fields including human action recognition, multi-modal translation, and sign language phonology (Rahman et al., 9 Jul 2025, Rahman et al., 9 Jul 2025).

The scientific community continues to use WLASL as the principal touchstone for evaluating isolated ASL recognition, phonological modeling, and multimodal fusion, driving methodological innovation and deeper understanding of sign language structure in machine learning contexts.