SignDATA: Reproducible Sign-Language Preprocessing

• SignDATA is a reproducible preprocessing framework that transforms heterogeneous raw sign-language datasets into standardized training artifacts using pose and video recipes.
• It offers two end-to-end recipes—one leveraging pose extraction via MediaPipe or MMPose and another generating signer-cropped video—to ensure backend interchangeability.
• The framework formalizes preprocessing decisions, addressing dataset fragmentation and enhancing reproducibility and experimental comparability in sign-language research.

Searching arXiv for the SignDATA paper and closely related sign-language preprocessing/context papers. SignDATA is a reproducible preprocessing framework for sign-language translation research that standardizes the conversion of heterogeneous raw sign-language corpora into training-ready artifacts. Introduced as an open-source, config-driven toolkit, it sits between raw dataset releases and downstream model training, producing either pose/keypoint representations or signer-cropped video outputs through explicit, reusable, and empirically comparable preprocessing pipelines. Its central premise is that preprocessing in sign-language translation is a research-critical systems problem rather than a minor implementation detail, because annotation schema, clip timing, signer framing, acquisition procedure, and privacy constraints vary substantially across corpora (Chen et al., 22 Apr 2026).

1. Definition and research role

SignDATA is a preprocessing layer rather than a translation model. Its purpose is to transform raw sign-language datasets into comparable outputs for learning, with the paper defining two end-to-end recipes: a pose recipe and a video recipe. The former performs acquisition, manifesting, person localization, clipping, cropping, landmark extraction, normalization, and WebDataset export; the latter replaces pose extraction with signer-cropped video packaging. The toolkit exposes interchangeable MediaPipe and MMPose backends behind a common interface, uses typed job schemas and experiment-level overrides, and organizes execution as a stage-based pipeline with deterministic run-scoped artifact paths (Chen et al., 22 Apr 2026).

This positioning is methodologically important. In much of prior sign-language translation work, preprocessing is embedded in paper-specific scripts and only partially documented. SignDATA formalizes that layer as reusable infrastructure. A plausible implication is that it makes preprocessing decisions experimentally visible in the same way that architecture, optimization, and evaluation choices are already treated as first-class research variables.

2. Motivation and problem formulation

The motivation for SignDATA is the heterogeneity of sign-language corpora. The paper identifies several axes of variation: annotation schema differences, clip timing differences, signer framing differences, acquisition differences, and privacy constraints. Examples in the system description include datasets that require web acquisition and transcript parsing, datasets that assume pre-downloaded local releases, and corpora in which signer location and scene composition differ markedly between controlled and open-domain conditions (Chen et al., 22 Apr 2026).

The toolkit is designed to address three forms of fragmentation. First, preprocessing is often paper-specific, with bespoke scripts tied to one dataset or representation. Second, preprocessing is often backend-specific, so switching from one extractor to another requires reengineering later stages. Third, the transformation from raw video to trainable tensors is often weakly documented, which impedes replication and controlled comparison. SignDATA addresses this by terminating dataset-specific logic at a canonical manifest layer and sharing all later processing across corpora. In effect, the system converts extractor choice, normalization policy, clipping behavior, and privacy-sensitive output form into explicit configuration choices rather than hidden implementation details (Chen et al., 22 Apr 2026).

The paper also frames privacy as intrinsic to the problem rather than ancillary. Sign-language videos are subject-centric: the signer’s face and body are central to the communicative signal. This makes representation choice a privacy-relevant design variable, not merely a storage or compute choice.

3. Architecture and core abstractions

At a high level, SignDATA is organized into four stages: dataset stage, processing stage, post-processing stage, and output stage. Dataset adapters perform source-specific acquisition or validation and produce a canonical manifest. One of the two processing recipes is then applied, followed by optional post-processing and sharded export such as WebDataset. The paper emphasizes that dataset heterogeneity should be handled once, at the adapter and manifest layer, after which later stages can be reused across corpora (Chen et al., 22 Apr 2026).

Several abstractions structure this design. Registries load dataset, processing, and output components at startup so they can be selected by name. Typed job schemas and typed, mergeable YAML configuration define runs as validated objects rather than informal scripts. A shared context object carries validated configuration, the active dataset adapter, run-scoped output directories, and per-stage execution metadata. The canonical manifest is the central interchange artifact; its required fields are sample_id and video_id, with optional timing, label, spatial, split, and metadata fields. The adapter layer also normalizes known column aliases so that downstream stages consume a uniform representation.

The design goals stated in the paper are summarized below.

Goal	Meaning in SignDATA
Modularity	Independent acquisition, detection, extraction, normalization, packaging stages
Reproducibility	Typed YAML configuration, explicit overrides, deterministic paths, checkpoint markers
Backend interchangeability	Extractor backends are swappable at config level
Dataset portability	Adapters converge to one canonical manifest schema
Scalability	Multi-worker processing and sharded export such as WebDataset

Checkpointing occupies an intermediate status in the current system. The abstract advertises per-stage checkpointing with config- and manifest-aware hashes, but the paper states that the checkpoint layer is only partially implemented and is not yet part of the stable execution contract. There is therefore no runtime stage-skipping on restart in the current release, and restarted runs may reprocess already-completed shards.

4. End-to-end preprocessing recipes

Both active recipes begin with the same dataset stage. Depending on the corpus, inputs may include raw videos, transcript files, alignment CSVs, split labels, signer identifiers, timing boundaries, auxiliary bounding boxes, release-directory metadata, and TSV or JSON annotations. Adapters normalize these sources into the canonical manifest and filter segments using rule-based criteria such as presence of required text fields, presence of required timing fields, text normalization, and configurable duration bounds (Chen et al., 22 Apr 2026).

The two recipes then diverge.

Recipe	Main operations	Output
Pose recipe	Sampling, localization, clipping, cropping, landmark extraction, normalization	Pose tensors in WebDataset shards
Video recipe	Detection, two-pass clipping and cropping, optional padding/resizing	Signer-cropped video in WebDataset shards

In pose mode, SignDATA reads temporally aligned segments from the manifest and samples frames at a configurable rate. Person detection is optional and may use YOLO or MMDetection; resulting bounding boxes are forwarded to the pose backend. Landmark extraction is then performed by MediaPipe Holistic or MMPose, producing per-frame keypoint arrays in a shared structured format, although the number of keypoints remains backend-specific. Post-processing can normalize outputs, reduce them to preset keypoint subsets, flatten them, and prepare them for export. Pose jobs package normalized keypoint arrays, or raw arrays if no post-processing is configured, together with caption text when available and metadata containing sample ID, video ID, timing boundaries, and processor used.

In video mode, the system performs a two-pass decoding workflow. The first pass samples frames for detection and computes a bounding region around the signer. The second pass clips and crops the source video to that region, with optional padding and resizing. The current pipeline assumes single-signer segments; multi-person clips and clips with no detection are skipped rather than propagated further. Video jobs package the resulting cropped clips with caption text when available and the same minimal metadata fields into WebDataset sharded tar archives.

5. Backend interchangeability, reproducibility, and privacy

A central technical claim of SignDATA is that MediaPipe Holistic and MMPose are exposed as interchangeable backends behind a common interface. This interchangeability is operational rather than ontological. The paper explicitly states that the outputs share a structured format, but the number of keypoints is backend-specific and the landmark semantics are not equivalent. The authors therefore conclude that MediaPipe and MMPose should be treated as different representation choices rather than simple drop-in substitutes (Chen et al., 22 Apr 2026).

The reproducibility story is built on configuration and execution mechanics. Jobs and experiments are declared in YAML; experiment files can execute ordered job lists with per-job configuration overrides; deterministic run-scoped paths isolate artifacts from different runs; and the canonical manifest provides a stable handoff object between dataset adaptation and later processing. Stage-level configuration can control frame sampling rate, detector choice, pose backend, cropping parameters, padding, resizing, normalization policy, and output format. The paper does not provide formal equations for hashes or deterministic execution, but it does describe the intended checkpointing design as config- and manifest-aware.

Privacy is treated as a first-class representational tradeoff. SignDATA supports two output families with different privacy, storage, compute, and information-retention profiles: pose-based outputs and signer-cropped video outputs. The paper stresses that pose should not be treated as an automatic anonymity mechanism, because even detailed facial landmarks may remain biometric identifiers. Cropped video generation is therefore presented as a privacy-aware processing mode rather than a universally preferable one. Planned privacy-preserving video obfuscation is identified as future work, indicating that the toolkit’s current contribution is to make privacy tradeoffs explicit and configurable rather than to solve privacy preservation outright.

6. Evaluation, limitations, and scope

The evaluation design is research-oriented and centers on backend comparison, preprocessing ablations, and privacy-aware video generation on datasets. The paper reports qualitative conclusions rather than detailed numerical benchmark tables. Its main findings are that backend choice matters; MediaPipe and MMPose differ in throughput, landmark coverage, failure modes, and behavior across controlled versus open-domain conditions; and preprocessing choices such as signer-region cropping, visibility masking, and depth-coordinate retention have measurable effects on usable-clip rate and storage footprint (Chen et al., 22 Apr 2026).

The scope of current dataset support is limited but concrete. The paper states that SignDATA currently supports How2Sign and YouTube-ASL, and it also mentions an implemented adapter path for OpenASL via a TSV-based adapter with optional bounding-box metadata. More broadly, the produced artifacts are presented as useful for sign-language AI research beyond translation, since pose tensors and signer-centric cropped video are generic upstream representations rather than model-specific outputs.

The system’s limitations are explicit. Backend outputs are not semantically equivalent, so switching backends after data generation can require re-extraction and changes in downstream modeling assumptions. The current processing path assumes one signer per segment and drops multi-person clips, which can reduce usable data in open-domain corpora. Exported WebDataset metadata is intentionally minimal and does not preserve every manifest field. Checkpoint utilities are present but not integrated into runtime stage skipping. A cross-backend landmark ontology is not yet available, so pose sequences extracted by different backends are not directly comparable. Future work identified by the paper includes runtime checkpoint integration, privacy-preserving video obfuscation, cross-backend landmark ontology normalization, and broader dataset adapter coverage beyond ASL-centric corpora.

In arXiv usage, SignDATA refers specifically to this sign-language preprocessing framework. It should be distinguished from unrelated homonymous or near-homonymous work, including the image-classification regularizer “Signed Input Regularization” (Taghanaki et al., 2019), XML-based architectures for digitally signed electronic documents (Berbecaru et al., 2019), and CSV-integrity schemes that embed digital signatures into file syntax (Ito, 2024). That distinction matters because SignDATA’s contribution lies in reproducible data systems infrastructure for sign-language research, not in cryptographic signing or in “signed” machine-learning objectives.