BdSLIG: Bengali Sign Instruction Generation Benchmark

• BdSLIG is a benchmark that defines sign instruction generation by producing step-by-step Bengali instructions for sign gestures.
• It employs SPI prompting by integrating core sign parameters like handshape, movement, orientation, and location to ensure structured, reproducible outputs.
• The dataset features 60 unique sign classes used in zero-shot evaluations on vision-language models, highlighting its role in under-resourced instructional domains.

BdSLIG is the first Bengali Sign Language Instruction Generation benchmark, designed for Sign Language Instruction Generation (SLIG), a task in which a model produces step-by-step textual instructions that enable non-SL users to imitate and learn sign gestures. In the reported formulation, BdSLIG serves as a zero-shot evaluation set for Vision LLMs (VLMs) on two axes: under-resourced SLIG and long-tail visual concepts, with the additional motivation that Bengali SL is unlikely to appear in VLM pre-training data. The dataset is paired with Sign Parameter-Infused (SPI) prompting, which inserts standard sign parameters such as handshape, movement, orientation, and location into the prompt so that generated instructions are more structured and reproducible than free-form natural text from vanilla prompting (Tariquzzaman et al., 22 Aug 2025).

1. Research setting and scope

BdSLIG occupies a specific niche within computational work on Bangla or Bangladeshi sign language. It is not a recognition corpus in the sense of a real-time classifier, and it is not an overview benchmark in the sense of image generation. Its purpose is instructional: given a sign video, the model must generate Bengali textual instructions that describe how to perform the sign (Tariquzzaman et al., 22 Aug 2025).

This focus distinguishes BdSLIG from adjacent systems. PerSign formulates a personalized image-to-image translation problem in which a profile image $I_P$ and a target Bengali letter $L$ are used to generate an image $I_L$ of the same person making the corresponding Bangladeshi Sign Language letter, while preserving face, skin tone, attire, and background (Jubair et al., 2022). The “Real-time Bangla Sign Language Translator” instead addresses recognition from video by combining MediaPipe Holistic with a three-layer LSTM and reports 94% accuracy and 93% $F_1$-score on isolated Bangla signs (Pranto et al., 2024). BdSLIG therefore complements recognition and synthesis research by targeting instruction generation rather than classification or visual rendering.

A common source of confusion is to conflate “instruction generation” with “translation” in the conventional sign-to-text sense. BdSLIG does not primarily seek a gloss or label; it evaluates whether a VLM can produce stepwise procedural guidance for sign imitation. This suggests a shift from semantic decoding alone toward pedagogical and reproducible descriptions of articulated motion (Tariquzzaman et al., 22 Aug 2025).

2. Dataset composition and annotation protocol

BdSLIG contains $N=60$ unique sign classes, with one representative video per BdSLW60 word, yielding a total of $60$ video-instruction pairs. In the reported experimental setup, the entire dataset is used as an evaluation set for zero-shot prompting, with no fine-tuning on BdSLIG:

$$|D_{\text{train}}|=0,\quad |D_{\text{val}}|=0,\quad |D_{\text{test}}|=60.$$

The source material consists of BdSLW60 one-word sign clips at 30 fps (Tariquzzaman et al., 22 Aug 2025).

The dataset creation pipeline is explicitly staged. First, videos are sourced from BdSLW60. Second, uniform frame sampling is applied every 20th frame, producing approximately 5–6 frames per clip. Third, three trained annotators write step-by-step instructions in Bengali, guided by the seven sign parameters defined in the work. Fourth, a two-stage validation procedure is applied: a grammatical and semantic pass by a second annotator, followed by a final check by a Bengali SL expert. Fifth, the validated steps are assembled into the final instruction sequence for each video (Tariquzzaman et al., 22 Aug 2025).

Because BdSLIG is used only for zero-shot evaluation in the reported experiments, it should not be interpreted as a trainable large-scale corpus in its current form. The design is closer to a benchmark for probing model capability under data scarcity than to a conventional supervised dataset for end-to-end optimization.

3. Sign parameter taxonomy and instruction representation

The dataset is organized around a sign-parameter taxonomy consisting of seven categories: Handshape (HS), Movement (MO), Location (LO), Palm Orientation (OR), Spatial Interaction (SI), Temporal Dynamics (TD), and Facial Cues (FC). For brevity, the core SPI template uses the first four parameters: HS, MO, OR, and LO (Tariquzzaman et al., 22 Aug 2025).

Parameter	Representative values
HS	flat, curved, pointed, bent, spread, twisted, hooked
MO	static, linear, circular, to-and-fro, oscillation, squeeze, rotate
LO	head, forehead, eye, ear, nose, mouth, chest, shoulder, neutral-space
OR	palm_up, palm_down, inward, outward, left, right
SI	right_active, left_active, both_symmetric, both_asymmetric
TD	hold, repeat, oscillate, tap
FC	neutral, brows_raised, brows_furrowed, mouth_open, mouth_closed, lips_pursed

Each of the four core parameters is annotated once for every video, yielding 60 annotations per category. The paper characterizes the resulting distribution as moderately imbalanced and long-tail in nature, while also noting that per-parameter frequencies or entropies are not reported explicitly. A plausible implication is that BdSLIG is intended not only as a low-resource benchmark, but also as a probe for compositional and underrepresented visual-linguistic structure (Tariquzzaman et al., 22 Aug 2025).

The annotation style is procedural rather than merely descriptive. For the “Toothpaste” sign, the paper gives a two-step example. Step 1 uses the tags [HS:CURVED] [MO:UP] [OR:PALM_IN] [LO:MOUTH] and the Bengali instruction “ডান হাতের আঙ্গুলগুলো নরম বেণুতে বাঁকানো অবস্থায় মুখের সামনে নিয়ে আসুন এবং হালকাভাবে ওভারহেড স্থানে থামান।” Step 2 uses [HS:CLOSED] [MO:SCRAPE] [OR:PALM_DOWN] [LO:MOUTH] and the instruction “কনিষ্ঠাঙ্গুলি দিয়ে দাঁতের উপর পাতলা রেখা আঁকার মতো স্ক্র্যাপিং মুভমেন্ট করুন।” This representation makes the link between visual articulation and linguistic instruction explicit (Tariquzzaman et al., 22 Aug 2025).

4. Sign Parameter-Infused prompting

SPI prompting formalizes the insertion of sign parameters into the textual prompt given to a generative VLM. Let $\mathcal{P}_0$ denote a base prompt and let $$\mathcal{S}=\{\alpha_{HS},\alpha_{MO},\alpha_{OR},\alpha_{LO}\}$$ denote the parameter set over the four core categories. The Sign Parameter-Infused prompt is defined as

$$f(\mathcal{P}_0,\mathcal{S}) \;=\; \bigl\langle \underbrace{\text{“Describe the sign in Bengali:”}}_{\mathcal{P}_0}, \,\alpha_{HS}([HS]), \,\alpha_{MO}([MO]), \,\alpha_{OR}([OR]), \,\alpha_{LO}([LO]) \bigr\rangle.$$

Examples given in the paper include $L$0 “handshape: curved” and $L$1 “movement: circular” (Tariquzzaman et al., 22 Aug 2025).

The contrast with vanilla prompting is also written explicitly. If $L$2 is a generative VLM, $L$3 denotes sampled frames from video $L$4, and $L$5 is the output token sequence, then vanilla prompting is

$L$6

whereas SPI prompting is

$L$7

In this formulation, SPI is a prompting strategy rather than a separate learned module. No fine-tuning on BdSLIG is performed in the reported experiments. The principal claim is that subsuming standard sign parameters into the prompt makes the generated instructions more structured and reproducible than free-form natural text from vanilla prompting (Tariquzzaman et al., 22 Aug 2025).

5. Evaluation protocol and empirical results

The evaluation is conducted in a zero-shot setting on expert-annotated BdSLIG instructions. The VLMs listed in the paper are GPT 4.1-mini, GPT 4.1, Gemini 2.0 Flash, and Gemini 2.5 Pro. The reported metrics include BLEU-$L$8, ROUGE-L, METEOR, BERTScore, ROUGE-1, and ROUGE-2 (Tariquzzaman et al., 22 Aug 2025).

For the main SPI setting, the paper reports key results for GPT 4.1 and Gemini 2.0 Flash:

Model	Scores
GPT 4.1	ROUGE-1 0.553; ROUGE-2 0.271; BLEU 0.196; METEOR 0.492
Gemini 2.0 Flash	ROUGE-1 0.505; ROUGE-2 0.250; BLEU 0.168; METEOR 0.359

The ablation comparing vanilla prompting with SPI for GPT 4.1 is central to the argument for parameter infusion:

Prompt	Scores
Vanilla	ROUGE-1 0.528; BLEU 0.149; METEOR 0.416
SPI	ROUGE-1 0.553; BLEU 0.196; METEOR 0.492

These numbers show that the reported gains are not merely architectural, since the compared condition is the prompt format rather than a change in the underlying VLM. This suggests that explicitly supplying sign-parameter structure is beneficial in a setting where the target sign language is underrepresented in pre-training data (Tariquzzaman et al., 22 Aug 2025).

6. Limitations, downstream uses, and relation to adjacent Bangla sign-language research

The paper identifies several limitations. BdSLIG is small, with 60 classes and 1 video per class, and it therefore requires multi-instance coverage for broader generalization. Uniform frame sampling may miss key movement phases, motivating future work on key-frame or motion-aware sampling. Instruction variations remain subjective, and metric-based evaluation is acknowledged as imperfect, with LLM-based or human-in-the-loop evaluation proposed as possible improvements. The paper also proposes extending SPI to include Spatial Interaction, Temporal Dynamics, and Facial Cues, and exploring fine-tuning on BdSLIG or larger expansions to reduce the gap from zero-shot performance (Tariquzzaman et al., 22 Aug 2025).

The downstream applications named in the work are sign-language tutoring systems, interpretability tools for SL recognition, and sign-to-text feedback for learners. In that sense, BdSLIG fits into a broader Bangla sign-language AI landscape. PerSign targets personalized letter synthesis to reduce the communication gap between signers and non-signers by generating a person-specific signed image from a profile image and a target letter (Jubair et al., 2022). The real-time translator targets live detection with a MediaPipe Holistic plus LSTM pipeline and reports real-time performance on an AMD Ryzen 7 5700X CPU, NVIDIA GeForce RTX 3060 GPU, and 16 GB RAM setup (Pranto et al., 2024). BdSLIG adds an instructional layer to this landscape by focusing on how signs can be explained procedurally to non-signers rather than merely recognized or rendered.

Taken together, these works indicate a diversification of Bangla or Bangladeshi sign-language research across recognition, synthesis, and instruction. A plausible implication is that future systems may combine these functions: recognition for input understanding, synthesis for output visualization, and instruction generation for pedagogy and user feedback.