A Hong Kong Sign Language Corpus Collected from Sign-interpreted TV News
Abstract: This paper introduces TVB-HKSL-News, a new Hong Kong sign language (HKSL) dataset collected from a TV news program over a period of 7 months. The dataset is collected to enrich resources for HKSL and support research in large-vocabulary continuous sign language recognition (SLR) and translation (SLT). It consists of 16.07 hours of sign videos of two signers with a vocabulary of 6,515 glosses (for SLR) and 2,850 Chinese characters or 18K Chinese words (for SLT). One signer has 11.66 hours of sign videos and the other has 4.41 hours. One objective in building the dataset is to support the investigation of how well large-vocabulary continuous sign language recognition/translation can be done for a single signer given a (relatively) large amount of his/her training data, which could potentially lead to the development of new modeling methods. Besides, most parts of the data collection pipeline are automated with little human intervention; we believe that our collection method can be scaled up to collect more sign language data easily for SLT in the future for any sign languages if such sign-interpreted videos are available. We also run a SOTA SLR/SLT model on the dataset and get a baseline SLR word error rate of 34.08% and a baseline SLT BLEU-4 score of 23.58 for benchmarking future research on the dataset.
- A comprehensive study on deep learning-based methods for sign language recognition. IEEE TMM, 24:1750–1762.
- BSL-1K: scaling up co-articulated sign language recognition using mouthing cues. In ECCV, volume 12356, pages 35–53. Springer.
- BBC-Oxford British sign language dataset. CoRR, abs/2111.03635.
- The American sign language lexicon video dataset. In CVPR, pages 1–8.
- LSA-T: the first continuous argentinian sign language dataset for sign language translation. In Advances in Artificial Intelligence - IBERAMIA 2022 - 17th Ibero-American Conference on AI, Cartagena de Indias, Colombia, November 23-25, 2022, Proceedings, volume 13788 of Lecture Notes in Computer Science, pages 293–304. Springer.
- Neural sign language translation. In CVPR, pages 7784–7793.
- Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In CVPR.
- The devisign large vocabulary of Chinese sign language database and baseline evaluations. In Technical report VIPL-TR-14-SLR-001. Key Lab of Intelligent Information Processing of Chinese Academy of Sciences (CAS). Institute of Computing Technology.
- A simple multi-modality transfer learning baseline for sign language translation. In CVPR, pages 5120–5130.
- Two-stream network for sign language recognition and translation. In NeurIPS.
- Deep residual learning for image recognition. In CVPR, pages 770–778.
- SignBERT: Pre-training of hand-model-aware representation for sign language recognition. In ICCV, pages 11087–11096.
- Attention-based 3D-CNNs for large-vocabulary sign language recognition. IEEE TCSVT., 29(9):2822–2832.
- Video-based sign language recognition without temporal segmentation. In AAAI, pages 2257–2264.
- Skeleton aware multi-modal sign language recognition. In CVPRW, pages 3413–3423.
- Chinese lexical analysis with deep bi-gru-crf network. arXiv preprint arXiv:1807.01882.
- Whole-body human pose estimation in the wild. In ECCV, pages 196–214.
- Hamid Reza Vaezi Joze and Oscar Koller. 2019. MS-ASL: A large-scale data set and benchmark for understanding american sign language. In BMVC, page 100.
- Towards automatic speech to sign language generation. In Interspeech, pages 3700–3704.
- Neural sign language translation based on human keypoint estimation. CoRR, abs/1811.11436.
- Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers. CVIU, 141:108–125.
- Temporal convolutional networks: A unified approach to action segmentation. In ECCVW, Lecture Notes in Computer Science, pages 47–54.
- Vladimir I Levenshtein et al. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707–710. Soviet Union.
- Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison. In WACV, pages 1448–1458.
- Is word segmentation necessary for deep learning of Chinese representations? In ACL, pages 3242–3252.
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
- Watch, read and lookup: Learning to spot signs from multiple supervisors. In ACCV, volume 12627, pages 291–308.
- BosphorusSign22k sign language recognition dataset. CoRR, abs/2004.01283.
- BLEu: A method for automatic evaluation of machine translation. In ACL, pages 311–318.
- U-net: Convolutional networks for biomedical image segmentation. In MICCAI, volume 9351, pages 234–241.
- The ASL-LEX 2.0 project: A database of lexical and phonological properties for 2,723 signs in american sign language. The Journal of Deaf Studies and Deaf Education, 26(2):263–277.
- Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR.
- Ozge Mercanoglu Sincan and Hacer Yalim Keles. 2020. AUTSL: A large scale multi-modal Turkish Sign Language dataset and baseline methods. IEEE Access, 8:181340–181355.
- Junyi Sun. 2012. Jieba chinese word segmentation tool.
- Deep high-resolution representation learning for human pose estimation. In CVPR, pages 5693–5703.
- TVB News. 2023. News report with sign language. https://news.tvb.com/tc/programme/newsreportwithsignlanguage.
- Attention is all you need. In NeurIPS, pages 5998–6008.
- S-pot - a benchmark in spotting signs within continuous signing. In LREC, pages 1892–1897.
- Rethinking spatiotemporal feature learning: Speed-accuracy trade-offs in video classification. In ECCV, pages 305–321.
- QANet: Combining local convolution with global self-attention for reading comprehension. In ICLR.
- Improving sign language translation with monolingual data by sign back-translation. In CVPR, pages 1316–1325.
- Spatial-temporal multi-cue network for continuous sign language recognition. In AAAI, pages 13009–13016.
- SignBERT: a BERT-based deep learning framework for continuous sign language recognition. IEEE Access, 9:161669–161682.
- A portable sign language collection and translation platform with smart watches using a BLSTM-based multi-feature framework. Micromachines, 13(2):333.
- Ronglai Zuo and Brian Mak. 2022a. C2SLR: Consistency-enhanced continuous sign language recognition. In CVPR, pages 5131–5140.
- Ronglai Zuo and Brian Mak. 2022b. Local context-aware self-attention for continuous sign language recognition. In Interspeech, pages 4810–4814.
- Natural language-assisted sign language recognition. In CVPR.
- Towards online sign language recognition and translation. arXiv preprint arXiv:2401.05336.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.
