CC-Riddle: A Question Answering Dataset of Chinese Character Riddles (2206.13778v2)
Abstract: The Chinese character riddle is a unique form of cultural entertainment specific to the Chinese language. It typically comprises two parts: the riddle description and the solution. The solution to the riddle is a single character, while the riddle description primarily describes the glyph of the solution, occasionally supplemented with its explanation and pronunciation. Solving Chinese character riddles is a challenging task that demands understanding of character glyph, general knowledge, and a grasp of figurative language. In this paper, we construct a \textbf{C}hinese \textbf{C}haracter riddle dataset named CC-Riddle, which covers the majority of common simplified Chinese characters. The construction process is a combination of web crawling, LLM generation and manual filtering. In generation stage, we input the Chinese phonetic alphabet, glyph and meaning of the solution character into the generation model, which then produces multiple riddle descriptions. The generated riddles are then manually filtered and the final CC-Riddle dataset is composed of both human-written riddles and these filtered, generated riddles. In order to assess the performance of LLMs on the task of solving character riddles, we use retrieval-based, generative and multiple-choice QA strategies to test three LLMs: BERT, ChatGPT and ChatGLM. The test results reveal that current LLMs still struggle to solve Chinese character riddles. CC-Riddle is publicly available at \url{https://github.com/pku0xff/CC-Riddle}.
- K. Binsted and G. Ritchie. An implemented model of punning riddles. Technical report, University of Edinburgh, Department of Artificial Intelligence, 1994.
- K. Binsted and G. Ritchie. A symbolic description of punning riddles and its computer implementation. arXiv preprint cmp-lg/9406021, 1994.
- Transcouplet: Transformer based chinese couplet generation. arXiv preprint arXiv:2112.01707, 2021.
- Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
- Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335, 2022.
- Cryptonite: A cryptic crossword benchmark for extreme ambiguity in language. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4186–4192, 2021.
- Riddle generation using word associations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 2407–2412, 2016.
- Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019.
- Component-enhanced chinese character embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 829–834, 2015.
- Glyphcrm: Bidirectional encoder representation for chinese character with its glyph. arXiv preprint arXiv:2107.00395, 2021.
- Riddlesense: Reasoning about riddle questions featuring linguistic creativity and commonsense knowledge. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1504–1515, 2021.
- Multi-prototype chinese character embedding. In Proceedings of the tenth international conference on language resources and evaluation (LREC’16), pages 855–859, 2016.
- Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318, 2002.
- N. Reimers and I. Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, 2019.
- Cpt: A pre-trained unbalanced transformer for both chinese language understanding and generation. arXiv preprint arXiv:2109.05729, 2021.
- Chinesebert: Chinese pretraining enhanced by glyph and pinyin information. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2065–2075, 2021.
- Solving and generating chinese character riddles. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 846–855, 2016.
- Eigencharacter: An embedding of chinese character orthography. In Proceedings of the Beyond Vision and LANguage: inTEgrating Real-world kNowledge (LANTERN), pages 24–28, 2019.
- Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38–45, 2020.
- C. Xiang-li. Chinese character rhetoric means as embodied in logogriph. Journal of Xiangnan University, 2008.
- Generating chinese classical poems with rnn encoder-decoder. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data, pages 211–223. Springer, 2017.
- Y. Zhang and X. Wan. BirdQA: A bilingual dataset for question answering on tricky riddles. arXiv preprint arXiv:2109.11087, 2021.
- Fan Xu (114 papers)
- Yunxiang Zhang (22 papers)
- Xiaojun Wan (99 papers)