Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction (2406.03019v1)

Abstract: Oracle Bone Inscriptions is one of the oldest existing forms of writing in the world. However, due to the great antiquity of the era, a large number of Oracle Bone Inscriptions (OBI) remain undeciphered, making it one of the global challenges in the field of paleography today. This paper introduces a novel approach, namely Puzzle Pieces Picker (P$^3$), to decipher these enigmatic characters through radical reconstruction. We deconstruct OBI into foundational strokes and radicals, then employ a Transformer model to reconstruct them into their modern (conterpart)\textcolor{blue}{counterparts}, offering a groundbreaking solution to ancient script analysis. To further this endeavor, a new Ancient Chinese Character Puzzles (ACCP) dataset was developed, comprising an extensive collection of character images from seven key historical stages, annotated with detailed radical sequences. The experiments have showcased considerable promising insights, underscoring the potential and effectiveness of our approach in deciphering the intricacies of ancient Chinese scripts. Through this novel dataset and methodology, we aim to bridge the gap between traditional philology and modern document analysis techniques, offering new insights into the rich history of Chinese linguistic heritage.

Overview of "Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction"

The paper "Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction" addresses a significant challenge in paleography, namely the decipherment of Oracle Bone Inscriptions (OBI), one of the oldest writing systems in Chinese history. The proposed methodology, Puzzle Pieces Picker (P$^3$), innovatively applies the principles of radical reconstruction to interpret these undeciphered ancient scripts. This paper not only introduces a novel approach but also contributes a comprehensive dataset, Ancient Chinese Character Puzzles (ACCP), which spans over 3,000 years of Chinese script evolution.

The approach deconstructs Oracle Bone Inscriptions into fundamental components—strokes and radicals—and employs a Transformer-based model for the reconstruction of these components into modern script counterparts. The paper delineates the process through which their method offers potential solutions for deciphering ancient scripts. The methodology bridges traditional philology with modern document analysis, leveraging artificial intelligence to reveal insights into the historical context of Chinese linguistic heritage.

Data and Methods

The cornerstone of their research is the ACCP dataset, a large-scale collection containing nearly 90,000 character categories and over 340,000 images from seven key historical stages of Chinese script evolution. This dataset serves as critical training data for their AI models. Key to their approach is the segmentation process that involves both contour-based and SAM-based methods to yield radical decomposition of ancient scripts, drawing on MoCo's representation learning to cluster and annotate radicals efficiently.

The paper's method for radical reconstruction utilizes a Transformer-based sequence prediction model. This framework not only captures the evolutionary narrative of Chinese characters but also extends its applicability to other radical-based script systems, such as those in the Japanese and Korean languages. The authors undertake extensive evaluations to validate the model's efficacy, as demonstrated by its accuracy across various periods including modern Regular characters and ancient scripts like Clerical and Seal scripts.

Experimental Findings

Through a series of decipherment experiments, the authors quantify their model's accuracy, highlighting the disparities between different time periods. The results are nuanced, indicating high accuracy when adequate data and closer proximities to modern script conventions are present, such as in the Kangxi dictionary period with nearly 96.4% accuracy. Conversely, the deeper antiquity of OBI presents more significant challenges, yielding lower accuracy figures due to greater variability and reduced standardization in script forms over time.

Their ablation studies further illustrate the significance of incorporating cross-era data, revealing patterns in script evolution that enhance model performance. Specifically, the inclusion of period-specific training data markedly improves the model's capacity to decipher ancient characters, a crucial aspect when facing periods with fewer standardizations like the OBIs.

Implications and Future Directions

The implications of this research are profound for both practical applications and theoretical advancements in the domain of textual analysis and artificial intelligence. The P$^3$ model holds promise not only for deciphering ancient Chinese scripts but also for applications across multiple domains requiring radical-based text analyses. Furthermore, the methodology lays the groundwork for integrating AI into the paper of historical languages, providing a computational lens through which to engage with antiquity.

Looking forward, the continuing enhancement of datasets like ACCP, together with advances in machine learning models, can potentially expand the capabilities of AI in historical linguistics. Future work may explore refining the P$^3$ model's capabilities, adapting it to real-time applications in archaeology, and extending its use to other ancient languages. The approach shows potential for transformative developments in deciphering and interpreting ancient texts, potentially illuminating new insights into our collective history.

In conclusion, this paper presents a compelling and methodologically sound approach to an enduring challenge in the decipherment of ancient scripts, illustrating the rich intersections of AI, history, and linguistics.