InteChar: Unified Encoding for Oracle Bones
- InteChar is a unified and extensible character set that integrates oracle bone, traditional, and simplified Chinese characters to support historical language modeling.
- It standardizes previously unencoded oracle glyphs by merging Unicode CJK data, historical font resources, and newly constructed characters through a semi-automatic pipeline.
- Paired with the OracleCS corpus, InteChar significantly improves transformer-based model performance by providing complete coverage of rare and ambiguous ancient scripts.
Searching arXiv for the exact topic and closely related historical-NLP work.
Search query: ti:"InteChar" OR abs:"InteChar" OR "oracle bone" ancient Chinese language modeling
InteChar, short for “Integrated Characters,” is a unified and structured character set for ancient Chinese NLP that includes oracle bone characters, traditional Chinese characters, and modern simplified characters. It was introduced to address a concrete bottleneck in historical language modeling: oracle bone inscriptions constitute an extreme low-resource domain, with only about 5,000 complete oracle bones and roughly 15,000 sentences with more than five characters, while many glyphs are undeciphered, rare, damaged, or absent from standard encodings. By merging official Unicode CJK coverage, machine-readable historical font resources, and newly constructed code points for previously unencoded oracle glyphs, InteChar converts oracle bone material from a partially image-bound source into a consistent text domain and provides the basis for OracleCS, a corpus and benchmark for historical language understanding (Diao et al., 12 Aug 2025).
1. Historical-NLP context and the encoding problem
Oracle Bone Inscriptions are the oldest large body of Chinese writing, dated to the late Shang period, and are central to reconstructing early Chinese language and orthography, dating and attributing archaeological finds, and studying ritual, divination, and political history. From an NLP perspective, the domain is characterized by simultaneous data scarcity, orthographic discontinuity, and encoding instability. Existing pipelines for language modeling assume large corpora, but the oracle-bone record contains only thousands of usable sentences; existing modern Chinese vocabularies omit many historically crucial glyphs; and Unicode does not provide full coverage for oracle bone characters, especially excavated variants, undeciphered or rarely attested characters, and damaged or idiosyncratic forms (Diao et al., 12 Aug 2025).
The resource is therefore motivated by four structural deficiencies. First, standard unsupervised pretraining is inefficient when the corpus contains only thousands of sentences. Second, the script evolved through oracle, bronze, small seal, clerical, and regular forms, so the absence of a unified encoding disrupts diachronic continuity. Third, domain-specific font sets are not standardized in the Unicode sense and are mutually inconsistent. Fourth, prior ancient-Chinese LLMs typically reuse modern Unicode vocabularies, add only a few high-frequency ancient characters, or ignore characters without Unicode code points. In historical texts, however, a single low-frequency character may encode a name, place, title, or ritual term. This suggests that the principal failure mode is not merely corpus size but the inability to represent historically informative characters as stable textual units.
2. Character inventory and construction pipeline
InteChar is defined as a unified and extensible character list built on top of the official Unicode character set. The Unicode CJK repertoire serves as the base layer, with original code points preserved. Onto this base, the system integrates characters from machine-readable ancient Chinese resources, especially the Zhongjian Library historical fonts, and then adds completely new internal code points for oracle glyphs absent from both Unicode and existing historical fonts. The final set contains 11,288 characters after de-duplication and proofreading, and it is delivered as a complete TrueType font with standardized glyph shapes (Diao et al., 12 Aug 2025).
The construction process has three principal sources. The first is the official Unicode CJK set. The second is the Zhongjian Library, whose 16 historical font sets include oracle bone, bronze, and bamboo-manuscript material; only characters that appear both in these resources and in the curated textual corpus are retained, and duplicate representations across fonts are collapsed to a single representative form. The third source consists of newly constructed oracle glyphs for previously unencoded characters, especially low-frequency, unique, undeciphered, or damaged forms.
The new-glyph pipeline is semi-automatic and radical-based. It begins with image collection and preprocessing from archaeological publications and databases, including resizing, contrast normalization, and geometric adjustment. Character images are then processed by an object detection model that predicts bounding boxes and radical categories. Predicted radicals are mapped to modern Chinese equivalents when possible, yielding an intermediate compositional glyph. Paleographers then verify or correct radical identification, placement, and structure. Verified forms are redrawn as scalable vector graphics, assigned new internal code points in a Unicode-style format, and integrated into the master list. After merging Unicode, Zhongjian, and newly constructed glyphs, de-duplication is supported by Siamese neural networks that flag highly similar glyph pairs for expert inspection. Experts then decide whether to merge or preserve them as distinct characters. A plausible implication is that InteChar is best understood not as a mere font supplement but as a curated equivalence structure over heterogeneous character evidence.
3. OracleCS and the textualization of oracle-bone material
InteChar is paired with OracleCS, described as a linguistically curated corpus specifically focused on oracle bone inscriptions and ancient Chinese texts. OracleCS encodes all characters by InteChar IDs, including undeciphered ones, and combines excavated oracle texts with selected pre-Qin classical materials. The corpus contains about 11,288 unique Chinese characters and 173,459 annotated samples. Character-level annotation includes radical decomposition, mapping to a modern Chinese word when decipherment is available, and semantic explanation from classical lexica where possible; for undeciphered characters, semantic fields remain blank, but the characters are still retained as InteChar tokens (Diao et al., 12 Aug 2025).
The source base is hybrid. Excavated oracle texts are derived from oracle bone rubbings and images, including data from the Anyang oracle bone database, and are cleaned and annotated by domain experts. Pre-Qin classics drawn from the ctext.org corpus, including the Analects, Spring and Autumn Annals, Mencius, and Xunzi, provide denser semantic and syntactic context for ancient vocabulary. Lexical resources such as Shuowen Jiezi and Hanyu Da Cidian support radical and semantic annotation.
OracleCS also incorporates LLM-assisted augmentation in an instruction-tuning style. Samples may involve sentence translation from ancient to modern Chinese, synonym substitution, glyph structure analysis, character decomposition, and semantic prediction. Ancient characters in prompts and outputs are aligned to InteChar IDs, and synthetic outputs are filtered and manually inspected where necessary. This design is significant because it uses the encoding layer to make oracle-bone material compatible with mainstream text-only learning regimes without collapsing undeciphered characters into placeholders.
4. Integration with LLMs
InteChar does not introduce a new transformer architecture. Its intervention point is the character inventory and the embedding interface. The evaluation uses existing transformer-based models, including BERT, GPT-2, LLaMA-3-8B, MiniRBT, guwenBERT-base, sikuBERT, Qwen-7B-Chat, GLM-4-9B, XunziALLM, and TongGu-LLM. Experiments are conducted under two settings: “Origin,” which uses the model’s original vocabulary, and “InteChar,” which replaces the input character vocabulary with an InteChar-based embedding layer. In the embedding regime, the pretrained backbone is frozen and only a new embedding matrix and light task-specific components are trained; in the fine-tuning regime, LoRA is used for parameter-efficient adaptation (Diao et al., 12 Aug 2025).
Formally, if the embedding dimension is , the new character embedding matrix is
In the frozen-backbone setting, training uses 10 epochs, batch size 32, learning rate , AdamW, and early stopping based on NDCG@10 on the development set. In the LoRA setting, training uses 10 epochs, batch size 32, learning rate , and AdamW.
This design differentiates InteChar from character-aware language-model architectures such as CharBERT, which augment BERT- or RoBERTa-style backbones with a dual-channel architecture, a BiGRU-based character encoder, a heterogeneous interaction module, and a Noisy Language Modeling objective (Ma et al., 2020). InteChar, by contrast, relocates the technical novelty to the character list, font infrastructure, and corpus. This suggests that, for oracle-bone NLP, encoding completeness may be at least as consequential as architectural modification.
5. Tasks, benchmarks, and empirical results
OracleCS supports both embedding-level and fine-tuning evaluations. The benchmark includes cloze completion on oracle bone inscriptions, commentary-to-text retrieval, ancient Chinese translation, polysemous word matching, and word parsing (Diao et al., 12 Aug 2025).
| Task | Data scale | Metrics |
|---|---|---|
| Cloze completion | 15,416 total; 12,416 train; 3,000 eval | NDCG@10, MRR@10, NDCG@20, MRR@20 |
| Commentary retrieval | 896 queries; 12,141 candidates | NDCG@400, MRR@400, NDCG@500, MRR@500 |
| Ancient Chinese translation | 15,868 train; 10,578 test | Accuracy |
| Polysemous word matching | 33,380 train; 22,253 test | Accuracy |
| Word parsing | 81,929 train; 54,619 test | Accuracy |
The cloze task directly measures contextual discrimination among oracle characters. Reported gains are substantial across model families. For BERT, NDCG@10 improves from 0.167 to 0.515 and MRR@10 from 0.134 to 0.375. For GPT-2, NDCG@10 improves from 0.216 to 0.584 and MRR@10 from 0.168 to 0.534. For Qwen-7B-Chat, NDCG@10 rises from 0.302 to 0.842 and MRR@10 from 0.254 to 0.736. For GLM-4-9B, NDCG@10 rises from 0.274 to 0.808 and MRR@10 from 0.278 to 0.752. Even TongGu-LLM improves from 0.238 to 0.723 on NDCG@10. These results indicate that making oracle characters first-class tokens materially changes ranking quality in contextual prediction.
Commentary-to-text retrieval shows consistent sentence-level gains. GLM-4-9B improves on NDCG@500 from 0.432 to 0.595; Qwen-7B-Chat improves on NDCG@500 from 0.418 to 0.579; and sikuBERT improves on MRR@500 from 0.133 to 0.216. This suggests that the unified vocabulary supports better alignment between modern Chinese commentary language and classical sentence content.
Fine-tuning results also improve across all reported models. guwenBERT-base improves in average score from 90.62 to 91.29, sikuBERT from 90.18 to 90.91, Qwen-7B-Chat from 92.32 to 93.06, GLM-4-9B from 91.66 to 92.16, XunziALLM from 92.61 to 93.27, and TongGu-LLM from 92.21 to 92.92. Translation gains are typically about 0.5–1.0 points, while polysemous matching and word parsing show smaller but consistent improvements. The paper interprets these results as evidence that unified character coverage improves character-level embeddings for rare and ambiguous forms and facilitates transfer across sentence-level and character-level tasks.
6. Limitations, extensibility, and broader significance
InteChar is explicitly designed to be continuously updatable. Newly discovered or reconstructed glyphs can be incorporated by passing them through the radical-recognition pipeline, manually verifying structure and semantics, redrawing and vectorizing the glyph, assigning a new internal code point, and updating the font and character list. The separation between the Unicode base layer, external historical fonts, and InteChar’s own extension zone is intended to preserve compatibility while permitting systematic growth (Diao et al., 12 Aug 2025).
Several limitations remain. Coverage is incomplete because some excavated glyphs are too damaged, too uncertain, or not yet incorporated. For undeciphered characters, semantic fields remain blank, so models must rely on distributional and structural signals rather than gloss-based supervision. OracleCS is biased toward Anyang oracle-bone materials and pre-Qin philosophical and historical texts, leaving other sites, periods, and genres underrepresented. The radical-recognition and vectorization pipeline is not error-free and still requires labor-intensive expert verification. InteChar is Unicode-style but not yet part of Unicode, so it remains a research-oriented encoding rather than a formal standard.
Its broader significance lies in establishing a shared substrate for historical Chinese NLP, digital humanities, and archaeology. For NLP, it provides a stable character vocabulary that allows standard transformer systems to process oracle-bone texts directly, including undeciphered forms. For digital humanities, the TrueType font and unified code space enable search, retrieval, and consistent digital editions. For archaeology and paleography, the resource supports provenance studies, dating, and distributional inference over rare characters. A plausible implication is that InteChar redefines the primary unit of computation in ancient Chinese language modeling: instead of approximating early scripts through later Unicode surrogates or external images, it makes the historically attested glyph itself the operative textual token.