Oracle-MNIST: Ancient Oracle Benchmark

• Oracle-MNIST is a curated dataset of 28×28 grayscale images of ancient Chinese oracle characters designed for robust machine learning evaluation.
• It adopts the MNIST IDX format, ensuring immediate compatibility while introducing severe real-world degradation and high intra-class variance.
• Baseline comparisons reveal significantly lower classification performance than MNIST, highlighting the dataset's challenging recognition tasks.

Oracle-MNIST is a curated benchmark dataset comprising 28×28 grayscale images of ancient Chinese oracle characters, specifically designed for machine learning algorithm evaluation with an emphasis on robust pattern classification under challenging real-world noise and style variabilities. Developed and released by Wang et al., it follows the exact IDX file format and labeling conventions of the canonical MNIST benchmark, thus enabling direct drop-in compatibility with MNIST-based pipelines. However, Oracle-MNIST is distinguished by the significant real-world degradation, high intra-class variance, and elevated inter-class similarities characteristic of three-thousand-year-old inscriptions, providing a more demanding testbed than MNIST for classification and robustness benchmarking (Wang et al., 2022).

1. Dataset Structure and Format

Oracle-MNIST consists of 30,222 grayscale images of ancient Chinese hieroglyphs categorized into 10 semantic classes. The dataset is partitioned into a training set and a test set as follows:

Split	Number of Images	Per-Class Counts (train)	Number of Classes
Training	27,222	2,328 – 3,399	10
Test	3,000 (300/class)	—	10

Classes correspond to prototypical oracle-bone glyphs: "big," "sun," "moon," "cattle," "next," "field," "not," "arrow," "time," and "wood." Image dimensions are 28×28 pixels, captured in 8-bit grayscale, and the files are compressed in IDX (ubyte) format, mirroring MNIST:

• train-images-idx3-ubyte.gz (27,222 images)
• train-labels-idx1-ubyte.gz (27,222 labels)
• t10k-images-idx3-ubyte.gz (3,000 images)
• t10k-labels-idx1-ubyte.gz (3,000 labels)

2. Data Characteristics and Recognition Challenges

Oracle-MNIST images originate from "rubbing" scans of oracle-bone surfaces, subject to both physical and stylistic sources of variability:

• Real-world noise: Severe abrasion, broken outlines, missing or uneven ink deposition, and sensor or thresholding artifacts create high-frequency and structured distortions, a consequence of millennia-long burial and aging.
• Stylistic variability: Each character exhibits substantial intra-class variance due to diverse ancient writing practices and stroke orders, alongside small inter-class differences; for instance, "wood" and "cattle" differ primarily by subtle stroke presentations.
• Mathematical abstraction: No explicit parametric model is given, but the observed image $I_{\text{obs}}(x,y)$ can be conceptualized as:

$$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$

where $I_{\text{true}}$ is the ideal glyph, $k$ is a blur kernel, $N_{\text{age}}$ encodes historical degradation, and $N_{\text{scan}}$ represents digitization noise.

These conditions collectively create a more demanding classification scenario than MNIST, requiring algorithms to handle both heavy real-world degradation and complex visual similarities.

3. Preprocessing Workflow and MNIST Compatibility

The construction pipeline is carefully designed to ensure identical usage semantics with MNIST while preserving authentic noise:

1. Scanned images are converted to 8-bit grayscale intensity, $I(x,y)$.
2. If images appear as dark ink on a light background, the intensities are optionally negated to match MNIST polarity, $I'(x,y) \leftarrow 255 - I(x,y)$.
3. The long edge is bicubically rescaled to 28 pixels: $s = 28/\max(H, W)$, $I_1 = \text{bicubic}(I', s)$.
4. The image is then center-padded along the shorter edge to 28×28 on a zero (black) background.

Attempts at applying histogram equalization or gray-stretch techniques were found to cause marginal decreases in test accuracy; as a result, all images are released in raw, unenhanced form.

The dataset preserves original MNIST IDX file naming, allowing unmodified use of existing deep learning frameworks such as PyTorch and TensorFlow—e.g., torchvision.datasets.MNIST and tf.keras.datasets.mnist—with the sole modification being download path redirection (Wang et al., 2022).

4. Benchmarking Tasks and Baseline Results

The standard classification task is a 10-way recognition of ancient character images. Comparative baselines from the original release demonstrate the increased difficulty of Oracle-MNIST relative to MNIST and Fashion-MNIST:

Classifier	Oracle-MNIST	Fashion-MNIST	MNIST
CNN (2×Conv→Pool→ReLU, 2×FC, Drop)	93.8%	92.1%	99.3%
CNN (no dropout)	92.8%	—	—
SVC (RBF, $$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$0)	75.5%	89.7%	97.3%
RandomForest (100 trees)	64.9%	87.1%	97.1%
GradientBoost (100 iters, depth=10)	72.5%	—	—
kNN ($$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$1)	62.7%	—	—
LogisticRegression ($$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$2)	59.8%	—	—
LinearSVC	58.1%	—	—
SGDClassifier	56.7%	—	—
MLPClassifier (ReLU, [100])	74.7%	—	—

Shallow classifiers such as SVC, RandomForest, and MLP degrade substantially more on Oracle-MNIST compared to MNIST, and even convolutional neural networks (CNNs) leave approximately 6.2% error, indicating the effect of real-world degradations and large style variations (Wang et al., 2022).

5. Comparison with MNIST: Difficulty and Motivations

Several structural differences make Oracle-MNIST a notably harder benchmark than MNIST:

• Physical degradation: Oracle-MNIST samples exhibit realistic noise and distortions due to historical aging, abrasion, and ink transfer, as opposed to MNIST’s clean handwritten digits.
• Stylistic diversity: The centuries-spanning diversity of character topology and stroke order results in high intra-class variance not present in MNIST’s uniform digit shapes.
• Inter-class similarity: Discrimination between certain characters (e.g., “wood” vs. “cattle”) requires sensitivity to subtle local stroke differences, exceeding the complexity required for MNIST’s well-separated digit classes.

Research motivation arises from the performance saturation observed on MNIST ($$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$399% CNN accuracy); Oracle-MNIST offers a more challenging, realistic evaluation of classifier robustness while maintaining immediate plug-and-play usability for MNIST-tuned pipelines (Wang et al., 2022). A plausible implication is that Oracle-MNIST is suitable for stress-testing architectures and for studying robustness under non-ideal image conditions.

6. Integration and Practical Usage Recommendations

Oracle-MNIST is available for unrestricted research use at https://github.com/wm-bupt/oracle-mnist. Download and usage patterns are identical to MNIST, as shown in typical shell and framework commands:

• Shell download example:

$$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$4

• PyTorch workflow example:

$$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$5

• TensorFlow/Keras workflow example:

$$I_{\text{obs}}(x,y) = (I_{\text{true}} \star k)(x,y) + N_{\text{age}}(x,y) + N_{\text{scan}}(x,y)$$6

Normalization (mean=0.1307, std=0.3081) and data augmentation strategies (random shift, rotation, elastic distortions) from MNIST are recommended to improve generalization. This suggests that reusing MNIST-optimized pipelines yields a fast baseline, while customized augmentation may further mitigate Oracle-MNIST’s noise and class imbalance.

By adhering strictly to the 28×28 IDX format, Oracle-MNIST serves as a direct MNIST replacement, facilitating immediate benchmarking of classical and state-of-the-art algorithms on a dataset that foregrounds authentic noise, style variability, and fine-grained classification challenges (Wang et al., 2022).