Radical-Pictographic Dual Matching in OCR

• Radical-Pictographic Dual Matching is a design pattern that fuses radical (medium-level semantic-structural) and pictographic (stroke/glyph-level) cues for Chinese character interpretation.
• It employs dual encoder–decoder networks and a two-stage matching strategy to reconcile ambiguous stroke sequences with radical-based semantic filtering.
• The approach is applied in zero-shot recognition and OBS decipherment, achieving superior accuracy and interpretability through complementary modality fusion.

Radical–Pictographic Dual Matching denotes, in current arXiv usage, a family of matching schemes that combine radical information with pictographic information for open-set character understanding. In zero-shot Chinese character recognition, the duality is instantiated as radical-level decomposition plus stroke-level decomposition, where strokes provide fine-grained glyph-level cues and radicals provide medium-level semantic-structural cues; the resulting system, STAR, couples dual encoder–decoder training with a two-stage matcher (Zeng et al., 2022). In Oracle Bone Script (OBS) decipherment, Radical-Pictographic Dual Matching (RPDM) is an explicit retrieval-and-rerank mechanism that uses radical analysis and pictographic-semantic analysis to retrieve modern Chinese character candidates from a large dictionary on top of Qwen2.5-VL-7B (Peng et al., 13 Aug 2025). In a separate descriptive exposition of an RSK-type correspondence, the same phrase is used for a bijection from pictures to pairs of Littlewood–Richardson crystals, giving the term a formally distinct combinatorial usage (Nakashima et al., 2010).

1. Problem regimes and terminological scope

In the recognition setting, the central problem is zero-shot Chinese character recognition: recognizing characters whose class labels, or some of their radicals, are not present in the training data. The operative assumption is that characters share reusable component-level decompositions. Stroke-level decomposition represents coarse stroke categories; radical-level decomposition represents radicals and spatial structures following ideographic description sequences. The two decompositions are treated as complementary because strokes supply granular visual evidence while radicals encode medium-level structural and semantic information (Zeng et al., 2022).

In the decipherment setting, the task is to predict a modern Chinese character, or semantic label, from an OBS glyph image by matching two complementary signals. The first signal is radical information, formalized through radical labels and radical analysis text. The second is pictographic semantics, formalized through analysis text that describes how the glyph’s overall shape relates to meaning. The candidate dictionary is explicitly multimodal, with entries of the form $$D=\{(r_i,a_{rad,i},a_{pic,i},a_{joint,i},y_i)\}_{i=1}^N$$, where $r_i$ is a radical label, $a_{rad,i}$ is radical analysis text, $a_{pic,i}$ is pictographic analysis text, $a_{joint,i}$ is radical-informed pictographic analysis text, and $y_i$ is the modern character identity (Peng et al., 13 Aug 2025).

Domain	Input	Matching basis
Zero-shot Chinese character recognition	Character image $x$	Stroke prototypes and radical prototypes
OBS decipherment	OBS glyph image $G$	Radical labels plus analysis texts in a dictionary

A common source of confusion is that “pictographic” does not denote the same representational object in the two applied settings. In STAR, pictographic information is effectively instantiated as stroke-level decomposition; in OBS decipherment, it is instantiated as pictographic-semantic analysis text. This suggests that the phrase names a design pattern—pairing structural decomposition with glyph-level evidence—rather than a single fixed algorithm.

2. Representational primitives

STAR formalizes the image space as $X$ and the set of commonly used Chinese character classes as $C$. In the datasets considered, $r_i$0 ranges from approximately $r_i$1 for preprocessed CTW to $r_i$2 for HWDB/ICDAR and Printed Artistic. Chinese characters are described through $r_i$3 basic strokes grouped into five major categories per GB18030-2005 and prior work, giving the stroke vocabulary $r_i$4 corresponding to horizontal, vertical, left-falling, right-falling, and turning. A stroke encoding is a variable-length sequence $r_i$5 with $r_i$6. Radical encoding uses a vocabulary $r_i$7 formed from $r_i$8 radicals and $r_i$9 spatial structures per GB13000.1 and CJKVI-IDS, with each character represented by an IDS-style sequence $a_{rad,i}$0, $a_{rad,i}$1 (Zeng et al., 2022).

STAR further defines probabilistic attribute encodings. Stroke-level attributes are $a_{rad,i}$2 with $a_{rad,i}$3, and radical-level attributes are $a_{rad,i}$4 with $a_{rad,i}$5. The decoder time dimensions $a_{rad,i}$6 and $a_{rad,i}$7 accommodate the variable-length sequence structure. These definitions make zero-shot recognition possible because unseen characters can still be represented through known component prototypes $a_{rad,i}$8 and $a_{rad,i}$9 without requiring seen-class supervision for the full character label (Zeng et al., 2022).

In OBS decipherment, the representational primitives are embeddings derived from images and texts. The notation distinguishes a radical feature embedding $a_{pic,i}$0, a pictographic-semantic embedding $a_{pic,i}$1, and a textual embedding $a_{pic,i}$2 of a candidate character obtained from analysis text. Text similarity is denoted by $a_{pic,i}$3 and instantiated as BERT-Score in the implementation, while a generic continuous similarity is denoted by $a_{pic,i}$4. Candidate ranking operates over an index set $a_{pic,i}$5 using radical-based scores $a_{pic,i}$6, pictographic-semantic scores $a_{pic,i}$7, and fused scores $a_{pic,i}$8 (Peng et al., 13 Aug 2025).

The representational contrast between the two systems is technically significant. STAR decomposes characters into discrete symbolic sequences that are predicted directly from images. RPDM for OBS instead turns both the query and the dictionary into analysis texts and then performs retrieval over those texts. A plausible implication is that STAR emphasizes compositional visual regularities, whereas the OBS system emphasizes interpretable glyph-to-semantics reasoning.

3. Architectures and training objectives

STAR uses two encoder–decoder pipelines. The stroke pipeline consists of an encoder $a_{pic,i}$9 and a decoder $a_{joint,i}$0; the radical pipeline consists of an encoder $a_{joint,i}$1 and a decoder $a_{joint,i}$2. Both encoders use two convolutional layers, one max-pooling layer, and $a_{joint,i}$3 residual blocks, with the same backbone as SLD of Chen et al., IJCAI’21, and both decoders are original Transformer decoders. The outputs are

$a_{joint,i}$4

The encoders share no weights, but they are regularized to align through a similarity loss (Zeng et al., 2022).

The STAR training objective is the sum of a stroke sequence loss, a radical sequence loss, and a cosine-based alignment loss:

$a_{joint,i}$5

$a_{joint,i}$6

$a_{joint,i}$7

$a_{joint,i}$8

In STAR, $a_{joint,i}$9 and $y_i$0, with ablation selecting $y_i$1 as best (Zeng et al., 2022).

The OBS system is built on Qwen2.5-VL-7B. Its backbone includes a vision encoder adapted to OBS by a spatial patch merger, the Qwen2.5-VL-7B LLM, and standard multimodal cross-attention. It adds a radical recognition head, a radical analysis head implemented as a LoRA module $y_i$2, a pictographic analysis head implemented as a LoRA module $y_i$3, and a mutual analysis component in which $y_i$4 consumes radical analysis to produce radical-informed pictographic analysis text (Peng et al., 13 Aug 2025).

Training proceeds in three stages. Stage 1 performs radical recognition and analysis with

$y_i$5

$y_i$6

$y_i$7

Stage 2 trains pictographic analysis with standard next-token cross-entropy

$y_i$8

and Stage 3 trains mutual analysis with

$y_i$9

The reported results use cross-entropy training for mutual analysis; a contrastive alignment objective is given only as an optional formalization and is explicitly not used in reported results (Peng et al., 13 Aug 2025).

Both systems therefore separate radical and pictographic channels at training time, but they operationalize the duality differently. STAR aligns visual feature spaces for the same character. The OBS system progressively teaches a large vision-LLM to move from radical recognition to pictographic analysis and then to radical-informed semantic reasoning. This suggests two distinct notions of duality: feature correlation in STAR and interpretable multimodal reasoning in RPDM.

4. Matching procedures and inference logic

STAR inference is a two-stage process composed of the Stroke Screening Module (SSM) and the Feature Matching Module (FMM). Given an input $x$0, SSM computes $x$1 and $x$2, decodes a discrete stroke sequence $x$3 by argmax decoding or beam search, and checks a predefined stroke encoding dictionary $x$4. If $x$5 and maps to a unique character $x$6, the system outputs that character directly. Ambiguity arises when $x$7 is not in $x$8 or when multiple characters share the same stroke encoding. STAR does not require top-$x$9 or confidence thresholds at this stage (Zeng et al., 2022).

FMM resolves ambiguous or inactive cases by stroke rectification and cosine matching. Using Levenshtein distance $G$0, it defines the nearest-neighbor set

$G$1

The rectified stroke set is

$G$2

and the candidate set is

$G$3

STAR then ranks candidates by

$G$4

with decision rule $G$5. An optional dual fusion is formalized as

$G$6

but the reported STAR inference uses $G$7, so matching is stroke-only even though training is dual (Zeng et al., 2022).

RPDM for OBS uses a different fusion mechanism. Radical-based matching is formalized by a hard filter

$G$8

and a soft extension

$G$9

though the implementation uses the hard filter and does not learn $X$0. Pictographic-semantic matching is

$X$1

and a general fusion rule is

$X$2

or alternatively a gate-modulated version. In the reported implementation, however, fusion is realized by two-channel retrieval rather than learned $X$3 (Peng et al., 13 Aug 2025).

The implemented RPDM pipeline first predicts $X$4 and generates $X$5, $X$6, and $X$7. Channel 1 restricts candidates to those with $X$8 and ranks them by BERT-Score $X$9. Channel 2 ranks the full dictionary by $C$0. The two top-$C$1 sets are merged and reranked to produce the final Top-$C$2 list. With dictionary size $C$3 and average text length $C$4, the matching cost is approximately $C$5, where $C$6 is the size of the radical-filtered subset and $C$7 is typically $C$8 (Peng et al., 13 Aug 2025).

A recurrent misconception is that dual matching always means simultaneous explicit use of both channels at final scoring. The reported systems do not support that generalization. In STAR, the radical branch mainly improves inference indirectly through training-time alignment, while the published inference rule is stroke-only. In OBS RPDM, duality is explicit at inference, but it is implemented as hard radical filtering plus two-channel text retrieval rather than a learned end-to-end fusion network.

5. Datasets, implementation particulars, and reported performance

STAR is evaluated on handwritten, printed artistic, and street-view data. The handwritten corpus combines CASIA-HWDB 1.0–1.1 with $C$9 images, $r_i$00 classes, and $r_i$01 writers, together with ICDAR2013 with $r_i$02 images, $r_i$03 classes, and $r_i$04 writers. The Printed Artistic dataset contains $r_i$05 samples, $r_i$06 artistic fonts, and $r_i$07 classes. CTW is preprocessed into $r_i$08 single-character images and $r_i$09 classes after SRGAN resolution enhancement and Laplacian filtering. Training uses Adadelta with learning rate $r_i$10, batch size $r_i$11, image resizing to $r_i$12, normalization to $r_i$13, and $r_i$14. The evaluation metric is recognition accuracy. In character zero-shot recognition, STAR reports handwritten accuracies of $r_i$15, $r_i$16, $r_i$17, $r_i$18, and $r_i$19 for $r_i$20, outperforming best SOTA values of $r_i$21, $r_i$22, $r_i$23, $r_i$24, and $r_i$25. On Printed Artistic, STAR reports $r_i$26, $r_i$27, $r_i$28, $r_i$29, and $r_i$30 versus best SOTA $r_i$31, $r_i$32, $r_i$33, $r_i$34, and $r_i$35. On CTW, STAR reports $r_i$36, $r_i$37, $r_i$38, $r_i$39, and $r_i$40 versus SLD values $r_i$41, $r_i$42, $r_i$43, $r_i$44, and $r_i$45. In radical zero-shot recognition, STAR also surpasses SLD across handwritten, Printed Artistic, and CTW, and on seen handwritten ICDAR2013 it reports $r_i$46, competitive with approximately $r_i$47 character-based SOTA and above DenseRAN $r_i$48, HDE $r_i$49, and SLD $r_i$50 (Zeng et al., 2022).

The OBS system introduces the Pictographic Decipherment OBS Dataset (PD-OBS), comprising $r_i$51 Chinese characters, including $r_i$52 characters with OBS images from HUST-OBC and EV-OBC, $r_i$53 characters with ancient Clerical Script images, and modern regular script images for all characters. Each character is annotated with radical analysis text and pictographic analysis text, and joint analyses are derived for mutual analysis. Annotation uses Shuowen Jiezi via Han Dian, GPT-4.1-based enrichment and self-checking, and manual review. Training uses Qwen2.5-VL-7B pretrained weights on $r_i$54 NVIDIA RTX 4090 GPUs. Stage 1 uses learning rate $r_i$55, batch size $r_i$56, epochs $r_i$57, and AdamW; Stage 2 uses learning rate $r_i$58, batch size $r_i$59, and $r_i$60 steps. The LoRA modules $r_i$61 and $r_i$62 have rank $r_i$63 and $r_i$64, with dropout $r_i$65 and $r_i$66 respectively. On validation, the method reports Top-1/Top-10 accuracies of $r_i$67 on HUST-OBC and $r_i$68 on EV-OBC, with Top-10 slightly above PyGT at $r_i$69 and $r_i$70. In zero-shot settings with $r_i$71 character classes held out, it reports $r_i$72 on HUST-OBC and $r_i$73 on EV-OBC, while OBSD reports $r_i$74 and $r_i$75. Generated-analysis quality measured by BERT-Score is also higher than GPT-4.1 on both validation and zero-shot splits (Peng et al., 13 Aug 2025).

These reported results indicate different operational strengths. STAR shows broad gains across handwritten, artistic, and street-view Chinese character recognition, including radical-zero-shot settings. RPDM shows especially strong Top-10 behavior in zero-shot OBS decipherment, where retrieval quality and interpretable evidence are central.

6. Interpretability, ablations, and failure modes

STAR attributes its gains to complementarity between modalities. Stroke-level information is highly granular and robust when radicals are unseen or imbalanced, but it can be ambiguous when distinct characters share similar stroke sequences. Radical-level information captures semantic and spatial composition, but it is affected by radical imbalance and radical-zero-shot issues. The alignment loss correlates $r_i$76 and $r_i$77 for the same character, and the stroke rectification scheme reduces misdiagnosis by expanding to all nearest neighbors in the stroke dictionary rather than only the first nearest prototype. Ablation on Printed Artistic character zero-shot shows that adding the radical branch improves performance over stroke-only training, for example from $r_i$78 to $r_i$79 under Infer_1st and from $r_i$80 to $r_i$81 under Infer_all at $r_i$82; using all nearest rectified encodings also improves over only the first nearest encoding, for example from $r_i$83 to $r_i$84 at $r_i$85; and $r_i$86 is best across all $r_i$87. Reported limitations include degraded performance on complex backgrounds such as CTW and persistent difficulty when characters share identical stroke encodings and extremely similar radical compositions (Zeng et al., 2022).

In the OBS system, interpretability is explicit. The model emits radical analysis, pictographic analysis, and mutual analysis texts that explain how visual features support the predicted semantics, and RPDM retrieves dictionary entries whose analyses best match those generated texts. Case descriptions include radical-origin explanations such as “radical ‘木’ indicates tree/wood category” and pictographic mappings such as “the glyph resembles antlers, indicating deer.” Ablation shows a progressive improvement path from baseline Qwen2.5-VL-7B at $r_i$88 on validation and $r_i$89 in zero-shot, to $r_i$90 at $r_i$91 and $r_i$92, to $r_i$93Radical-Pictographic Mutual Analysis at $r_i$94 and $r_i$95, to $r_i$96Radical Recognition at $r_i$97 and $r_i$98, and finally to $r_i$99RPDM at $a_{rad,i}$00 and $a_{rad,i}$01. Top-$a_{rad,i}$02 sensitivity shows zero-shot Top-10, Top-50, and Top-100 gains on both HUST-OBC and EV-OBC. Reported limitations include rare radicals, highly abstract or noisy inscriptions, domain shift across historical stages, and a tendency under LoRA-only supervised fine-tuning to shortcut reasoning by memorizing visually similar known characters, such as three “口” versus four “口” configurations. Proposed future work includes reinforcement learning with task-specific rewards, improved radical ontologies, integration of phonetic components, and cross-script transfer (Peng et al., 13 Aug 2025).

Taken together, the two applied literatures support a precise but nontrivial interpretation of dual matching. It is not merely late fusion. In STAR it is dual learning plus dual-stage matching, with the radical channel largely acting through representation shaping. In RPDM it is dual retrieval over interpretable analyses, with the radical channel acting as a semantic gate and the pictographic channel acting as a semantic reranker.

7. Descriptive combinatorial usage

In a descriptive exposition of Nakashima–Shimojo’s result, “Radical-Pictographic Dual Matching” is used as a name for the fundamental bijection between the set of pictures and the set of pairs of Littlewood–Richardson crystals. Here a picture is a bijection $a_{rad,i}$03 between skew diagrams that is PJ-standard together with its inverse, where PJ-standardness means preservation from the partial order $a_{rad,i}$04 to the total order $a_{rad,i}$05. The main theorem states that there are well-defined bijections

$a_{rad,i}$06

with inverse $a_{rad,i}$07, where the output is a same-shaped pair of LR-crystal elements (Nakashima et al., 2010).

The construction factors through a chain

$a_{rad,i}$08

The forward map builds a skew tableau from the picture, converts it via middle-eastern reading into a lexicographic two-rowed array, and then applies classical column RSK bumping to obtain insertion and recording tableaux $a_{rad,i}$09 and $a_{rad,i}$10. The inverse performs reverse bumping, reconstructs the skew tableau, and then reconstructs the picture by the coordinate rule

$a_{rad,i}$11

This usage is formally unrelated to OCR or decipherment, but it preserves the phrase’s structural emphasis on a picture-like object, a core matching mechanism, and a dual output (Nakashima et al., 2010).

Within this combinatorial setting, the descriptive term has a different semantics from the applied vision literature. “Pictographic” refers to the picture input, “radical” refers to reduction to the core bumping-and-recording mechanism, and “dual” refers to the symmetric pair of LR crystals. The juxtaposition is terminological rather than methodological, but it highlights that the phrase has acquired a broader descriptive utility across otherwise disconnected domains.