Sejong and Penn Korean Treebanks

• The paper demonstrates that BN-FLAT, NULL-DROP, and EOJ-ALIGN normalization methods yield isomorphic constituency structures over eojeol terminals.
• Sejong and Penn Korean Treebanks are foundational resources that capture Korean morphosyntax by treating eojeol as the atomic terminal while separating syntax from morphology.
• Empirical studies show that cross-treebank parsing incurs minimal F1 score drops, enabling robust model transfer and unified syntactic analysis across diverse annotations.

The Sejong and Penn Korean Treebanks are foundational resources for syntactic annotation in Korean, designed to capture phrase structure and morphological information amid the complexities of Korean morphosyntax. Both treebanks treat the orthographic word (eojeol)—a sequence of lexical and functional morphemes constituting a single spacing unit—as the fundamental terminal unit in syntactic trees. The divergent syntactic annotation strategies in Sejong and Penn Korean, concerning binarity, null element representation, and POS granularity, raise critical representational questions. Recent analysis demonstrates that, under specific normalization procedures, the two resources are isomorphic at the eojeol-based constituency level and can be unified into a common annotation scheme that sharply separates syntactic constituency from morphological detail (Park et al., 27 Dec 2025).

1. Linguistic Background and Eojeol as Terminal

Korean presents a unique challenge for syntactic annotation, as the basic scriptural unit, eojeol, aggregates multiple morphemes (lexical stems and grammatical particles) into a single orthographic token. Formally, if Morphemes denotes the inventory of Korean morphemes, the set of eojeol terminals is defined as:

$\text{EOJEOL} \coloneqq \text{Morpheme}^+$

where any non-empty sequence of morphemes $M = \langle m_1, m_2, ..., m_k \rangle$ constitutes an eojeol $e$. The mapping

$f: \text{Morpheme}^+ \rightarrow \text{EOJEOL}$

is such that

$$f(M) = e \iff e \text{ is the orthographic concatenation of } M$$

This design precludes the conflation of word-internal and phrase-level structure, supporting a clear separation between syntactic terminals and sub-word morphological analysis (Park et al., 27 Dec 2025).

2. Annotation Architectures in Sejong and Penn Korean

Both resources encode multi-layered linguistic structures through distinct methodological conventions:

• Sejong Treebank: Implements strict binary branching (Chomsky Normal Form), does not introduce empty categories, and assigns Sejong XPOS tags at terminal level. Internal morpheme structure is not represented in the constituent tree; surface eojeols serve as atomic terminals.
• Penn Korean Treebank: Allows flat phrase structures particularly in adjunct positions, employs composite XPOS tags (reflecting morpheme sequences), and systematically introduces null elements (e.g., pro, trace) to capture ellipsis or movement.

To encode detailed morphology and part-of-speech, both treebanks maintain a parallel morphological layer, defined for each eojeol $e \in \text{EOJEOL}$ as:

$$\sigma(e) = \langle (m_1, x_1), (m_2, x_2), ..., (m_k, x_k) \rangle$$

where $m_i$ are morphemes and $x_i$ elements drawn from either Sejong's XPOS or Penn’s composite XPOS inventories. Universal POS (UPOS) tags (Petrov et al. 2012, e.g., NOUN, VERB) are projected above each eojeol to unify syntactic categorization (Park et al., 27 Dec 2025).

3. Normalization and Representational Equivalence

A small set of normalization operations enables representational equivalence between Sejong and Penn Korean at the constituency level:

• BN-FLAT (Debinarization): Removes artificially introduced binary nodes in Sejong, flattening constituents to reflect only linguistically motivated dominance relations.
• NULL-DROP: Removes Penn’s null elements (empty categories), reconstructing trees with only overt eojeol terminals.
• EOJ-ALIGN: Aligns surface tokens to ensure that every terminal corresponds exactly to an eojeol, removing any inconsistent sub-token divisions.

Under these transformations, Sejong and Penn trees become isomorphic at the eojeol-based constituency level, differing only in label inventory and tagset conventions. This structural isomorphism is formalized as:

Proposition 1: Under BN-FLAT, NULL-DROP, and EOJ-ALIGN, the Sejong treebank and the Penn Korean treebank yield isomorphic constituency structures over eojeol terminals (up to label naming conventions) (Park et al., 27 Dec 2025).

The normalization process is illustrated in constituent examples, with post-normalization trees showing consistent segmentation, absence of null elements, and syntactic labels projected over syntactically atomic eojeol units.

4. Unified Eojeol-Based Scheme: Structure and Formalism

The unified annotation scheme derived from this equivalence separates the syntax and morphology of Korean. It consists of:

• Surface token sequence: $e_1, e_2, ..., e_n \in \text{EOJEOL}$.
• Morphological layer: $M = \langle m_1, m_2, ..., m_k \rangle$0 for each eojeol, listing $M = \langle m_1, m_2, ..., m_k \rangle$1 morpheme-tag pairs.
• UPOS preterminals: $M = \langle m_1, m_2, ..., m_k \rangle$2 for each terminal.
• Phrase structure tree: Nonterminal nodes from a typology $M = \langle m_1, m_2, ..., m_k \rangle$3 such as $M = \langle m_1, m_2, ..., m_k \rangle$4S, NP, VP, NP-SBJ, ADJ, NML, ...$M = \langle m_1, m_2, ..., m_k \rangle$5.

POS grammars are strictly defined as:

• $M = \langle m_1, m_2, ..., m_k \rangle$6NOUN, VERB, ADJ, ADV, PROPN, DET, PRON, ADP, AUX, PUNCT, NUM, SYM, X$M = \langle m_1, m_2, ..., m_k \rangle$7
• $M = \langle m_1, m_2, ..., m_k \rangle$8: the standard Sejong tagset (NNP, NNG, JKS, etc.)

Morphological features such as Case, Tense, Number, Polarity are encoded in an attribute-value structure, not as part of the constituency backbone.

A constituency-to-dependency conversion function is specified using head-percolation principles. The procedure relies on UPOS tags and operates solely over eojeol terminals, supporting robust transformation of unified, Sejong, or Penn-normalized trees (Park et al., 27 Dec 2025).

5. Comparative Analysis and Case Studies

Key original differences and convergences post-normalization are summarized in the following table:

Dimension	Sejong	Penn Korean
Terminal granularity	Eojeol (opaque)	Eojeol, with allomorphy / null elements
POS tag inventories	Sejong XPOS	Penn compound XPOS
Branching	Strictly binary	Flat allowed in adjuncts
Null elements	Absent	Present (pro, trace)

After applying BN-FLAT and NULL-DROP, post-normalization, both treebanks yield flat structures with identical surface tokenization and isomorphic constituency trees over eojeol terminals. Empirical case studies support this equivalence:

• SPMRL 2013 Shared Task: Conversion of the KAIST treebank to an eojeol-based flat format allowed constituency models trained on Sejong data to transfer to KAIST outputs with negligible loss in head attachment.
• UD for Korean v2: Aligns eojeol-based Universal Dependencies directly with normalized constituency structures, enhancing cross-formalism compatibility.
• Cross-treebank parsing (Kim & Park 2022): Parsers trained on normalized Sejong transfer to normalized Penn Korean data with a small ($M = \langle m_1, m_2, ..., m_k \rangle$9 pp) drop in F$e$0, empirically verifying representational equivalence (Park et al., 27 Dec 2025).

6. Implications and Significance

Explicit separation of syntax and morphology via an eojeol-based terminal architecture clarifies linguistic representation for Korean, supports consistent constituency and dependency annotation, and grounds empirical resource interoperability. The representational unification under BN-FLAT, NULL-DROP, and EOJ-ALIGN demonstrates that ad hoc morpheme merging is unnecessary for cross-treebank applications.

A plausible implication is that future resource development and annotation efforts can build directly on this eojeol-based schema, leveraging isomorphic representations for model transfer, conversion, and linguistic analysis across existing and forthcoming Korean treebanks.

7. Conclusion

The Sejong and Penn Korean Treebanks, once normalized to a shared eojeol-based constituency scheme with explicit morphology outside the tree, provide a linguistically stable and interoperable substrate for Korean syntactic research. This architecture underpins robust cross-treebank comparison, facilitates constituency-to-dependency conversion, and enables empirical transfer without information loss, establishing a robust and extensible annotation platform for morphologically rich languages like Korean (Park et al., 27 Dec 2025).