Eojeol-Based Annotation Scheme

• The eojeol-based annotation scheme is a linguistic framework that treats whitespace-delimited eojeols as atomic syntactic terminals, separating phrase-level syntax from internal morphology.
• It employs a two-layer architecture that decouples constituency structure from fine-grained morphological segmentation, promoting modularity and robust treebank conversion.
• It enables cross-resource interoperability and normalization through deterministic approaches for converting between constituency and dependency representations.

An eojeol-based annotation scheme is a linguistic annotation framework where the basic unit of syntactic bracketing and segmentation for Korean is the eojeol—a whitespace-delimited token corresponding to a stem plus its grammatical affixes—rather than minimal morphemes. This approach separates phrase-level syntax from word-internal morphology and provides a framework for representational equivalence and cross-resource conversion in Korean treebanks. Recent developments standardize the use of eojeol as the atomic unit for both constituency and dependency representations, with independent but aligned morphological and syntactic annotation layers (Park et al., 27 Dec 2025, Park et al., 2023, Sung et al., 18 Mar 2025).

1. Eojeol as Syntactic Terminal: Definitions and Distinctions

In Korean orthography, an eojeol is defined as the contiguous sequence of non-space symbols between whitespaces. Formally, let $\Sigma$ be the alphabet of valid surface symbols (Hangul, Latin, numerals, etc.), then an eojeol is any non-empty string:

$$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$

Sentences are sequences $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$ of eojeols. Each eojeol can be morphologically segmented into morphemes: $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$, where $m_j$ are (possibly bound) minimal expression units (stems, suffixes, particles, etc.) (Park et al., 2023, Park et al., 27 Dec 2025).

Traditional Korean treebanks (e.g., Sejong, Penn) have used morphemes as constituency terminals—interleaving word-internal morphology with phrase-structure bracketing. The eojeol-based proposal treats each eojeol as a syntactic atom: terminals are the set $\Sigma_\text{surface}$ of observed eojeols, with finer-grained morphological information encapsulated in a non-tree layer.

This distinction is critical for compatibility: dependency resources (KUD, UD-Korean) already use eojeol tokens, and the eojeol-based constituency guarantees one-to-one terminal correspondence, facilitating conversion and multi-framework interoperability (Park et al., 27 Dec 2025, Sung et al., 18 Mar 2025).

2. Two-Layer Annotation Architecture

The modern eojeol-based scheme employs a strict two-layer annotation:

• Constituency layer $C=(V,N,P,\text{root})$:
  • $V$ = $(v_1,\dots,v_n)$, linear surface eojeol sequence, $v_i \in \Sigma_\text{surface}$
  • $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$0 = set of nonterminal phrase categories (S, NP, VP, etc.)
  • $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$1 = production set: $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$2, where Pre are preterminal UPOS categories
  • Each eojeol $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$3 is dominated by exactly one UPOS preterminal.
• Morphological layer $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$4:
  • For each $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$5, $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$6, with fine-grained XPOS (e.g., Sejong tagset).

The paired annotation for sentence $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$7 is $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$8, with $$e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1$$9 over $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$0 and $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$1 (Park et al., 27 Dec 2025).

This architecture decouples syntax (constituency over eojeol terminals) from detailed morphosyntactic segmentation, promoting modularity. The morphological segmentation and XPOS annotation are equivalent to a parallel table lookup or stand-off annotation, enabling representation-neutral enrichment and downstream task flexibility (Park et al., 2023, Sung et al., 18 Mar 2025).

3. Normalization and Cross-Treebank Equivalence

Explicit normalization operations establish formal equivalence between previously divergent Korean treebanks:

1. Debinarization ($S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$2): Converts Chomsky-normal (binary) branching to n-ary (flat) structures, e.g., collapsing $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$3 chains to $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$4.
2. Null removal and canonicalization ($S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$5): Prunes empty-category leaves (e.g., pro, trace), reattaching siblings to preserve overt surface dominance.
3. Eojeol boundary realignment ($S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$6): Ensures a uniform segmentation by re-tokenizing so that both resources exhibit identical eojeol boundaries.

Sejong trees ($S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$7) are mapped via $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$8, Penn trees ($S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$9) via $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$0. The claim holds:

$$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$1

where $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$2 is the canonical eojeol-based constituency tree for sentence $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$3. This demonstrates that under normalization, these treebanks are representationally equivalent at the eojeol-based constituency level (Park et al., 27 Dec 2025).

4. Annotation Guidelines, Tag Inventories, and Segmentation Conventions

4.1 Tree Conversion Procedure

To convert legacy morpheme-terminal trees to the eojeol scheme:

1. Identify terminal nodes not corresponding to a complete eojeol.
2. Collapse all sibling morpheme terminals from the same eojeol into a single leaf labeled by the eojeol.
3. Prune unary branches due solely to morpheme structure.
4. Debinarize using $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$4.
5. Remove null categories, reattaching overt siblings via $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$5 (Park et al., 27 Dec 2025).

4.2 Morphological Layer Conventions

• UPOS (preterminal inventory): {NOUN, VERB, PROPN, ADJ, ADV, PRON, ADP, SCONJ, CCONJ, PART, NUM, PUNCT, X, ...}
• XPOS (fine-grained, e.g., Sejong tags): NNP, NNG, VCP, VV, EP, EF, EC, ETM, ETN, JKS, JKG, JKO, JKB, SF, ...
• Segmentation notation: $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$6 ("+" marks morpheme boundaries).

Eojeol internal segmentation follows a functional-analytic principle: only functional morphemes (case markers and verbal endings) are split off, while derivational suffixes remain attached to their stems (Park et al., 2023). This is termed Level 4 segmentation and yields optimal statistical and linguistic outcomes, outperforming both coarser and finer schemes (Level 4 $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$7 vs. Level 1 $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$8, Level 5 $$\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$$9 for phrase-structure parsing) (Park et al., 2023).

A recommended six-column TSV format for annotation:

Col 1: ID	Col 2: Surface	Col 3: MorphSeg (XPOS)	Col 4: UPOS	Col 5: Left-brackets	Col 6: Right-brackets

Columns 5–6 encode bracketed syntactic constituency over eojeol terminals (Park et al., 27 Dec 2025).

5. Algorithms and Conversion Between Constituency and Dependency

5.1 Constituency to Dependency (Head-Projection)

A deterministic head-projection procedure computes dependencies from constituency trees over eojeol terminals:

$m_j$2

Head rules reflect Korean-specific syntactic typologies: NP heads are the rightmost noun or pronoun, VP heads are verbs with highest XPOS priority, and PP heads are the postposition-bearing eojeol.

5.2 Dependency to Constituency (Eojeol-Grouping)

A "span-growing" algorithm reconstructs flat, n-ary constituency structure from dependency trees:

$m_j$3 (Park et al., 27 Dec 2025)

This formalism enables one-to-one conversions and supports application in parsers, transfer learning, and cross-framework synthesis.

6. Empirical Examples and Practical Implementation

Empirical evaluation and illustration appear in all major works. For instance, the normalized tree for the sentence “프랑스의 세계적인 의상 디자이너 엠마누엘 웅가로가 실내 장식용 직물 디자이너로 나섰다.” is annotated as follows under Level 4/eojeol-based guidelines:

Eojeol	Morpheme Segmentation (POS)
프랑스의	프랑스/NNP + 의/JKG
세계적인	세계/NNG+적/XSN+이/VCP + ㄴ/ETM
의상	의상/NNG
디자이너	디자이너/NNG
엠마누엘	엠마누엘/NNP
웅가로가	웅가로/NNP + 가/JKS
실내	실내/NNG
장식용	장식용/NNG
직물	직물/NNG
디자이너로	디자이너/NNG + 로/JKB
나섰다.	나서/VV + 었/EP + 다/EF + ./SF

(Park et al., 2023)

This approach scales to both constituency and dependency frameworks, as exemplified in the Universal Dependencies L2-Korean treebank, which aligns eojeol tokenization protocols and XPOS inventories for robust cross-linguistic comparison and modeling (Sung et al., 18 Mar 2025).

7. Rationale, Implications, and Resource Interoperability

Fixing $m_j$0 as the common terminal set and relegating morphology to a parallel layer yields several system-level advantages:

• Stable terminals across treebanks: All major resources represent sentences as sequences of eojeol, preventing ad hoc splits/merges and supporting universal tools.
• Predictable normalization: Limited, transparent operations suffice to align Sejong, Penn, and related resources.
• Straightforward conversion: Both dependency and constituency analyses become mutually accessible, as the same atomic units (eojeol) and tag inventories are leveraged throughout.
• Annotation modularity: Updates or refinements to the morphological lexicon, feature set, or POS granularity do not necessitate whole-treebank rewrites.
• Efficient downstream task integration: Eojeol-based segmentation, especially Level 4, combines linguistic adequacy (explicitly marked case and verb inflection) and statistical learnability for parsing, as reflected in increased $m_j$1 scores for phrase-structure tasks (Park et al., 2023).

A plausible implication is that as additional corpora (e.g., L2-Korean UD, dialectal resources) adopt eojeol-based annotation, corpus-level resource alignment, parser transferability, and multi-treebank learning are enabled or simplified.

---

References:

(Park et al., 27 Dec 2025) "Constituency Structure over Eojeol in Korean Treebanks" (Park et al., 2023) "Word segmentation granularity in Korean" (Sung et al., 18 Mar 2025) "Second language Korean Universal Dependency treebank v1.2: Focus on data augmentation and annotation scheme refinement"