---
title: Eojeol-Based Annotation Scheme
url: https://www.emergentmind.com/topics/eojeol-based-annotation-scheme
type: topic
---

# Eojeol-Based Annotation Scheme

An eojeol-based annotation scheme is a linguistic annotation framework where the basic unit of syntactic bracketing and segmentation for Korean is the eojeol—a whitespace-delimited token corresponding to a stem plus its grammatical affixes—rather than minimal morphemes. This approach separates phrase-level syntax from word-internal morphology and provides a framework for representational equivalence and cross-resource conversion in Korean treebanks. Recent developments standardize the use of eojeol as the atomic unit for both constituency and dependency representations, with independent but aligned morphological and syntactic annotation layers [2512.22487][2309.03713][2503.14718].

## 1. Eojeol as Syntactic Terminal: Definitions and Distinctions

In Korean orthography, an eojeol is defined as the contiguous sequence of non-space symbols between whitespaces. Formally, let $\Sigma$ be the alphabet of valid surface symbols (Hangul, Latin, numerals, etc.), then an eojeol is any non-empty string:

\[
e = c_1 c_2 \dots c_k,\;\; c_i \in \Sigma,\;\; k \geq 1
\]

Sentences are sequences $S = e_1\,␣\,e_2\,␣\,\dots\,␣\,e_n$ of eojeols. Each eojeol can be morphologically segmented into morphemes: $\operatorname{seg}(e) = \langle m_1, \ldots, m_k \rangle$, where $m_j$ are (possibly bound) minimal expression units (stems, suffixes, particles, etc.) [2309.03713][2512.22487].

Traditional Korean treebanks (e.g., Sejong, Penn) have used morphemes as constituency terminals—interleaving word-internal morphology with phrase-structure bracketing. The eojeol-based proposal treats each eojeol as a syntactic atom: terminals are the set $\Sigma_\text{surface}$ of observed eojeols, with finer-grained morphological information encapsulated in a non-tree layer.

This distinction is critical for compatibility: dependency resources (KUD, UD-Korean) already use eojeol tokens, and the eojeol-based constituency guarantees one-to-one terminal correspondence, facilitating conversion and multi-framework interoperability [2512.22487][2503.14718].

## 2. Two-Layer Annotation Architecture

The modern eojeol-based scheme employs a strict two-layer annotation:

- **Constituency layer $C=(V,N,P,\text{root})$:**
  - $V$ = $(v_1,\dots,v_n)$, linear surface eojeol sequence, $v_i \in \Sigma_\text{surface}$
  - $N$ = set of nonterminal phrase categories (S, NP, VP, etc.)
  - $P$ = production set: $N \rightarrow (N \cup \text{Pre})^+$, where Pre are preterminal UPOS categories
  - Each eojeol $v_i$ is dominated by exactly one UPOS preterminal.

- **Morphological layer $M$:**
  - For each $v_i$, $\operatorname{seg}(v_i) = \langle (m_1, X_1), \dots, (m_k, X_k) \rangle$, with fine-grained XPOS (e.g., Sejong tagset).

The paired annotation for sentence $j$ is $(C_j,\,M_j)$, with $C_j$ over $\Sigma_\text{surface}$ and $M_j : \Sigma_\text{surface} \to (\Sigma_\text{morph} \times \text{XPOS})^*$ [2512.22487].

This architecture decouples syntax (constituency over eojeol terminals) from detailed morphosyntactic segmentation, promoting modularity. The morphological segmentation and XPOS annotation are equivalent to a parallel table lookup or stand-off annotation, enabling representation-neutral enrichment and downstream task flexibility [2309.03713][2503.14718].

## 3. Normalization and Cross-Treebank Equivalence

Explicit normalization operations establish formal equivalence between previously divergent Korean treebanks:

1. **Debinarization ($d$):** Converts Chomsky-normal (binary) branching to n-ary (flat) structures, e.g., collapsing $NP \to NP\ NP$ chains to $NP \to NP^+$.
2. **Null removal and canonicalization ($r$):** Prunes empty-category leaves (e.g., *pro*, *trace*), reattaching siblings to preserve overt surface dominance.
3. **Eojeol boundary realignment ($a$):** Ensures a uniform segmentation by re-tokenizing so that both resources exhibit identical eojeol boundaries.

Sejong trees ($T_S$) are mapped via $f_S = a \circ r \circ d$, Penn trees ($T_P$) via $f_P = a \circ r$. The claim holds:

\[
\forall s\!: f_S(T_S(s)) = f_P(T_P(s)) = T_0(s)
\]

where $T_0(s)$ is the canonical eojeol-based constituency tree for sentence $s$. This demonstrates that under normalization, these treebanks are representationally equivalent at the eojeol-based constituency level [2512.22487].

## 4. Annotation Guidelines, Tag Inventories, and Segmentation Conventions

### 4.1 Tree Conversion Procedure

To convert legacy morpheme-terminal trees to the eojeol scheme:

1. Identify terminal nodes not corresponding to a complete eojeol.
2. Collapse all sibling morpheme terminals from the same eojeol into a single leaf labeled by the eojeol.
3. Prune unary branches due solely to morpheme structure.
4. Debinarize using $d$.
5. Remove null categories, reattaching overt siblings via $r$ [2512.22487].

### 4.2 Morphological Layer Conventions

- **UPOS** (preterminal inventory): {NOUN, VERB, PROPN, ADJ, ADV, PRON, ADP, SCONJ, CCONJ, PART, NUM, PUNCT, X, ...}
- **XPOS** (fine-grained, e.g., Sejong tags): NNP, NNG, VCP, VV, EP, EF, EC, ETM, ETN, JKS, JKG, JKO, JKB, SF, ...
- **Segmentation notation:** $e = m_1/X_1 + \ldots + m_k/X_k$ ("+" marks morpheme boundaries).

Eojeol internal segmentation follows a functional-analytic principle: only functional morphemes (case markers and verbal endings) are split off, while derivational suffixes remain attached to their stems [2309.03713]. This is termed Level 4 segmentation and yields optimal statistical and linguistic outcomes, outperforming both coarser and finer schemes (Level 4 $F_1 = 84.64\%$ vs. Level 1 $F_1 = 76.69\%$, Level 5 $F_1 = 82.23\%$ for phrase-structure parsing) [2309.03713].

A recommended six-column TSV format for annotation:

| Col 1: ID | Col 2: Surface | Col 3: MorphSeg (XPOS) | Col 4: UPOS | Col 5: Left-brackets | Col 6: Right-brackets |
|-----------|----------------|------------------------|-------------|---------------------|----------------------|

Columns 5–6 encode bracketed syntactic constituency over eojeol terminals [2512.22487].

## 5. Algorithms and Conversion Between Constituency and Dependency

### 5.1 Constituency to Dependency (Head-Projection)

A deterministic head-projection procedure computes dependencies from constituency trees over eojeol terminals:

```pseudo
Input: Constituency tree C over terminals v₁...vₙ
Output: Dependency arcs head→dependent among vᵢ

function HEADDEP(C):
    for each nonterminal N with children C₁...Cₖ:
        h = HeadChild(N) // by head-rule table
        for each i ≠ h with subtree Cᵢ:
            for each terminal v in yield(Cᵢ):
                add arc (head = first_leaf(yield(C_h)), dep = v)
return set of all arcs
```

Head rules reflect Korean-specific syntactic typologies: NP heads are the rightmost noun or pronoun, VP heads are verbs with highest XPOS priority, and PP heads are the postposition-bearing eojeol.

### 5.2 Dependency to Constituency (Eojeol-Grouping)

A "span-growing" algorithm reconstructs flat, n-ary constituency structure from dependency trees:

```pseudo
Input: Dependency tree D over v₁...vₙ
Output: Shallow constituency tree C′

1. Initialize each vᵢ as its own constituent C_i = leaf(vᵢ).
2. While ∃ nonterminal n such that dependents form contiguous [i..j]:
    H = head(n), span = [min_dep..max_dep]
    create new constituent N spanning v_min_dep...v_max_dep
    attach C_min_dep...C_max_dep under N
3. Assign labels N according to dependency label or POS of H
4. Return start symbol S dominating all
```
[2512.22487]

This formalism enables one-to-one conversions and supports application in parsers, transfer learning, and cross-framework synthesis.

## 6. Empirical Examples and Practical Implementation

Empirical evaluation and illustration appear in all major works. For instance, the normalized tree for the sentence “프랑스의 세계적인 의상 디자이너 엠마누엘 웅가로가 실내 장식용 직물 디자이너로 나섰다.” is annotated as follows under Level 4/eojeol-based guidelines:

| Eojeol         | Morpheme Segmentation (POS)                          |
|----------------|------------------------------------------------------|
| 프랑스의       | 프랑스/NNP + 의/JKG                                  |
| 세계적인       | 세계/NNG+적/XSN+이/VCP + ㄴ/ETM                       |
| 의상           | 의상/NNG                                             |
| 디자이너       | 디자이너/NNG                                         |
| 엠마누엘       | 엠마누엘/NNP                                         |
| 웅가로가       | 웅가로/NNP + 가/JKS                                   |
| 실내           | 실내/NNG                                             |
| 장식용         | 장식용/NNG                                           |
| 직물           | 직물/NNG                                             |
| 디자이너로     | 디자이너/NNG + 로/JKB                                |
| 나섰다.        | 나서/VV + 었/EP + 다/EF + ./SF                       |

[2309.03713]

This approach scales to both constituency and dependency frameworks, as exemplified in the Universal Dependencies L2-Korean treebank, which aligns eojeol tokenization protocols and XPOS inventories for robust cross-linguistic comparison and modeling [2503.14718].

## 7. Rationale, Implications, and Resource Interoperability

Fixing $\Sigma_\text{surface}$ as the common terminal set and relegating morphology to a parallel layer yields several system-level advantages:

- **Stable terminals across treebanks**: All major resources represent sentences as sequences of eojeol, preventing ad hoc splits/merges and supporting universal tools.
- **Predictable normalization**: Limited, transparent operations suffice to align Sejong, Penn, and related resources.
- **Straightforward conversion**: Both dependency and constituency analyses become mutually accessible, as the same atomic units (eojeol) and tag inventories are leveraged throughout.
- **Annotation modularity**: Updates or refinements to the morphological lexicon, feature set, or POS granularity do not necessitate whole-treebank rewrites.
- **Efficient downstream task integration**: Eojeol-based segmentation, especially Level 4, combines linguistic adequacy (explicitly marked case and verb inflection) and statistical learnability for parsing, as reflected in increased $F_1$ scores for phrase-structure tasks [2309.03713].

A plausible implication is that as additional corpora (e.g., L2-Korean UD, dialectal resources) adopt eojeol-based annotation, corpus-level resource alignment, parser transferability, and multi-treebank learning are enabled or simplified.

---

**References**:  
[2512.22487] "Constituency Structure over Eojeol in Korean Treebanks"  
[2309.03713] "Word segmentation granularity in Korean"  
[2503.14718] "Second language Korean Universal Dependency treebank v1.2: Focus on data augmentation and annotation scheme refinement"

Source: https://www.emergentmind.com/topics/eojeol-based-annotation-scheme