Language Dependency Parsing

Updated 4 July 2026

Language dependency parsing is a mechanism that converts sentences into directed trees where each token is attached to one head under constraints like acyclicity and single-headness.
It encompasses diverse paradigms such as transition-based, graph-based, and text-to-text approaches, each employing neural or algorithmic scoring to determine head assignments and dependency labels.
Recent research integrates segmentation, morphology, and multilingual adaptation, achieving high attachment scores (UAS/LAS) and addressing challenges like long-distance dependencies.

Language dependency parsing is the family of mechanisms that map a sentence to a directed dependency structure in which each token is attached to exactly one head token or to a special ROOT, typically yielding a tree with labeled head–dependent arcs. Across the literature, the mechanism has been instantiated as graph-based scoring with maximum spanning tree or projective dynamic programming, transition-based action sequences over parser states, fully neural head selection from characters or contextual encoders, sequence generation over serialized structures, and joint models that integrate segmentation, morphology, syntax, or multilingual adaptation. The common objective is to recover head assignment and dependency labels under constraints such as single-headness, acyclicity, connectivity, and, in some settings, projectivity; standard evaluation uses Unlabeled Attachment Score (UAS) and Labeled Attachment Score (LAS) (Cheng et al., 2016, Chorowski et al., 2016, Zhang et al., 2020, Kim et al., 24 Feb 2025).

1. Formal representation and structural constraints

Dependency parsing is typically formulated over a sentence $x = (w_1,\dots,w_n)$ , optionally augmented with an artificial root. A dependency tree specifies, for each token $i$ , a head index $h_i$ and a relation label $r_i$ , so that the parse can be written as $T=\{(h_i,i,r_i)\}_{i=1}^n$ . The resulting directed graph is required to satisfy single-headness, acyclicity, and connectivity; many systems also assume or enforce projectivity, while others explicitly permit non-projective arcs. Several papers state these constraints directly: the delexicalized MST-based transfer framework defines a dependency tree as a directed spanning arborescence rooted at $r$ with exactly one incoming edge per node and no cycles, with non-projective structures allowed (Rosa, 2015); the structuralized prompt formulation makes the same constraints explicit through head-index tokens and notes that projectivity is optional (Kim et al., 24 Feb 2025); the joint segmentation–parsing model for Hebrew constrains decoding to choose exactly one analysis per token and to output a single best maximum spanning tree over the selected morphemes (Levi et al., 2024).

Evaluation is standardized around attachment metrics. UAS is the fraction of tokens assigned the correct head, while LAS additionally requires the correct dependency label. Some work also reports LA, MLAS, or BLEX in settings where label correctness, morphology, or lemma accuracy are salient (Cheng et al., 2016, Habib, 2024, Qi et al., 2019). For semantic dependency parsing, the output need not be a tree: the sequence-generation model distinguishes syntactic dependency parsing, where each non-root token has exactly one head, from semantic dependency parsing, where the structure is a labeled directed acyclic graph and a token may have multiple heads or none (Lin et al., 2022).

Projectivity divides major decoding regimes. First-order projective parsers commonly use Eisner-style dynamic programming with $O(n^3)$ complexity, as in the BiLSTM feature-based graph parser and in Chinese biaffine parsing when projective decoding is required (Kiperwasser et al., 2016, Zhang et al., 2020). Non-projective systems instead apply Chu–Liu/Edmonds maximum spanning arborescence decoding in $O(n^2)$ dense-graph settings, as in attention-based graph parsing, multilingual biaffine parsing, and several transfer or lattice-based models (Cheng et al., 2016, Üstün et al., 2020, Rosa, 2015, Levi et al., 2024). A different line, represented by the fundamental incremental algorithm, frames parsing as attaching words “as soon as possible” while maintaining projectivity through list-based constraints rather than dynamic programming (Covington, 22 Oct 2025).

2. Principal algorithmic paradigms

Two classical paradigms organize most dependency parsing mechanisms: transition-based parsing and graph-based parsing. Transition-based parsers build the tree incrementally from parser states such as stack, buffer, and arc set. Graph-based parsers score candidate arcs globally and then decode a well-formed tree. The distinction appears repeatedly in the literature, including the BiAtt-DP work, the Turkish hybrid biaffine parser, the Urdu MaltParser system, and the BiLSTM feature-learning paper (Cheng et al., 2016, Özateş et al., 2020, Habib, 2024, Kiperwasser et al., 2016).

Transition-based mechanisms are exemplified by NivreEager and pointer-network decoders. In the Urdu system, the parser state is $c=(\sigma,\beta,A)$ , with SHIFT, LEFT-ARC, RIGHT-ARC, and REDUCE transitions, and NivreEager is described as a projective, linear-time algorithm that performed best among nine MaltParser algorithms for the annotated Urdu corpus (Habib, 2024). In neural transition parsing, the BiLSTM-feature parser uses Arc-Hybrid with configurations over stack, buffer, and arc set, and scores legal transitions by an MLP over a very small number of contextual token vectors (Kiperwasser et al., 2016). Pointer-network models go further by selecting heads directly at each step: bottom-up hierarchical pointer networks decode in left-to-right, right-to-left, or outside-in orders, using biaffine attention between a decoder state and candidate heads, while integrating subtree summaries such as leftmost, rightmost, and last-attached dependents (Fernández-González et al., 2021).

Graph-based mechanisms cover first-order MST parsers, biaffine neural parsers, attention-based parsers, and arc-factored cross-lingual models. The delexicalized cross-lingual transfer system uses MSTperl, an implementation of the unlabeled, single-best, first-order MSTParser trained with 3 iterations of MIRA and decoded with Chu–Liu/Edmonds or Eisner depending on the setting (Rosa, 2015). The fully neural parser of “Read, Tag, and Parse All at Once” factorizes head selection independently per token, scoring all locations $l\in\{0,\dots,n\}$ for each dependent $i$ 0 by a small MLP and optionally repairing rare cycles with Chu–Liu/Edmonds (Chorowski et al., 2016). DDParser adopts the deep biaffine architecture, computing arc scores and relation scores by biaffine transformations over BiLSTM-based head and dependent representations, with an in-order traversal pre-check and Eisner decoding when the greedy tree is not already projective (Zhang et al., 2020).

A separate line replaces parser-specific decision heads with text generation. DPSG serializes each dependency arc into a unit of the form “dependent [REL(label)] head-position” and trains an encoder–decoder PLM to generate the serialized structure autoregressively, without an auxiliary decoder, biaffine module, transition system, or pointer network (Lin et al., 2022). SPT-DP applies a related text-to-text idea to encoder-only models: each token is wrapped in a fixed template such as $i$ 1, and masked language modeling reconstructs head indices and labels directly from the LM head (Kim et al., 24 Feb 2025). These mechanisms do not eliminate structural assumptions, but they relocate them into serialization and prompting rather than dedicated parsing layers.

The range of mechanisms can be summarized as follows.

Paradigm	Representative mechanism	Representative paper
Transition-based	Stack/buffer actions, pointer decoding, hierarchical subtree-aware state updates	(Habib, 2024, Fernández-González et al., 2021, Kiperwasser et al., 2016)
Graph-based	Arc scoring over all pairs with Eisner or MST decoding	(Cheng et al., 2016, Zhang et al., 2020, Rosa, 2015)
Text-to-text / generation	Serialized dependencies predicted by PLMs or encoder-only prompt templates	(Lin et al., 2022, Kim et al., 24 Feb 2025)

This suggests that “dependency parsing mechanism” is best understood not as a single algorithm, but as a design space defined by how candidate heads are represented, how structural constraints are enforced, and where inductive bias is placed.

3. Representational layers: words, characters, morphology, and lattices

A central development in modern dependency parsing is the shift from discrete feature templates toward learned token representations. The fully neural parser of 2016 is exemplary: each token is represented as a character sequence with boundary markers such as $i$ 2, embedded into $i$ 3, passed through a bank of 1D convolutional filters with max-over-time pooling, projected to 512 dimensions, and transformed by 3 Highway layers to produce a word embedding $i$ 4 (Chorowski et al., 2016). The paper explicitly argues that character-only input is beneficial for morphologically rich languages because suffixes, inflections, and boundary-sensitive patterns carry strong grammatical information.

Several systems adopt character- or morpheme-aware tokenization for related reasons. DDParser concatenates a word embedding with a character-level representation learned by a BiLSTM over the token’s characters, and reports that on DuCTB, replacing POS tag embeddings with character-level encodings improved robustness to Chinese word segmentation noise and OOVs (Zhang et al., 2020). The Turkish hybrid parser augments a biaffine parser with rule embeddings and three alternative morphology channels: an Inflectional Suffixes Model, a Last Suffix Model, and a Suffix Vector Model derived from lemma–suffix usage profiles (Özateş et al., 2020). The Urdu MaltParser configuration similarly emphasizes POS tags and morphological FEATS such as gender, number, and suffixes, arguing that free word order reduces the utility of purely positional cues and increases dependence on morphology (Habib, 2024).

In multilingual and pretrained settings, contextual encoders replace much of this manual engineering. UDapter inserts Houlsby-style adapters into frozen mBERT and generates both adapter parameters and biaffine parser parameters from typology-based language embeddings. Only the first WordPiece per word is aligned to the UD token, and the downstream parser uses that contextual representation in a biaffine scorer (Üstün et al., 2020). “Universal Dependency Parsing from Scratch” combines pretrained word embeddings, trainable frequent-word and lemma embeddings, character-level word embeddings, and summed POS/morphology embeddings before a 3-layer highway BiLSTM parser (Qi et al., 2019). The cross-lingual order-sensitivity study compares BiLSTM encoders with modified self-attentive encoders using relative positional self-attention, specifically to test how much parser behavior depends on English-like sequential order (Ahmad et al., 2018).

The richest representational treatment appears when token boundaries themselves are ambiguous. The joint Hebrew architecture begins not from a token sequence but from a morphological lattice produced by an analyzer. Each token contributes multiple candidate analyses, each analysis is a morpheme sequence, and the entire sentence-level lattice is linearized into a list of morphemes. The parser then scores arcs between all candidate morphemes, subject to constraints that exactly one analysis per token is chosen, all morphemes of that analysis are included, and arcs cannot mix morphemes from different analyses of the same token (Levi et al., 2024). This replaces the standard assumption that parser nodes are known in advance with a joint segmentation-and-parsing mechanism.

These representational choices directly affect parser scope. Character models and morphological embeddings primarily strengthen parsing under rich inflectional systems; lattice representations generalize this by making segmentation itself part of parsing; typology-conditioned contextual encoders address cross-language interference. A plausible implication is that the history of dependency parsing mechanisms can be read as progressive movement of linguistic uncertainty—from labels, to heads, to token identity, to segmentation—into learned internal representations.

4. Scoring functions, objectives, and decoding procedures

Despite wide architectural variation, most mechanisms decompose into three components: a scorer for candidate heads, a scorer for labels, and a decoding procedure that enforces or approximates tree well-formedness. In arc-factored graph parsing, the scorer is typically local. The fully neural character-based parser computes

$i$ 5

normalizes over candidate head positions with a softmax, and minimizes head cross-entropy

$i$ 6

Label prediction uses a Maxout classifier on the dependent and either a soft or hard head representation, with a combined objective

$i$ 7

and best weights $i$ 8 (Chorowski et al., 2016).

BiAtt-DP uses a related but explicitly bidirectional mechanism. A bi-directional GRU encodes memory vectors $i$ 9 for all candidate heads, two uni-directional GRUs produce forward and backward query states, and attention scores are computed as

$h_i$ 0

The two directional head distributions are trained against the gold head via a joint likelihood that includes label probabilities, while test-time arc scores are combined as $h_i$ 1. The model can use local argmax or MST decoding, with MST typically improving UAS/LAS by less than $h_i$ 2 absolute while adding about $h_i$ 3– $h_i$ 4 runtime overhead (Cheng et al., 2016).

Biaffine scoring became the dominant neural graph-based mechanism. DDParser computes arc scores

$h_i$ 5

relation scores $h_i$ 6 by an analogous label-specific biaffine form, and optimizes

$h_i$ 7

It uses projective Eisner decoding, but only after a pre-check determines that the greedy tree is not already projective (Zhang et al., 2020). Stanford’s CoNLL 2018 system extends deep biaffine parsing with explicit linearization and distance terms: a sign-sensitive linearization score and a Cauchy-like distance penalty are added to the arc score before Chu–Liu/Edmonds decoding (Qi et al., 2019).

Transition-based objectives instead score action sequences or pointer choices. The BiLSTM Arc-Hybrid parser constructs a feature vector $h_i$ 8 from just four BiLSTM token vectors and trains with a margin-based hinge loss over legal and gold transitions under a dynamic oracle (Kiperwasser et al., 2016). Hierarchical pointer networks define the head loss

$h_i$ 9

and a separate label loss over created arcs, while maintaining decoder-state summaries of leftmost, rightmost, and last-attached dependents (Fernández-González et al., 2021). Classical MaltParser systems train a classifier over SHIFT, LEFT-ARC, RIGHT-ARC, and REDUCE transitions without explicit global normalization (Habib, 2024).

Generation-based methods internalize scoring in language-model likelihoods. DPSG trains with the standard autoregressive objective

$r_i$ 0

and then deserializes generated dependency units into trees or DAGs (Lin et al., 2022). SPT-DP instead uses token-wise cross-entropy on masked template positions, selecting head tokens from the sub-vocabulary $r_i$ 1 and relation tokens from the dependency-label sub-vocabulary, with optional lightweight post-checks for cycles or invalid indices (Kim et al., 24 Feb 2025).

Unsupervised formulations replace supervised attachment loss with marginal likelihood, reconstruction, or regularized variational objectives. The survey of unsupervised dependency parsing gives the generic objective

$r_i$ 2

where sentence likelihood marginalizes over latent trees, and highlights EM, Viterbi EM, autoencoding, variational inference, and posterior regularization as the main optimization families (Han et al., 2020). This broadens the notion of dependency parsing mechanism beyond labeled supervised prediction to latent-structure induction.

5. Joint, multilingual, and cross-lingual mechanisms

A major theme in later work is that parsing need not be isolated. It can be regularized by POS tagging, integrated with segmentation, tied to semantic parsing, or adapted to many languages through soft parameter sharing. The fully neural character-only parser already includes an auxiliary POS-tagging branch that taps an intermediate BiGRU layer and predicts categorical attributes such as coarse POS, case, number, and gender; the POS loss is added to the parsing losses and yields clear gains on low-resource Polish (Chorowski et al., 2016). DDParser similarly uses different token-feature mixes on DuCTB and CTB5, replacing POS embeddings with CharLSTM encodings for its large Chinese industrial corpus while retaining gold POS embeddings on CTB5 (Zhang et al., 2020).

True joint morpho-syntactic parsing appears in the Hebrew lattice model. Rather than running segmentation and parsing in a pipeline, it treats the morphological disambiguation and dependency tree as a single constrained decoding problem,

$r_i$ 3

where $r_i$ 4 encodes exactly-one-analysis-per-token and related lattice constraints (Levi et al., 2024). This is a direct neural realization of the joint morpho-syntactic hypothesis described in earlier pre-neural work.

Cross-lingual transfer introduces a different kind of jointness: parameter sharing across languages. The delexicalized transfer framework trains MST parsers on multiple source treebanks, estimates source–target similarity with the KLcpos3 measure over UPOS trigram distributions, and combines models by weighted interpolation

$r_i$ 5

Weighted interpolation matches weighted tree combination while being cheaper at inference time (Rosa, 2015). UDapter replaces manual source selection with contextual parameter generation: typology features are mapped to a 32-dimensional language embedding, and shared linear generators produce language-specific adapter and biaffine parameters inside a frozen multilingual BERT. This yields average LAS 87.3 on 13 high-resource languages and 36.5 average LAS on 30 zero-shot languages, outperforming monolingual and multilingual baselines reported in the paper (Üstün et al., 2020).

The effect of word order on transfer is treated explicitly in the English-to-30-languages study. There, RNN encoders and modified self-attentive encoders are paired with graph-based or stack-based decoders to test whether order-agnostic mechanisms transfer better. The paper reports that self-attentive graph-based parsing achieves the best average zero-shot performance and that self-attention is especially advantageous on distant languages whose word-order distributions diverge from English (Ahmad et al., 2018). This suggests that dependency parsing mechanism is sensitive not only to local morpho-syntactic cues, but also to the form of contextualization used before arc scoring.

Jointness can also cross representational formalisms and tasks. “Parsing All” shares one self-attention encoder across constituency parsing, dependency parsing, span SRL, and dependency SRL. For syntactic parsing, it combines constituent span scores and dependency head scores inside a joint span dynamic program, with best PTB development performance at $r_i$ 6 for balancing the two (Zhou et al., 2019). DPSG extends this idea differently: a single PLM-only sequence generator handles both syntactic and semantic dependency parsing, and even multi-schemata parsing, by switching only the prefix token and relation-token inventory (Lin et al., 2022).

6. Performance patterns, limitations, and emerging directions

Across these mechanisms, several performance regularities recur. Character-based and morphology-aware parsers are especially effective for morphologically rich languages. The fully neural parser reports that, on UD v1.3 with characters only at inference, its LAS exceeds SyntaxNet on Czech and Russian and is competitive with ParseySaurus on Polish (Chorowski et al., 2016). The Turkish hybrid parser improves over the Stanford biaffine baseline on IMST-UD, with the combined rule + last-suffix model reaching UAS 74.37 and LAS 68.63, both statistically better than the baseline according to the paper (Özateş et al., 2020). The Urdu transition-based parser reaches UAS $r_i$ 7 and LAS $r_i$ 8 with NivreEager and Liblinear on its news-domain corpus (Habib, 2024).

Large-scale neural graph parsing on well-resourced corpora reaches substantially higher attachment accuracy. DDParser reports UAS 94.8% and LAS 92.9% on the standard DuCTB test set and UAS 89.7% and LAS 86.9% on a random test from unseen sources, quantifying domain-shift sensitivity (Zhang et al., 2020). SPT-DP reports PTB UAS 96.95 and LAS 95.88 with XLNet-large, and average LAS about 91.31 for its multilingual global model on UD 2.2 with XLM-RoBERTa-large (Kim et al., 24 Feb 2025). The sequence-generation model DPSG reports PTB UAS 96.48 and LAS 95.04, as well as state-of-the-art results on SemEval16 and CODT benchmarks under its evaluation setup (Lin et al., 2022).

Multilingual adaptation and truly joint models tend to improve low-resource or ambiguous settings. UDapter shows larger gains for smaller treebanks and a sharp zero-shot drop when typology-based language embeddings are replaced by a centroid embedding, which the paper reports collapses zero-shot average LAS to 9.0 from 36.5 (Üstün et al., 2020). The Hebrew lattice model narrows the gap between predicted and oracle segmentation by resolving segmentation and parsing jointly, and with POS multitask learning reaches SEG 97.71, POS 94.41, and DEP 85.45 on the realistic uninfused setting (Levi et al., 2024).

At the same time, limitations are persistent and well documented. Arc-factored parsers ignore global tree structure during training and only weakly impose it during decoding (Chorowski et al., 2016). First-order biaffine and transition-based models remain vulnerable to long-distance dependencies, PP attachment, coordination, and domain shift (Zhang et al., 2020, Fernández-González et al., 2021, Levi et al., 2024). Sequence-generation approaches occasionally produce structurally illegal outputs; DPSG reports structural legality of 99.7% on PTB, implying a nonzero residual error rate without explicit constraints (Lin et al., 2022). Prompt-template approaches do not enforce acyclicity or connectivity during encoder-only decoding, though they admit optional repair heuristics (Kim et al., 24 Feb 2025). Unsupervised parsing remains far below supervised performance and is highly sensitive to initialization, priors, and constraints (Han et al., 2020).

Several future directions are repeatedly identified. One is stronger structural learning: global structured objectives, CRF-style tree constraints, differentiable MST, or better integration of head-selection uncertainty are explicitly suggested in the fully neural parser and elsewhere (Chorowski et al., 2016). Another is richer contextualization and parameter sharing: Transformer encoders, multilingual sharing, and language-conditioned hypernetworks appear as natural successors to recurrent encoders and static delexicalized models (Üstün et al., 2020, Ahmad et al., 2018). A third is wider task unification: joint segmentation–parsing, syntax–semantics multitask learning, and schema-free generation all point toward parsers that treat dependency structure not as an isolated product, but as one layer in a broader structured prediction system (Levi et al., 2024, Zhou et al., 2019, Lin et al., 2022).

Taken together, the literature portrays language dependency parsing mechanisms as a progression from explicit feature templates and deterministic transitions toward neural scoring over richer latent representations, then toward mechanisms that jointly resolve multiple sources of linguistic uncertainty. This suggests that the central technical question is no longer merely how to decode a tree, but how to decide what the nodes are, what contextual information should condition arc decisions, and how much of linguistic structure should be learned jointly rather than pipelined.