---
title: Split-Sequence Transition System for AMR Parsing
url: https://www.emergentmind.com/topics/split-sequence-transition-system
type: topic
---

# Split-Sequence Transition System for AMR Parsing

Searching arXiv for the main paper and related AMR transition-based parsing work mentioned in the provided data.
The "Split-Sequence Transition System" (*Editor's term*) denotes the simplified transition set introduced for structure-aware fine-tuning of sequence-to-sequence Transformers in transition-based AMR parsing. In the underlying formulation, the parser scans the source sentence from left to right using a token cursor, may either advance that cursor with `shift` or generate graph components while remaining on the same token, and emits actions as decoder targets for BART. The system is explicitly presented as a simplification of the earlier Action-Pointer Transition System of Zhou et al. (2021), while preserving monotonic construction, built-in word-to-node alignments, and guaranteed graph well-formedness; in the reported experiments it reaches the new parsing state of the art for AMR 2.0 without the need for graph re-categorization [2110.15534].

## 1. Position within AMR parsing

Predicting linearized Abstract Meaning Representation graphs with pre-trained sequence-to-sequence Transformer models had already led to large improvements on AMR parsing benchmarks, but those parsers avoided explicit modeling of structure and therefore lacked desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments [2110.15534]. The simplified transition system was proposed to integrate general pre-trained sequence-to-sequence language models with a structure-aware transition-based approach rather than treating AMR parsing as unconstrained string generation.

The resulting design is deliberately left-to-right and monotonic. At any moment, the cursor points to one source token, and the parser may either move one word to the right with `shift` or generate any number of graph components while staying on the same token. This makes the system resemble a monotonic transducer from sentence to action sequence, with the action sequence serving as the structured target representation.

A central point of the proposal is that simplification is not treated as a loss of structure. Instead, special subgraph-specific and hand-crafted operations are removed while the main structural benefits of transition-based AMR parsing are retained. This suggests that the simplification is aimed less at reducing formal expressive power than at improving the fit between transition-based parsing and a pre-trained encoder-decoder Transformer.

## 2. Transition state and action inventory

The transition state is not defined through heavy symbolic stack/buffer machinery. It is essentially encoded by the source sentence with a **cursor position** over tokens, the **history of previously generated actions**, and the **current partial AMR graph** induced by those actions [2110.15534]. This formulation shifts the burden of state representation from explicit symbolic data structures to the encoder-decoder model and its action history.

The action inventory consists of six base actions:

- `shift`: move the token cursor one word to the right.
- `<string>`: create a node whose concept label is the symbol `<string>`.
- `copy`: create a node whose label is the token under the current cursor.
- `la(j, lbl)`: create a labeled arc from the **last generated node** to the node generated at the `j`-th transition step.
- `ra(j, lbl)`: same as `la`, but with the arc direction reversed.
- `root`: declare the last predicted node as the root.

A crucial design choice is that node identity is tied to the **action that created the node**. Edge actions therefore do not point to graph nodes directly, but to the transition step index `j` in the action history. This preserves the pointer-based character of earlier work while reducing the action inventory itself.

The simplification is defined as much by removal as by retention. Compared to prior transition systems, the proposal removes `pred(<string>)`, `confirm`, `subgraph`, `entity`, `merge`, and `reduce` [2110.15534]. In this sense, the system keeps only the minimal actions needed for AMR: `shift`, concept creation, arc creation via pointers, and `root`.

## 3. Oracle construction and decoding order

The paper makes the decoding logic explicit even though it does not introduce complex equations for transition dynamics. The oracle that creates gold action sequences follows a fixed priority order: first, create the next gold arc between the last created node and previously created nodes; second, create the next gold node aligned to the token under the cursor; third, if not at sentence end, emit `shift`; fourth, finish [2110.15534].

If possible, nodes are emitted as `copy`; otherwise as `<string>`. Arcs are created using `la` and `ra`, and the oracle prefers connecting to nodes that are **closer** to the current node before farther ones. When multiple nodes align to the same token, they are produced in a fixed topological order, with edge-generation actions interleaved as needed. The `root` action is emitted as soon as the root node is created.

Under these rules, the action sequence is deterministic given the graph and alignments. That determinism is important because it converts AMR parsing into a supervised sequence prediction problem without abandoning a graph-oriented construction process.

Figure 2 provides an illustrative sequence for the sentence “Employees liked their Boston trip.” The sequence is shown as something like `person`, `ra(1,arg1-of)`, `shift`, `like-01`, `la(1,arg0)`, `root`, further `shift` actions, then `city`, `name`, `copy`, and later `trip-03` with arc attachments. Interpreting the example, graph construction is interleaved with source traversal: at **Employees** the parser generates an employment/person structure; at **liked** it generates `like-01` and marks it as root; at **Boston** it creates a `city`/`name` structure and uses `copy` to generate `Boston`; at **trip** it generates `trip-03` and introduces additional attachments, including a re-entrancy to `person`. The example illustrates that the action history encodes graph topology while the cursor enforces monotonic sentence traversal.

## 4. Internalization of parser state in BART

The parser is implemented by fine-tuning BART into a structured decoder that “internalizes parser states induced by our transition system” through three mechanisms [2110.15534]. The first is a **pointer network over action history**. Target actions are factored into bare action symbols and pointer values for edges, and one decoder self-attention head is repurposed as the pointer mechanism. Because the pointing occurs over the history of actions, decoder self-attention becomes the pointer network. That head is trained with an extra cross-entropy loss for pointer prediction.

The second mechanism is **hard attention over source alignments**. The monotonic source-to-action alignment induced by the cursor is encoded by masking cross-attention heads so that they can attend only to the aligned source word or words. Since BART operates on subwords, encoder outputs are average-pooled back to word-level states before being used for hard attention. This ties action generation directly to the cursor-defined alignment structure rather than leaving source access unconstrained.

The third mechanism is **output-space masking**. The decoder softmax is hard-masked so that only actions valid under the current transition state can be generated. This is the principal mechanism by which the sequence model is made structure-aware at decoding time.

The paper also states that it does not use the GNN-style step-wise graph embedding decoder from Zhou et al. (2021), because its gains were modest. A plausible implication is that the simplified transition system is intended to concentrate structural bias in the action space, attention pattern, and masking scheme rather than in an additional graph-state module.

## 5. Vocabulary strategies and structural guarantees

Vocabulary design is treated as a central component of making the transition system work with a pre-trained seq2seq model [2110.15534]. Two strategies are explored.

In the **separate-vocabulary model** (`StructBART-S` / `sep-voc`), the action vocabulary is separate from BART’s original subword vocabulary; the decoder input and output use stand-alone action embeddings; and those embeddings are initialized from the average of the corresponding BART subword embeddings. This cleanly separates action generation from text generation.

In the **joint-vocabulary model** (`StructBART-J` / `joint-voc`), source and target share a vocabulary; frequent node-creating actions are added as atomic vocabulary items; less frequent concepts are split into BART subwords; and non-node actions such as `shift` and `copy` are added as-is. If a node label is generated by multiple subword pieces, arc transitions always point to the **beginning** position of that node string. This avoids blind subword splitting of action symbols, which would otherwise break alignment and pointer bookkeeping.

The transition system preserves several explicit guarantees:

- **Well-formed graphs**: valid graphs are guaranteed without post-processing.
- **Built-in alignments**: because generation is monotonic and tied to the source cursor, every node is associated with a source token alignment.
- **No unattachable nodes**: this is achieved by removing subgraph actions.
- **Full recovery under alignments**: the oracle can recover the original graph exactly as long as every node has an alignment.
- **Monotonicity**: the cursor moves strictly left-to-right, and nodes are generated at the current token position before moving on.

The paper also addresses the fact that some AMR nodes are usually unaligned. To guarantee complete alignments, it uses a recursive heuristic: if a node is unaligned, copy the alignment from its first child if possible; otherwise copy from its first parent. Because the graph is assumed connected and at least some alignments exist, this recursive propagation assigns an alignment to all nodes [2110.15534].

## 6. Empirical profile and interpretation

The empirical results are presented as evidence that simplifying the transition system does not hurt performance and can help BART exploit structure better [2110.15534]. The comparison reported in the paper is summarized below.

| System | Base actions | Smatch oracle coverage / Avg. action length |
|---|---:|---:|
| Astudillo et al. (2020) | 12 | 98.0 / 76.2 |
| Zhou et al. (2021) | 10 | 98.9 / 41.6 |
| Proposed system | 6 | 99.9 / 45.6 |

A common misconception is that removing special subgraph machinery necessarily reduces oracle recoverability. The reported comparison does not support that view: the simplified system removes special subgraph machinery entirely, yet achieves **99.9** Smatch oracle coverage, compared with **98.0** for Astudillo et al. and **98.9** for Zhou et al. It also remains competitive in average action sequence length, at **45.6** actions versus **41.6** for Zhou et al. and **76.2** for Astudillo et al.

The reported parsing results further strengthen that interpretation. On AMR 2.0, `StructBART-J` reaches **84.2 Smatch** without extra silver data; with only **47K silver examples**, it reaches **84.7**, surpassing prior systems; and it also performs strongly on AMR 1.0 and AMR 3.0 [2110.15534]. The paper additionally notes that the same transition system does **not** help as much when used with the older APT model based on fixed RoBERTa features, but **does** help BART. This supports the claim that the simplified transition system is specifically well matched to a powerful seq2seq decoder.

The ablation findings are consistent with that interpretation. Hard attention over alignments is best; removing alignment modeling hurts; the joint-vocabulary setup benefits from BART’s subword generation, especially for rare concepts; and the `copy` action helps the separate-vocabulary model but is less critical in the joint-vocabulary model. Taken together, these results characterize the split-sequence style simplification as a transition system whose effectiveness depends not only on its reduced action set, but also on its coupling to pointer decoding, hard alignment control, and vocabulary design within BART [2110.15534].

Source: https://www.emergentmind.com/topics/split-sequence-transition-system