Split-Sequence Transition System for AMR Parsing

• Split-Sequence Transition System is a simplified method for integrating structure-aware AMR parsing into pre-trained sequence-to-sequence models.
• It uses a left-to-right, monotonic token cursor that generates deterministic action sequences, ensuring graph well-formedness and built-in alignments.
• Integration with BART employs pointer networks, hard attention, and output masking to boost parsing accuracy, achieving state-of-the-art Smatch scores.

Searching arXiv for the main paper and related AMR transition-based parsing work mentioned in the provided data. The "Split-Sequence Transition System" (Editor's term) denotes the simplified transition set introduced for structure-aware fine-tuning of sequence-to-sequence Transformers in transition-based AMR parsing. In the underlying formulation, the parser scans the source sentence from left to right using a token cursor, may either advance that cursor with shift or generate graph components while remaining on the same token, and emits actions as decoder targets for BART. The system is explicitly presented as a simplification of the earlier Action-Pointer Transition System of Zhou et al. (2021), while preserving monotonic construction, built-in word-to-node alignments, and guaranteed graph well-formedness; in the reported experiments it reaches the new parsing state of the art for AMR 2.0 without the need for graph re-categorization (Zhou et al., 2021).

1. Position within AMR parsing

Predicting linearized Abstract Meaning Representation graphs with pre-trained sequence-to-sequence Transformer models had already led to large improvements on AMR parsing benchmarks, but those parsers avoided explicit modeling of structure and therefore lacked desirable properties such as graph well-formedness guarantees or built-in graph-sentence alignments (Zhou et al., 2021). The simplified transition system was proposed to integrate general pre-trained sequence-to-sequence LLMs with a structure-aware transition-based approach rather than treating AMR parsing as unconstrained string generation.

The resulting design is deliberately left-to-right and monotonic. At any moment, the cursor points to one source token, and the parser may either move one word to the right with shift or generate any number of graph components while staying on the same token. This makes the system resemble a monotonic transducer from sentence to action sequence, with the action sequence serving as the structured target representation.

A central point of the proposal is that simplification is not treated as a loss of structure. Instead, special subgraph-specific and hand-crafted operations are removed while the main structural benefits of transition-based AMR parsing are retained. This suggests that the simplification is aimed less at reducing formal expressive power than at improving the fit between transition-based parsing and a pre-trained encoder-decoder Transformer.

2. Transition state and action inventory

The transition state is not defined through heavy symbolic stack/buffer machinery. It is essentially encoded by the source sentence with a cursor position over tokens, the history of previously generated actions, and the current partial AMR graph induced by those actions (Zhou et al., 2021). This formulation shifts the burden of state representation from explicit symbolic data structures to the encoder-decoder model and its action history.

The action inventory consists of six base actions:

• shift: move the token cursor one word to the right.
• <string>: create a node whose concept label is the symbol <string>.
• copy: create a node whose label is the token under the current cursor.
• la(j, lbl): create a labeled arc from the last generated node to the node generated at the j-th transition step.
• ra(j, lbl): same as la, but with the arc direction reversed.
• root: declare the last predicted node as the root.

A crucial design choice is that node identity is tied to the action that created the node. Edge actions therefore do not point to graph nodes directly, but to the transition step index j in the action history. This preserves the pointer-based character of earlier work while reducing the action inventory itself.

The simplification is defined as much by removal as by retention. Compared to prior transition systems, the proposal removes pred(<string>), confirm, subgraph, entity, merge, and reduce (Zhou et al., 2021). In this sense, the system keeps only the minimal actions needed for AMR: shift, concept creation, arc creation via pointers, and root.

3. Oracle construction and decoding order

The paper makes the decoding logic explicit even though it does not introduce complex equations for transition dynamics. The oracle that creates gold action sequences follows a fixed priority order: first, create the next gold arc between the last created node and previously created nodes; second, create the next gold node aligned to the token under the cursor; third, if not at sentence end, emit shift; fourth, finish (Zhou et al., 2021).

If possible, nodes are emitted as copy; otherwise as <string>. Arcs are created using la and ra, and the oracle prefers connecting to nodes that are closer to the current node before farther ones. When multiple nodes align to the same token, they are produced in a fixed topological order, with edge-generation actions interleaved as needed. The root action is emitted as soon as the root node is created.

Under these rules, the action sequence is deterministic given the graph and alignments. That determinism is important because it converts AMR parsing into a supervised sequence prediction problem without abandoning a graph-oriented construction process.

Figure 1 provides an illustrative sequence for the sentence “Employees liked their Boston trip.” The sequence is shown as something like person, ra(1,arg1-of), shift, like-01, la(1,arg0), root, further shift actions, then city, name, copy, and later trip-03 with arc attachments. Interpreting the example, graph construction is interleaved with source traversal: at Employees the parser generates an employment/person structure; at liked it generates like-01 and marks it as root; at Boston it creates a city/name structure and uses copy to generate Boston; at trip it generates trip-03 and introduces additional attachments, including a re-entrancy to person. The example illustrates that the action history encodes graph topology while the cursor enforces monotonic sentence traversal.

4. Internalization of parser state in BART

The parser is implemented by fine-tuning BART into a structured decoder that “internalizes parser states induced by our transition system” through three mechanisms (Zhou et al., 2021). The first is a pointer network over action history. Target actions are factored into bare action symbols and pointer values for edges, and one decoder self-attention head is repurposed as the pointer mechanism. Because the pointing occurs over the history of actions, decoder self-attention becomes the pointer network. That head is trained with an extra cross-entropy loss for pointer prediction.

The second mechanism is hard attention over source alignments. The monotonic source-to-action alignment induced by the cursor is encoded by masking cross-attention heads so that they can attend only to the aligned source word or words. Since BART operates on subwords, encoder outputs are average-pooled back to word-level states before being used for hard attention. This ties action generation directly to the cursor-defined alignment structure rather than leaving source access unconstrained.

The third mechanism is output-space masking. The decoder softmax is hard-masked so that only actions valid under the current transition state can be generated. This is the principal mechanism by which the sequence model is made structure-aware at decoding time.

The paper also states that it does not use the GNN-style step-wise graph embedding decoder from Zhou et al. (2021), because its gains were modest. A plausible implication is that the simplified transition system is intended to concentrate structural bias in the action space, attention pattern, and masking scheme rather than in an additional graph-state module.

5. Vocabulary strategies and structural guarantees

Vocabulary design is treated as a central component of making the transition system work with a pre-trained seq2seq model (Zhou et al., 2021). Two strategies are explored.

In the separate-vocabulary model (StructBART-S / sep-voc), the action vocabulary is separate from BART’s original subword vocabulary; the decoder input and output use stand-alone action embeddings; and those embeddings are initialized from the average of the corresponding BART subword embeddings. This cleanly separates action generation from text generation.

In the joint-vocabulary model (StructBART-J / joint-voc), source and target share a vocabulary; frequent node-creating actions are added as atomic vocabulary items; less frequent concepts are split into BART subwords; and non-node actions such as shift and copy are added as-is. If a node label is generated by multiple subword pieces, arc transitions always point to the beginning position of that node string. This avoids blind subword splitting of action symbols, which would otherwise break alignment and pointer bookkeeping.

The transition system preserves several explicit guarantees:

• Well-formed graphs: valid graphs are guaranteed without post-processing.
• Built-in alignments: because generation is monotonic and tied to the source cursor, every node is associated with a source token alignment.
• No unattachable nodes: this is achieved by removing subgraph actions.
• Full recovery under alignments: the oracle can recover the original graph exactly as long as every node has an alignment.
• Monotonicity: the cursor moves strictly left-to-right, and nodes are generated at the current token position before moving on.

The paper also addresses the fact that some AMR nodes are usually unaligned. To guarantee complete alignments, it uses a recursive heuristic: if a node is unaligned, copy the alignment from its first child if possible; otherwise copy from its first parent. Because the graph is assumed connected and at least some alignments exist, this recursive propagation assigns an alignment to all nodes (Zhou et al., 2021).

6. Empirical profile and interpretation

The empirical results are presented as evidence that simplifying the transition system does not hurt performance and can help BART exploit structure better (Zhou et al., 2021). The comparison reported in the paper is summarized below.

System	Base actions	Smatch oracle coverage / Avg. action length
Astudillo et al. (2020)	12	98.0 / 76.2
Zhou et al. (2021)	10	98.9 / 41.6
Proposed system	6	99.9 / 45.6

A common misconception is that removing special subgraph machinery necessarily reduces oracle recoverability. The reported comparison does not support that view: the simplified system removes special subgraph machinery entirely, yet achieves 99.9 Smatch oracle coverage, compared with 98.0 for Astudillo et al. and 98.9 for Zhou et al. It also remains competitive in average action sequence length, at 45.6 actions versus 41.6 for Zhou et al. and 76.2 for Astudillo et al.

The reported parsing results further strengthen that interpretation. On AMR 2.0, StructBART-J reaches 84.2 Smatch without extra silver data; with only 47K silver examples, it reaches 84.7, surpassing prior systems; and it also performs strongly on AMR 1.0 and AMR 3.0 (Zhou et al., 2021). The paper additionally notes that the same transition system does not help as much when used with the older APT model based on fixed RoBERTa features, but does help BART. This supports the claim that the simplified transition system is specifically well matched to a powerful seq2seq decoder.

The ablation findings are consistent with that interpretation. Hard attention over alignments is best; removing alignment modeling hurts; the joint-vocabulary setup benefits from BART’s subword generation, especially for rare concepts; and the copy action helps the separate-vocabulary model but is less critical in the joint-vocabulary model. Taken together, these results characterize the split-sequence style simplification as a transition system whose effectiveness depends not only on its reduced action set, but also on its coupling to pointer decoding, hard alignment control, and vocabulary design within BART (Zhou et al., 2021).