Intra-Ticket Parsing: Implicit Discourse

• The paper introduces a two-stage pipeline that first identifies argument spans via BIO tagging and then classifies semantic sense using contextualized models.
• Intra-ticket parsing is the automated analysis of intra-sentential implicit discourse relations, crucial for uncovering hidden semantic connections in text.
• The approach integrates BiLSTM+CRF and BERT-based models enhanced with syntactic features to address challenges like span discontinuity and order ambiguity.

Intra-ticket parsing refers to the automated identification and interpretation of implicit relationships between spans within a single textual unit (“ticket”), with a particular focus on intra-sentential implicit discourse relations. This process requires both the localization of argument spans—commonly labeled Arg1 and Arg2—within a sentence and the assignment of a semantic sense to the relation between them, in the absence of an explicit connective. In current annotated resources such as PDTB-3, intra-sentential implicits have been robustly cataloged, greatly increasing the scope and complexity of discourse parsing relative to earlier resources focused on inter-sentential relations. The primary challenges arise from the free positioning, potential discontinuity, and sense ambiguity of arguments in intra-sentential implicits, making parsing a fundamentally more underdetermined task than when working across sentence boundaries.

1. Characteristics of Intra-Ticket (Intra-Sentential) Implicit Discourse Relations

Intra-sentential implicit discourse relations, as systematically annotated in PDTB-3, are not anchored at predictable locations such as sentence boundaries. Instead, the contiguous or non-contiguous spans corresponding to Arg1 and Arg2 may appear in either order and anywhere within the sentence. The majority of arguments are continuous spans, though a significant minority (approximately 4.2%) exhibit discontinuities, a property not directly modeled by most extant parsers.

Unlike inter-sentential implicits, for which argument order is typically left-to-right, intra-sentential implicits may instantiate either canonical or reverse ordering. Furthermore, the absence of explicit connectives increases the semantic gap to be bridged by computational models: sense determination must rely on contextual and structural cues rather than anchor expressions.

2. Pipeline Architecture for Intra-Ticket Parsing

The parsing of intra-sentential implicit relations is operationalized as a two-stage pipeline:

1. Argument Identification: The system must identify the positional boundaries of Arg1 and Arg2 within the tokenized sentence. This is framed as a sequence tagging problem using the BIO scheme (B-Arg1, I-Arg1, B-Arg2, I-Arg2, O), allowing for flexible encoding of spans, their order, and—in principle—support for discontinuities.
2. Sense Classification: Given the extracted argument spans, a separate component assigns the appropriate semantic sense (e.g., CONTINGENCY.PURPOSE, CONTINGENCY.CAUSE) using contextualized embeddings and optionally syntactic features.

This architecture addresses the dual challenges of span localization and semantic sense classification in the absence of explicit connectives.

3. Argument Identification via Sequence Tagging

The argument identification task operates over an input token sequence $S = \{t_1, t_2, \ldots, t_n\}$, with the model predicting a tag sequence $Y = \{y_1, y_2, \ldots, y_n\}$, $$y_j \in \{\text{B-Arg1}, \text{I-Arg1}, \text{B-Arg2}, \text{I-Arg2}, O\}$$. Two principal architectures are developed:

• BiLSTM+CRF Baseline: Tokens are represented using pre-trained GloVe embeddings ($e_i$). Contextualization is achieved via a bidirectional LSTM:

$$h_i = [\text{LSTM}_f(e_i, h_{i-1});\, \text{LSTM}_b(e_i, h_{i+1})]$$

Sequence decoding is performed by a CRF layer trained to maximize the conditional log likelihood over tag sequences, with loss

$L_Y = -\sum_{i} \sum_{j} \log p(y_j)$

• BERT-Based Models: Token representations are obtained from BERT in two modalities: (a) as fixed representations (frozen BERT) or (b) fully fine-tuned. Linearized syntactic parse features can be concatenated with BERT outputs to inject structural information, improving span boundary detection particularly for nuanced or ambiguous cases.

The training objective over the argument identifier parameters $w$ (including BERT or BiLSTM and CRF parameters) with input parse features $P^{(i)}$ is:

$$w^* = \underset{w}{\mathrm{argmax}} \sum_i \log p(Y^{(i)} \mid S^{(i)}, P^{(i)}; w)$$

4. Sense Classification and Integration of Syntactic Features

Given argument spans determined from the previous stage, sense classification is performed by a dedicated BERT model. The architectural input follows:

$$[\text{CLS}]\, \text{Arg1}\, [\text{SEP}]\, \text{Arg2}\, [\text{SEP}]$$

BERT computes a sequence representation and the embedding at the [CLS] position is used as the basis for a softmax classifier for Level-2 sense labels. Syntactic parse features, when present, are concatenated with the [CLS] vector to enrich the semantic input with structural cues, e.g., detecting clause-initial “to” phrases associated with purposes.

The cross-entropy loss for sense classification is:

$L_c = -\sum_i \log p(c_i)$

where $c_i$ is the gold label for the $i$-th instance.

5. Empirical Results and Evaluation Protocols

Model performance is evaluated using Precision, Recall, and F1 scores. Argument identification is considered correct only if the predicted span exactly matches the gold standard for Arg1 or Arg2. Results are reported under two regimes: a fixed 60/20/20 train/development/test split and 10-fold cross-validation to assess generalization.

Key findings include:

• The best fine-tuned BERT model with parse tree features attains F1 in the high 40s for Arg1 and high 50s for Arg2, surpassing baseline LSTM models with statistically significant improvements ($p < 0.05$).
• For sense classification, cross-validation accuracy reaches 69.54%, and test set accuracy reaches 75.19%. The drop in performance when using predicted (as opposed to gold) argument spans in the pipeline is relatively modest.
• Detailed analysis by sense label (including CONTINGENCY.CAUSE and CONTINGENCY.PURPOSE) demonstrates robust performance of BERT-based classifiers, including for relations with less canonical argument orderings.

6. Methodological Challenges and Remedial Strategies

Several intrinsic challenges are illuminated:

• Location Ambiguity: Argument spans are not bounded by regular sentence operators. The BIO tagging in combination with syntactic parse features helps to disambiguate argument boundaries.
• Order Ambiguity: Argument order is not fixed and often reversed from the canonical left-to-right. Sequence tagging without order assumptions, and additional classification experiments, demonstrate BERT’s relative strength in handling atypical orders.
• Sense Ambiguity: Without explicit connectives, sense classification defaults to reliance on lexical and syntactic cues; BERT’s contextualized representations, especially when enhanced with parse information, offer competitive robustness.

7. Limitations and Future Research Directions

Several open problems and next steps are identified:

• Handling multiple intra-sentential relations per sentence: The current approach duplicates sentences for independent processing rather than jointly modeling multiple relations.
• Modeling discontinuous argument spans: While most arguments are continuous, modeling discontinuity (present in $\sim 4.2\%$ of instances) is an outstanding limitation.
• Integrating linked explicit and implicit discourse relations remains unaddressed.
• Improving candidate argument selection in the pipeline via downstream likelihood scores offers a plausible improvement path.
• The outlined techniques and analytic framework have potential relevance for other intra-ticket parsing scenarios where extraction of implicit relations within a single text unit is required.

In summary, intra-ticket parsing, particularly for intra-sentential implicit discourse relations, is a fundamentally challenging task addressed by a two-step pipeline centered on (1) span identification using sequence tagging with contextual and syntactic signals and (2) BERT-based sense classification. The principled use of parse features and contextual LLMs leads to empirically validated performance gains, with remaining limitations centered around span structure and complex relation linking.