SSPC: Sequential Sentence Pair Classifier

• The paper introduces a sequential sentence pair classifier that predicts style boundaries by comparing contextualized embeddings from adjacent sentences.
• It leverages document-level context through a frozen pre-trained encoder and multi-layer BiLSTM, enabling robust detection despite short or boilerplate text.
• Empirical results on PAN 2025 data across EASY, MEDIUM, and HARD settings demonstrate SSPC’s effective adaptation to diverse document structures.

Searching arXiv for the SSPC paper and closely related sentence-sequence and sentence-pair modeling work. {"queries":[{"query":"(Schmidt et al., 1 Aug 2025) Sequential Sentence Pair Classifier style change detection"},{"query":"(Cohan et al., 2019) sequential sentence classification BERT"},{"query":"(Jin et al., 2018) hierarchical sequential sentence classification medical abstracts"},{"query":"(Barkan et al., 2019) Distilled Sentence Embedding sentence pair"},{"query":"(Lam et al., 2024) LSTM-based sequential sentence classification medical abstracts"},{"query":"(Patro et al., 2018) sequential pair-wise discriminator sentence embeddings"}]} Sequential Sentence Pair Classifier (SSPC) denotes a sequence-aware boundary classification architecture introduced for sentence-level style change detection in the PAN 2025 shared task. In this formulation, a document is treated as an ordered sequence of sentences, and the predictive target is defined over adjacencies: for each transition between sentence $i$ and sentence $i+1$, the model predicts whether a style shift occurs. The architecture combines sentence representations from a pre-trained LLM (PLM), bidirectional recurrent contextualization over the full sentence sequence, pair construction by concatenating adjacent contextualized sentence vectors, and a multi-layer perceptron (MLP) for binary boundary prediction. Its principal motivation is that many benchmark sentences are short, duplicated, boilerplate, or even punctuation-only, so isolated sentence-level cues are often insufficient; sequential context is therefore treated as essential evidence (Schmidt et al., 1 Aug 2025).

1. Problem formulation and task scope

SSPC was proposed in the setting of style change detection for multi-author writing style analysis at PAN 2025. The task is to identify style changes in documents at sentence-level granularity. Operationally, the document is modeled as a sequence of sentences, and the system predicts, for each adjacent sentence transition, whether a style boundary is present. The benchmark is partitioned into three difficulty settings: EASY, where documents always cover multiple topics; MEDIUM, where documents have limited thematic diversity; and HARD, where documents are single-topic and topical cues are largely removed (Schmidt et al., 1 Aug 2025).

This task formulation is explicitly relational rather than sentence-intrinsic. A sentence is not treated as stylistically diagnostic in isolation; instead, style difference is inferred relative to neighboring sentences. The PAN 2025 setting intensifies this requirement because the data contains relatively short sentences, over 10% exact duplicate sentences, repeated boilerplate or moderation sentences, and many punctuation-only “sentences” due to sentence segmentation conventions. These properties weaken the evidential value of individual sentences and make adjacency prediction a more direct objective than independent sentence classification (Schmidt et al., 1 Aug 2025).

A common misconception is to equate SSPC with generic sentence-pair classification in the sense of paraphrase, textual similarity, or natural language inference. That characterization is too broad. In SSPC, the pair is not an arbitrary sentence pair sampled from a corpus; it is a local transition embedded in an ordered document sequence, and the model is designed to exploit both local discontinuity and global context. This distinguishes SSPC from sentence-pair scorers whose central concern is efficient pairwise semantic matching rather than ordered sequence modeling. A clear contrast is provided by Distilled Sentence Embedding (DSE), which addresses scoring or classifying a pair of sentences $(A,B)$ through distilled sentence representations and a lightweight similarity function, but is explicitly described as not a sequential classifier over a stream of sentence pairs (Barkan et al., 2019).

2. Architectural definition

The SSPC architecture is a four-stage pipeline: sentence encoding with a pre-trained LLM, BiLSTM contextualization over the sentence sequence, concatenation of adjacent contextual sentence vectors, and MLP classification for boundary prediction (Schmidt et al., 1 Aug 2025).

At the first stage, each sentence is passed through a PLM to obtain token embeddings. The paper states that “Applying straightforward mean pooling to token embeddings, fixed-length representations of the problem’s sentences are obtained.” If sentence $s_i$ has token embeddings $t_{i1}, t_{i2}, \dots, t_{in}$, the sentence representation is effectively

$x_i = \frac{1}{n}\sum_{j=1}^{n} t_{ij}.$

The submitted system uses StyleDistance/styledistance as the encoder. Several encoders were tested, including RoBERTa-base, XLM-RoBERTa-base, all-MiniLM-L6-v2, and StyleDistance/styledistance, with StyleDistance selected for the final submission. The transformer is frozen during training, so the PLM functions as a fixed feature extractor rather than an end-to-end trainable encoder (Schmidt et al., 1 Aug 2025).

At the second stage, the sentence vectors $x_1, x_2, \dots, x_T$ are processed by a bidirectional LSTM. In standard notation,

$$\overrightarrow{h}_i = \mathrm{LSTM}_f(x_i, \overrightarrow{h}_{i-1}), \qquad \overleftarrow{h}_i = \mathrm{LSTM}_b(x_i, \overleftarrow{h}_{i+1}),$$

with final contextual representation

$$h_i = [\overrightarrow{h}_i ; \overleftarrow{h}_i].$$

The BiLSTM is described as producing “context-aware sentence representations enriching raw mean-pooled vectors with information from the sentences across the entire problem.” The architecture uses 5 BiLSTM layers and 0.2 dropout (Schmidt et al., 1 Aug 2025).

At the third stage, adjacent sentence representations are converted into boundary candidates. For each possible boundary between sentence $i$ and sentence $i+1$0, the model forms

$i+1$1

that is, concatenation of the two contextual vectors across feature dimensions. This pairwise representation preserves both order and sentence-specific information while inheriting document-level context from the BiLSTM (Schmidt et al., 1 Aug 2025).

At the final stage, $i+1$2 is passed to a three-layer feedforward network with linear layers, GELU activations, and dropout, producing a scalar logit for binary style-boundary prediction. The training loss is Binary Cross-Entropy (BCE). For gold label $i+1$3 and predicted probability $i+1$4,

$i+1$5

The overall architecture is therefore best described as a sequence-aware adjacent-pair classifier rather than a sentence-in-isolation classifier or a global document segmentation model without local pair formation (Schmidt et al., 1 Aug 2025).

3. Training regime and empirical results

The SSPC system was implemented in PyTorch Lightning. To increase training data, the authors incorporated PAN 2024 datasets and converted paragraph-level data into sentence-level boundary labels: transitions inside a paragraph were labeled 0, and the first sentence of a new paragraph was labeled 1. A single model was trained over all three sentence-level datasets from PAN 2025 together with all three paragraph-turned-sentence-level datasets from PAN 2024 (Schmidt et al., 1 Aug 2025).

The hyperparameter configuration reported for the final system is concise and conservative. The Base transformer is StyleDistance/styledistance; Transformer frozen is True; Pooling is Mean; BiLSTM layers are 5; BiLSTM dropout is 0.2; MLP is a three-layer feedforward network; MLP dropout is 0.2; Batch size is 4; Learning rate is 0.0005; Minimal learning rate is 0.00005; Scheduler is cosine; Training steps are 30000; Warmup steps are 2600; and Loss is Binary Cross-Entropy (Schmidt et al., 1 Aug 2025).

The official PAN 2025 test results are reported in macro-F1. The main results narrative gives 0.923 on EASY, 0.828 on MEDIUM, and 0.724 on HARD. The paper also reports a comparison against claude-3.7-sonnet: Claude obtains 0.856 / 0.818 / 0.661 on easy / medium / hard, whereas SSPC obtains 0.929 / 0.815 / 0.731 in one table. The text explicitly notes a minor discrepancy between the abstract and the table for the medium and hard scores, and identifies the safest “official test” summary from the main results narrative as 0.923, 0.828, and 0.724, which also matches the abstract (Schmidt et al., 1 Aug 2025).

The reported internal backbone comparison further contextualizes the encoder choice.

Model	Easy / Medium / Hard	Easy + 2024 / Medium + 2024 / Hard + 2024
RoBERTa-base	0.929 / 0.764 / 0.613	0.930 / 0.787 / 0.656
XLM-RoBERTa-base	0.927 / 0.779 / 0.540	0.929 / 0.779 / 0.658
all-MiniLM-L6-v2	0.878 / 0.763 / 0.607	0.906 / 0.777 / 0.654
StyleDistance	0.891 / 0.780 / 0.629	0.922 / 0.828 / 0.723

This comparison shows that StyleDistance is the best-performing backbone, especially when augmented with 2024 data, where the hard setting reaches 0.723 in the internal evaluation table (Schmidt et al., 1 Aug 2025).

A plausible implication is that the gains are due not only to local adjacency comparison but also to the interaction between a stylistically oriented frozen encoder and full-document contextualization. The paper nevertheless characterizes the system as relatively conservative and lightweight rather than architecturally elaborate (Schmidt et al., 1 Aug 2025).

4. Conceptual relation to sequential sentence classification

SSPC belongs to the broader family of sequence-sensitive sentence modeling methods, but its predictive unit is the adjacency rather than the sentence. The closest antecedents in the supplied literature come from Sequential Sentence Classification (SSC) in scientific abstracts, where the goal is to assign each sentence a rhetorical role while exploiting neighboring context (Jin et al., 2018, Cohan et al., 2019, Lam et al., 2024).

A foundational formulation is the Hierarchical Sequential Labeling Network (HSLN) for medical scientific abstracts. HSLN is organized into four stages: word embedding layer, sentence encoding layer, context enriching layer, and label sequence optimization layer. Each sentence is encoded from words, then the sequence of sentence vectors is passed through an abstract-level bi-LSTM, and a final CRF jointly optimizes the label sequence. The paper argues that sentence labels are not independent, because abstracts often follow a regular rhetorical progression such as Background → Objective → Methods → Results → Conclusion, and because similar sentences can receive different labels depending on context. Its best models improve weighted F1 by 2%–3% absolute over prior methods, reaching 92.6 on PubMed 20k, 93.9 on PubMed 200k, and 84.7 or 84.3 on NICTA-PIBOSO, depending on encoder variant (Jin et al., 2018).

The 2019 BERT-based work on sequential sentence classification replaces hierarchical encoders and CRFs with joint encoding of the entire sentence sequence. Sentences are concatenated into a single BERT input,

$i+1$6

and the contextualized $i+1$7 representations are used for classification. The authors report that a CRF is not helpful empirically, likely because self-attention already captures relevant contextual information. Their model achieves state-of-the-art micro F1 on PubMed (92.9), CSAbstruct (83.1), and NICTA (84.8) (Cohan et al., 2019).

A later LSTM-based approach again emphasizes sentence representation quality. Its Sen-Model combines GloVe word embeddings, randomly initialized character embeddings, stacked Bi-LSTM encoders with scaled dot-product attention, sentence statistics, and a BiomedBERT-based PubMed embedding; the resulting sentence embedding is then used by an abstract-level C-RNN and a segment-level MLP. The reported improvements over the bi-ANN baseline are +1.0 F1 on PubMed 200K, +2.8 F1 on PubMed 20K, and +2.6 F1 on NICTA-PIBOSO, with the combined model reaching 92.8 F1 on PubMed 20K and 85.3 F1 on NICTA-PIBOSO (Lam et al., 2024).

These SSC models clarify what SSPC inherits and what it changes. Like SSC systems, SSPC assumes that local units cannot be interpreted independently. Unlike SSC systems, its primary target is not a label per sentence but a binary decision per adjacency. This suggests that SSPC can be understood as an “adjacency-labeled” specialization of sequence-aware sentence modeling: it retains document order and contextual dependency modeling, but it moves the prediction locus from nodes to edges in the sentence sequence.

5. Relation to sentence-pair modeling and pairwise discriminators

SSPC also connects to work on sentence-pair modeling, but it differs materially from sentence-pair scorers and embedding models that are not sequential in the document sense. The most direct contrast is with Distilled Sentence Embedding (DSE). DSE addresses scoring or classifying a pair of sentences $i+1$8 for tasks such as semantic textual similarity, paraphrase, natural language inference, and question matching. It uses a teacher–student distillation setup in which a fine-tuned cross-attentive teacher, specifically BERT-Large in the main experiments, supervises a sentence-embedding student. The student is defined as

$i+1$9

and in practice the model uses a symmetric Siamese student with shared embedding function $(A,B)$0. The sentence encoder applies BERT-Large to each sentence individually, takes average pooling over hidden tokens in each of the last four encoder layers, and concatenates them into a 4096-dimensional sentence embedding. The pair score is then computed by

$(A,B)$1

where

$(A,B)$2

DSE achieves 82.83 average score on five GLUE sentence-pair tasks with 4.6% relative degradation versus BERT-Large, while providing large speedups such as 934× faster for offline 1M pair similarities and about 13.5K× faster for online query-versus-100K-candidates similarity computation (Barkan et al., 2019).

The relevance of DSE to SSPC is therefore limited and specific. Both operate on sentence pairs, but their modeling assumptions diverge. DSE is explicitly described as a sentence-pair scorer/ranker, a representation-learning approach for sentence-pair classification, and a distilled Siamese sentence embedding model, not a sequential classifier. SSPC, by contrast, depends on sentence order, contextualized sentence trajectories, and adjacency-level labeling inside a document (Barkan et al., 2019, Schmidt et al., 1 Aug 2025).

Another related but distinct line is “Learning Semantic Sentence Embeddings using Sequential Pair-wise Discriminator”, which augments a seq2seq paraphrase generator with a shared-weight pairwise discriminator. The architecture contains an Encoder LSTM, a Decoder LSTM, and a Pair-wise discriminator LSTM, with shared encoder–discriminator parameters. Its training objective combines a token-level cross-entropy local loss with a global margin-based ranking loss,

$(A,B)$3

The method is pairwise and sequential, but not a standalone sentence-pair classifier. Instead, it is a pairwise semantic discriminator embedded in a generation framework, intended to improve paraphrase generation and transfer sentence embeddings to sentiment analysis (Patro et al., 2018).

A plausible implication is that SSPC occupies an intermediate position between these traditions. It is more local and explicitly pair-centered than sentence-sequence classifiers, yet more order-sensitive and document-conditioned than ordinary sentence-pair classification or embedding distillation. That hybrid position explains both its suitability for style boundary detection and its limited interchangeability with non-sequential pairwise models.

6. Strengths, limitations, and interpretive issues

The SSPC paper identifies several strengths. First, the model is well matched to the PAN 2025 challenge of stylistically shallow, short sentences: by using context, it can make decisions even when a target sentence is itself weak evidence. Second, the BiLSTM injects sequential information into each sentence representation, which is useful when sentence-level style cues are weak, transitions depend on neighboring context, or repeated boilerplate sentences appear. Third, the system is lightweight and practical because the PLM is frozen and the classifier is relatively small. Fourth, it generalizes across EASY, MEDIUM, and HARD, with especially strong performance on HARD, where topic cues are least helpful (Schmidt et al., 1 Aug 2025).

The limitations are equally explicit. A central concern is that the model may exploit macrostructure or contextual regularities more than pure stylistic features. In that sense, success on the benchmark may partly reflect learned document continuity and sequence structure rather than deep authorial style representation. The authors also note that short, content-free sentences remain intrinsically difficult even with context; that a BiLSTM is effective but not necessarily optimal for long or complex sequences; that a frozen encoder limits task adaptation; and that padding for batch processing may hurt scalability and efficiency on much longer or highly variable inputs (Schmidt et al., 1 Aug 2025).

These concerns resonate with the broader literature. In HSLN, the largest performance drop in ablation studies comes from removing the context enriching layer, indicating that contextual sentence representations already capture much of the sequential structure even before CRF decoding. On PubMed 20k, for example, HSLN-RNN drops from 92.6 to 90.0 without the context enriching layer, whereas removing the sequence optimization layer leaves performance at 92.3. This suggests that sequential context can dominate explicit label-transition modeling (Jin et al., 2018). Similarly, the BERT joint-encoding model reports that a CRF is not empirically helpful, implying that contextual self-attention may absorb dependencies previously handled by structured decoders (Cohan et al., 2019).

A plausible implication is that SSPC’s effectiveness may arise less from the specific adjacent-pair classifier than from contextualized sentence representations learned over the whole document. That interpretation does not diminish the model’s utility, but it does clarify what SSPC is actually measuring: not merely pairwise stylistic difference, but pairwise difference as filtered through sequential context. For evaluation and future use, this distinction matters whenever “style change” can be confounded with topic drift, discourse phase, or recurring document templates (Schmidt et al., 1 Aug 2025).

7. Position within the literature

Within the supplied literature, SSPC is most accurately classified as a sequence-aware boundary classifier for sentence-level style change detection. Its defining recipe is stable and compact: encode each sentence with a frozen PLM, contextualize the entire sentence sequence with a BiLSTM, concatenate adjacent contextual sentence vectors, and classify each adjacency with an MLP under Binary Cross-Entropy (Schmidt et al., 1 Aug 2025).

Its nearest neighbors are not generic pairwise similarity models but sequence models that reject sentence-level independence assumptions. SSC work in biomedical and scientific abstracts established the importance of neighboring sentence semantics, sentence order, and abstract-level structure, whether through hierarchical bi-LSTM + CRF pipelines or through joint Transformer encoding of full sentence sequences (Jin et al., 2018, Cohan et al., 2019, Lam et al., 2024). SSPC transfers that intuition from rhetorical role labeling to style boundary detection and changes the supervised unit from sentence labels to adjacency labels.

At the same time, SSPC is not reducible to the sentence-pair modeling literature. DSE shows how to replace expensive cross-attention with precomputed sentence embeddings and a cheap similarity function for pair scoring, but that paradigm is explicitly different from sequential sentence-pair classification because it does not classify sentence pairs sequentially and does not use document-level order as part of the model definition (Barkan et al., 2019). Likewise, the sequential pair-wise discriminator for paraphrase generation is pairwise and sequential only in the sense of seq2seq training and batchwise ranking; it is not a boundary classifier over ordered sentence transitions (Patro et al., 2018).

Accordingly, SSPC is best understood as a specialized synthesis. It inherits the contextual dependency emphasis of sequential sentence classification, the local comparative focus of sentence-pair modeling, and the practical preference for lightweight architectures under noisy data conditions. Its core contribution is to frame style change detection as a context-sensitive adjacency prediction problem rather than as sentence-in-isolation classification or unconstrained sentence-pair scoring.