StructCoh: Structured Contrastive Learning

• StructCoh is a structured contrastive learning framework that employs dual graphs and hierarchical objectives to align semantic and structural features of texts.
• It integrates syntactic dependency graphs with topic interaction graphs using cross-graph attention to capture fine-grained semantic distinctions.
• Empirical results show superior performance in legal statute matching, academic plagiarism detection, and robustness against text perturbations.

StructCoh is a structured contrastive learning framework for context-aware text semantic matching. In its named usage, the term denotes the model introduced in "StructCoh: Structured Contrastive Learning for Context-Aware Text Semantic Matching," which formulates semantic matching as structured representation alignment by combining a dual-graph encoder with hierarchical contrastive learning (Xue et al., 2 Sep 2025). The framework is designed for domains such as law and academic writing, where semantic equivalence depends not only on lexical overlap but also on internal argumentative and logical structure. In the supplied literature, the same label also appears in expository senses connected to structural cohomology and to structure constants and coherence; those usages are distinct from the text-matching framework and are noted separately below (Finegan, 2023, Burciu, 2020).

1. Problem setting and conceptual motivation

StructCoh addresses text semantic matching in settings where two texts are semantically equivalent, or closely related, only when their internal structures also align. The task is formulated as: given two texts $A$ and $B$, estimate a similarity score $S(A,B)$ indicating how semantically and structurally equivalent they are. The motivating applications explicitly include legal statute matching, legal case retrieval, and academic plagiarism detection (Xue et al., 2 Sep 2025).

The framework is motivated by two limitations. First, BERT-like pre-trained LLMs encode texts as token sequences and may implicitly capture some structure through self-attention, but they do not explicitly represent dependency relations and higher-level structure, nor do they enforce structure alignment across documents. Second, standard contrastive learning methods such as SimCSE typically operate at the sentence or document level only, with positives often obtained from the same sentence under dropout or simple augmentation. This improves global embedding quality, but it does not model node-wise or substructure alignment between texts and does not exploit task-specific structural negatives (Xue et al., 2 Sep 2025).

Within this problem formulation, false positives arise when texts share many keywords but differ in core semantics, while false negatives arise when surface wording diverges but argumentative structure is preserved. StructCoh is therefore designed to preserve fine-grained semantic distinctions while explicitly modeling hierarchical structural patterns. A common misconception is that it is merely a syntax-aware encoder; the framework is instead built around both syntactic and topic-level structure, and it couples that structure to contrastive objectives at multiple granularities (Xue et al., 2 Sep 2025).

2. Dual-graph encoder and representation construction

For each text $T$, StructCoh constructs two graphs. The first is a syntactic dependency graph,

$G^S_T = (V^S_T, E^S_T),$

obtained via dependency parsing. Its nodes are tokens or short phrases, and its directed edges encode syntactic relations such as subject, object, and modifier. Node initialization is defined as

$$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$

where $\mathbf{e}_i^{\text{BERT}}$ is the contextual token embedding from a PLM, $\mathbf{e}_i^{\text{pos}}$ is a positional encoding, and $\mathbf{e}_i^{\text{type}}$ is a POS-tag embedding. Each edge $(i,j)$ additionally carries a relation embedding $B$0 (Xue et al., 2 Sep 2025).

The second graph is a topic or semantic interaction graph,

$B$1

which captures coarser semantic structure through named entities and latent topic clusters extracted by NER and LDA or similar topic modeling. A topic/entity node $B$2 is initialized by mention aggregation: $B$3 and edge weights may be constructed from co-occurrence statistics,

$B$4

This graph is intended to capture which entities or topics are prominent and how they interact, thereby supporting alignment of argument structure across documents (Xue et al., 2 Sep 2025).

Both graphs are encoded with Graph Isomorphism Networks. For a general graph $B$5, a GIN layer is written as

$B$6

with separate application to the syntactic graph and the topic graph, producing $B$7 and $B$8 (Xue et al., 2 Sep 2025).

The two graph views are fused by cross-graph attention. For each syntactic node $B$9, StructCoh computes

$S(A,B)$0

and then fuses the representations as

$S(A,B)$1

The resulting fused syntactic nodes are pooled into a document representation by self-attention pooling: $S(A,B)$2 This $S(A,B)$3 is the structure-aware, topic-enhanced document representation used for downstream contrastive learning and final similarity scoring (Xue et al., 2 Sep 2025).

3. Hierarchical contrastive objective and matching mechanism

StructCoh trains these representations with a hierarchical contrastive objective consisting of a node-level term and a graph-level term. The node-level contrastive regularization is defined over aligned node pairs $S(A,B)$4 between two matched texts $S(A,B)$5 and $S(A,B)$6. For each aligned pair $S(A,B)$7, it uses an InfoNCE-style objective,

$S(A,B)$8

where only $S(A,B)$9 is positive and the remaining candidates act as negatives. The alignments are obtained via string matching, entity match, or heuristic role alignment, such as the same statute ID, the same named entity, or the same surface form (Xue et al., 2 Sep 2025).

At the document level, StructCoh applies a graph-level contrastive objective to align whole-document structure-aware embeddings and repel non-matching texts: $T$0 Negatives include both explicit negatives, which are labeled non-matching pairs, and implicit negatives, including adversarially similar examples in the batch. Hard negative mining is emphasized through

$T$1

or equivalently by selecting the top-$T$2 most similar negatives (Xue et al., 2 Sep 2025).

The total objective is

$T$3

The description supplied for StructCoh states that there is no explicit cross-entropy classification term in the formulas given; the matching task can instead be solved directly by similarity thresholds or retrieval ranking. After training, the similarity between texts is computed by cosine similarity,

$T$4

which can be used for retrieval, ranking, or threshold-based binary matching (Xue et al., 2 Sep 2025).

4. Datasets, baselines, and empirical results

The reported experiments cover specialized-domain datasets, general-domain benchmarks, and robustness benchmarks. The specialized-domain datasets are COLIEE-2023 for legal statute matching, CaseLaw for legal case retrieval, and SPD-1.0 for academic plagiarism detection. The general-domain evaluation uses MRPC, QQP, STS-B, MNLI, QNLI, and RTE. Robustness is evaluated on TextFlint perturbation sets including SwapAnt, NumWord, AddSent, InsertAdv, Appendlrr, AddPunc, BackTrans, TwitterType, SwapNamedEnt, and SwapSyn-WordNet (Xue et al., 2 Sep 2025).

Dataset	Task	Reported StructCoh result
COLIEE-2023	Legal statute matching	F1 $T$5 86.7%
CaseLaw	Legal case retrieval	F1 $T$6 84.5%
SPD-1.0	Academic plagiarism detection	Accuracy $T$7 82.4%

On COLIEE-2023, StructCoh achieves 86.7% F1-score, with a $T$8 absolute F1 gain over the previous best Coref-GNN. On CaseLaw, it reaches 84.5% F1 and outperforms TextGCN at approximately 79.7% and HGAT at approximately 80.8%. On SPD-1.0, it attains 82.4% accuracy, which is reported as a $T$9 absolute gain over BERT-based baselines. These results are explicitly attributed to improved capture of legal argument structures, cause-effect relationships, and structural equivalence under heavy paraphrasing (Xue et al., 2 Sep 2025).

On GLUE, StructCoh-base reports an average of 85.59, compared with 83.83 for BERT-base, 85.20 for SyntaxBERT-base, and 84.90 for SemBERT. Its selected scores are MRPC 89.1, QQP 91.2, STS-B 88.3, MNLI-m/mm 84.8/84.7, QNLI 91.5, and RTE 69.6. StructCoh-large reports an average of 88.16, compared with 85.80 for BERT-large and 87.26 for SyntaxBERT-large; its selected scores are MRPC 91.4, QQP 91.8, STS-B 89.6, MNLI-m/mm 87.2/87.0, QNLI 94.9, and RTE 75.4 (Xue et al., 2 Sep 2025).

The robustness results indicate materially stronger resistance to perturbations. On Quora, StructCoh scores 60.43 on SwapAnt, 62.76 on NumWord, and 87.49 on BackTrans, compared with lower scores reported for BERT and SyntaxBERT on the same perturbations. On SNLI, it reports 81.06 on AddSent, 96.85 on SwapAnt, and 60.58 on SwapNamedEnt. On MNLI-m/mm, it reports 60.14/59.25 on AddSent, 60.89/61.37 on SwapAnt, and 82.15/82.97 on SwapSyn, the last being described as competitive with UERBERT (Xue et al., 2 Sep 2025).

5. Ablations, case studies, implementation, and limitations

The ablation study on COLIEE-2023 isolates the contribution of each major component. Full StructCoh is reported at 86.7% accuracy/F1. Removing hierarchical contrastive learning reduces F1 to 82.2, a drop of 4.5 points. Removing the graph fusion layer reduces F1 to 80.5, a drop of 6.2 points. Removing dependency graph encoding reduces F1 to 78.1, a drop of 8.6 points. Removing the topic interaction graph reduces F1 to 76.9, a drop of 9.8 points (Xue et al., 2 Sep 2025).

These ablations indicate that both graph views and the fusion mechanism materially contribute to performance. A plausible implication is that the topic or entity interaction graph is especially important on COLIEE-2023, since its removal produces the largest drop. The reported pattern also shows that StructCoh is not reducible to a single mechanism: the contrastive objective, cross-graph fusion, syntactic structure, and topic-level structure each contribute independently (Xue et al., 2 Sep 2025).

The case studies emphasize sensitivity to subtle semantic changes. Examples include “this girl loves me” versus “this boy loves me,” and “12 digits number” versus “42 digits number.” Baseline models often classify these as paraphrases, while StructCoh assigns low similarity scores in the range of 11–20% and correctly labels them as non-matches. The explanation provided is that the number change or entity change manifests in specific nodes whose representations no longer align under node-level contrast, while graph context highlights that those nodes play crucial roles in the proposition (Xue et al., 2 Sep 2025).

The implementation details reported in the description use BERT as the base PLM, with base and large variants, BERT tokenization, Stanford CoreNLP for dependency parsing, NER for entities, LDA for topics, Adam optimization, mini-batch training, dropout, layer normalization, and early stopping on validation metrics. Graph construction is pre-processed to save training time. The hidden dimension $G^S_T = (V^S_T, E^S_T),$0, the number of GIN layers $G^S_T = (V^S_T, E^S_T),$1, the temperatures $G^S_T = (V^S_T, E^S_T),$2 and $G^S_T = (V^S_T, E^S_T),$3, the loss weights $G^S_T = (V^S_T, E^S_T),$4 and $G^S_T = (V^S_T, E^S_T),$5, and the hard negative threshold $G^S_T = (V^S_T, E^S_T),$6 or top-$G^S_T = (V^S_T, E^S_T),$7 selection are all treated as tunable hyperparameters (Xue et al., 2 Sep 2025).

The stated limitations are dependence on parsing and topic modeling quality, computational overhead from dependency parsing, LDA, and NER, and domain specificity. Legal texts are described as long and complex, making parsing challenging, and performance on very short or structurally simple texts may show smaller gains. Suggested future directions include end-to-end differentiable or jointly learned structural induction, more sophisticated inter-document graph alignment beyond pairwise contrast, and extension to multi-document reasoning and multi-hop QA (Xue et al., 2 Sep 2025).

6. Terminological scope and distinct uses of the label

Although StructCoh is the explicit name of the graph-enhanced text matching framework of (Xue et al., 2 Sep 2025), the supplied corpus also contains other, unrelated uses of the label. In one algebraic-geometric exposition linked to "A Multitor Formula for a Non-lci Intersection," “StructCoh” is taken to mean “the systematic study of the behavior of cohomology of structure sheaves under intersection.” In that usage, the emphasis is on the cohomology sheaves of

$G^S_T = (V^S_T, E^S_T),$8

and on explicit formulas for

$G^S_T = (V^S_T, E^S_T),$9

when a non-lci intersection decomposes as $$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$0 with $$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$1 of codimension $$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$2 and $$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$3 of arbitrary codimension (Finegan, 2023).

A second expository use appears in connection with "Structure constants for pre-modular categories," where “StructCoh” designates “Structure Constants and Coherence for Premodular Categories.” There the central objects are conjugacy classes, conjugacy class sums, and structure constants

$$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$4

together with a Burnside-type formula and the statement that for a premodular category, the structure constants multiplied by $$\mathbf{x}_i^{(0)}= \left[\mathbf{e}_i^{\text{BERT}} \,\|\, \mathbf{e}_i^{\text{pos}} \,\|\, \mathbf{e}_i^{\text{type}}\right],$$5 are algebraic integers, and for a weakly integral premodular category they are non-negative integers (Burciu, 2020).

These usages are conceptually unrelated. One concerns context-aware text semantic matching via dual graphs and hierarchical contrastive learning; the others concern structural cohomology in algebraic geometry and structure constants in fusion-category theory. The shared label therefore functions as a terminological coincidence rather than a unified research program across the cited works (Xue et al., 2 Sep 2025, Finegan, 2023, Burciu, 2020).