---
title: Structure Coherence Score (SCS)
url: https://www.emergentmind.com/topics/structure-coherence-score-scs
type: topic
---

# Structure Coherence Score (SCS)

Searching arXiv for recent and relevant papers on “Structure Coherence Score (SCS)” and closely related uses of the acronym.
“Structure Coherence Score” (SCS) is not a single standardized metric across the literature. Instead, the term designates several distinct scoring constructions that quantify coherence or structural consistency in different domains, including multi-modal fact verification, explanation quality, discourse coherence, narrative organization, image composition, and conditional generation. In some cases SCS is an explicit scalar with a stated formula, as in the “System Causability Scale” for explanation quality [1912.09024]; in other cases it is an interpretive label for a structured feature set, such as the “structure coherence” representation used for claim–document alignment in multi-modal fact verification [2303.01510]. The common thread is that SCS-like measures do not treat coherence as a purely local similarity signal. They instead model relations among parts—sentences, topics, entities, images, explanations, or conditioning signals—and score how consistently those parts fit together.

## 1. Terminological scope and competing definitions

The acronym “SCS” has multiple established meanings in arXiv literature. The most explicit usage is the **System Causability Scale**, a 10-item, 5-point Likert questionnaire for measuring explanation quality in human–AI interfaces [1912.09024]. A different line of work uses “structure coherence” as the organizing principle for multi-modal fact verification, but does **not** define a single scalar called SCS; instead, it concatenates feature groups and lets a classifier learn the decision boundary [2303.01510]. In image generation and structural fidelity, “Scene Composition Structure (SCS)” denotes the geometric arrangement of visual elements, and the corresponding scalar metric is the **SCS Similarity Index Measure (SCSSIM)** [2508.05037].

Text coherence research supplies further variants. One unsupervised discourse metric defines paragraph coherence as
$$
CS = \kappa \times E(T) \times \lambda(T),
$$
where $E(T)$ is the conditional entropy of topics with respect to the paragraph and $\lambda(T)$ is semantic relatedness among topics [1809.00410]. Another tradition models document coherence through bipartite sentence–entity graphs and derives scalar scores such as $\mathtt{bipDCC}$, $\mathtt{bipACC}$, and especially $\mathtt{bipLC}$, which can plausibly be treated as structure coherence scores because they are computed directly from document structure [1608.00758]. More recent work on generated-text assessment formalizes coherence as the conjunction of cohesion, consistency, and relevance, and learns coherence scores from those structural conditions rather than introducing a single closed-form formula [2310.00598].

This multiplicity implies that “Structure Coherence Score” is best understood as an umbrella notion rather than a canonically fixed metric. A plausible implication is that SCS functions less as a universally agreed benchmark than as a family of domain-specific scoring schemes for structural alignment.

## 2. Multi-modal fact verification as structure coherence

In “INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification,” structure coherence is defined as alignment between a **claim** and a **document** across text and image modalities [2303.01510]. The paper states that structure coherence is reflected in four aspects: **literal text similarity, text semantic similarity, text length, and image similarity**. These are not collapsed into a single named scalar. Instead, they form a feature vector used by a Random Forest classifier.

The four feature groups are specified concretely. **Text length** is used as a scalar feature because “the authors do not specify whether they measure length in characters or tokens; they simply call it ‘text length’ and describe it as a scalar feature” [2303.01510]. **Vocabulary similarity** is extracted with ROUGE, motivated by the observation that “we observe a lot of vocabulary overlap in the category of refute” [2303.01510]. **Semantic similarity** uses Sentence-BERT cosine similarity as a baseline semantic coherence signal, while CLIP text embeddings are processed through an MLP that outputs a three-category result used as additional features [2303.01510]. **Image similarity** is computed primarily with ResNet50 cosine similarity between claim and document images [2303.01510].

The full representation is the concatenation of length, ROUGE, Sentence-BERT, CLIP-MLP, and image-similarity features, after normalization because “the order of magnitude of text length and other features varies greatly” [2303.01510]. The normalized vector is then fed into a Random Forest classifier with `max_depth = 40`, producing one of five FACTIFY2 labels: Support_Text, Support_Multimodal, Insufficient_Text, Insufficient_Multimodal, or Refute [2303.01510].

Empirically, the system achieved a **weighted average F1 score of 0.8079**, reaching **2nd place** in FACTIFY2 [2303.01510]. The ablation study shows that this coherence decomposition is not ornamental. Validation F1 drops from **0.8078** to **0.6007** without ResNet50, to **0.7709** without ROUGE plus length, to **0.7926** without Sentence-BERT, and to **0.7911** without CLIP [2303.01510]. This suggests that in this setting “structure coherence” is a learned decision function over heterogeneous structural cues rather than a handcrafted scalar score.

## 3. Explanation quality and the System Causability Scale

The most explicit SCS formulation appears in “Measuring the Quality of Explanations: The System Causability Scale (SCS)” [1912.09024]. Here SCS is a psychometric instrument for evaluating whether an explanation, explainable user interface, or explanation process is suitable for its intended purpose. It is grounded in **causability**, defined as “the extent to which an explanation of a statement to a user achieves a specified level of causal understanding with effectiveness, efficiency and satisfaction in a specified context of use” [1912.09024].

The scale consists of **10 items**, each answered on a **5-point Likert scale** from strongly disagree to strongly agree [1912.09024]. The items cover causal factor completeness, contextual understandability, adjustable level of detail, independence from support, causal clarity, integration with the user’s knowledge base, internal consistency, learnability, sufficiency without external references, and efficiency or timeliness [1912.09024]. Several of these items map directly onto structural coherence in explanations, particularly contextual understandability, integration with prior knowledge, and the absence of inconsistencies.

The scoring rule is explicit:
$$
\text{SCS} = \sum_{i=1}^{10} \frac{\text{Rating}_i}{50}.
$$
Because the total raw score lies between 10 and 50, SCS is normalized to the interval $[0.2, 1.0]$ [1912.09024]. The paper does not define threshold bands such as “good” or “poor,” but reports an example on the Framingham Risk Tool with an overall reported SCS of **0.86** [1912.09024]. The authors also caution that Likert scales are ordinal, so interval-style statistical treatment can be problematic [1912.09024].

This version of SCS is distinct from the feature-engineering usage in fact verification. It is explicitly user-centered, intended to assess explanation quality rather than model-internal coherence, and closely tied to causal understanding rather than semantic similarity. A plausible implication is that the System Causability Scale operationalizes structural coherence at the human-interface level: explanations are coherent when they are complete, consistent, appropriately detailed, and aligned with the user’s mental model.

## 4. Text and narrative coherence as structural scoring

In discourse modeling, several papers define scalar coherence scores from structural properties of text. One unsupervised approach measures **topical coherence** by first extracting paragraph topics and then combining topic uncertainty with topic relatedness:
$$
CS = \kappa \times E(T) \times \lambda(T).
$$
Here
$$
E(T) = -\sum_{i=1}^{N} p(t_i \mid P)\log p(t_i \mid P),
$$
and
$$
\lambda(T) = \frac{\text{NodeSim}(T)\cdot \text{ND}(T)}{\text{TC}(T)\cdot \text{ED}(T)}.
$$
This score is derived from a topic distribution over the paragraph and a WordNet subgraph built from topic words [1809.00410]. The method is fully unsupervised, relying on unlabeled corpora, GloVe, DBOW, WordNet, and TransE embeddings [1809.00410]. On evaluation, the resulting coherence scores correlate positively with gold labels, and human evaluation on synthetic data shows **79.3% agreement** [1809.00410].

A different structural tradition represents a document as a **bipartite graph** of sentences and discourse entities, rather than projecting to a one-mode graph. Three parameter-free coherence metrics are defined on that graph: $\mathtt{bipDCC}$, $\mathtt{bipACC}$, and $\mathtt{bipLC}$ [1608.00758]. Of these, $\mathtt{bipLC}$ performs best, reaching **80.9** coherence accuracy on Earthquakes and **94.0** on Accidents, with **87.5** average, beating the compared baselines [1608.00758]. This line of work is important because it treats coherence as a structural property of entity continuity and linkage rather than a sentence-pair classification problem.

Recent coherence assessment for generated texts formalizes coherence in terms of **cohesion, consistency, and relevance**, following Reinhart’s definition [2310.00598]. Instead of giving a closed-form SCS formula, the paper constructs proxy tasks—sentence reordering, discourse-relation recognition, NP enrichment, natural language inference, and irrelevant-sentence recognition—and shows that joint training improves coherence scoring on GCDC and CoheSentia [2310.00598]. The best joint T5-large model reaches **76.4** accuracy on GCDC and **62.3** on CoheSentia, outperforming the reported baselines [2310.00598]. This suggests that structural coherence scoring in modern NLP increasingly emerges from multi-task supervision over structurally grounded subproblems.

The benchmark “CoheSentia” pushes this decomposition further by annotating coherence both **holistically** and **incrementally** [2310.16329]. Incremental annotation yields higher agreement than holistic scoring: paragraph-level ICC is **0.968** for incremental annotation versus **0.804** for holistic annotation [2310.16329]. The total number of local errors correlates with final coherence at **-0.89**, while cohesion, consistency, and relevance errors correlate at **-0.81**, **-0.63**, and **-0.44**, respectively [2310.16329]. This supports the view that structural coherence scores should be built from local relational judgments aggregated across a text rather than from a single opaque holistic classifier.

Narrative-specific work reaches a similar conclusion. “Skeleton-based Coherence Modeling in Narratives” proposes a learned similarity over sentence skeletons and shows that a document-level score can be computed by averaging adjacent-pair similarities:
$$
\text{SCS}(d) = \frac{1}{N-1}\sum_{i=1}^{N-1} C_{\text{struct}}(k_i, k_{i+1}).
$$
However, sentence-level SSN models outperform skeleton-based models on all reported metrics, including **92.9** sentence-order accuracy for SSN-3 on sentences versus **84.2** on skeletons [2604.02451]. This indicates that structural abstraction alone does not guarantee a better coherence score.

## 5. Narrative, representation, and generation-oriented coherence scores

Narrative-generation research has introduced more composite scoring designs. “SCORE: Story Coherence and Retrieval Enhancement for AI Narratives” does not define a single scalar by default, but specifies component metrics for **Consistency**, **Coherence**, **Item Status**, and **Complex Question accuracy** [2503.23512]. It also models item states
$$
S_i(t) \in \{\text{active}, \text{lost}, \text{destroyed}\},
$$
and defines narrative entropy
$$
E(S_0, S_1, \dots, S_T)= - \sum_{k=0}^{T-1} \ln \Bigl(P\bigl(S_{k+1} \mid S_k\bigr)\Bigr).
$$
These ingredients motivate a composite structure score over state consistency, episodic continuity, and functional coherence [2503.23512]. The ablation study shows that removing Dynamic State Tracking drops Consistency from **89.7%** to **72.1%** and Item accuracy from **98.3%** to **61.2%**, while removing Context Summary drops Coherence from **91.2%** to **68.7%** [2503.23512]. This indicates that structural coherence in long narratives depends heavily on explicit state tracking and summarization.

A related multimodal narrative study defines a unified **Narrative Coherence Score (NCS)** over five dimensions: coreference, discourse relation diversity, topic switch, character persistence, and multimodal character grounding [2603.25537]. The two aggregation rules are
$$
\mathrm{NCS}^{\mathrm{arith}} = \frac{1}{5}(R + D + T + \mathrm{ChP} + \mathrm{MCG})
$$
and
$$
\mathrm{NCS}^{\mathrm{geom}} = \sqrt[5]{R \cdot D \cdot T \cdot \mathrm{ChP} \cdot \mathrm{MCG}}.
$$
On visually grounded stories, humans score **0.50** arithmetic and **0.36** geometric under the short prompt, while GPT-4o scores **0.46** and **0.32**, and Llama-4-Scout scores **0.32** and **0.06** [2603.25537]. The geometric form is notable because it penalizes imbalance across dimensions, a property highly relevant to any general “structure coherence score.”

Representation learning work provides yet another usage. “Statistical Coherence Alignment for Large Language Model Representation Learning Through Tensor Field Convergence” introduces a coherence-alignment loss
$$
\mathcal{L}_{\text{SCA}} \approx \sum_{i=1}^{n} \left\| \mathbf{T}_i - \mathbb{E}[\mathbf{T}] \right\|_F^2
$$
and proposes structure-level normalization of this loss into a scalar
$$
\text{SCS}(X) = 1 - \frac{1}{Z |X|} \sum_{i \in X} \left\|\mathbf{T}_i - \mathbb{E}[\mathbf{T}] \right\|_F^2.
$$
Reported results include **Accuracy** improvement from **82.3%** to **88.7%**, **Perplexity** reduction from **15.6** to **12.4**, and **Coherence Score** increase from **0.72** to **0.85** [2502.09815]. This is an internal representation-space notion of structural coherence rather than a human-judged discourse metric.

Conditional diffusion takes a complementary perspective. “Coherence-Aware Diffusion” assumes a sample-level scalar coherence score $c \in [0,1]$ and conditions the denoising model on both the condition $y$ and coherence $c$:
$$
L_\text{simple}
= \mathbb{E}_{(X,y,c),\, t} \left[\left\lVert \epsilon - \epsilon_{\theta}(X_t, y, c, t)\right\rVert\right].
$$
The paper interprets low coherence as effectively unconditional conditioning under a coherence-consistent embedding definition [2405.20324]. While this is not called a “Structure Coherence Score,” it offers a clean operational role for SCS-like variables: a scalar measure of structural reliability that modulates how strongly a generative model should trust its conditioning signal.

## 6. Visual and geometric structure: scene composition and related limits

In image analysis, “Scene Composition Structure (SCS)” denotes the spatial arrangement of regions, objects, orientations, and large-scale lines of a scene [2508.05037]. The corresponding scalar similarity metric, **SCSSIM**, is based on hierarchical image partitioning via SSE reduction. For an image, the normalized cumulative gain curve is
$$
c_i = \frac{1}{e}\sum_{j=1}^{i} \hat{g}_j,
$$
and the comparison between two images is summarized by
$$
\mathrm{M}(I, I_0) = e^{-\lambda \bar{k}^2},
$$
followed by the symmetric similarity
$$
\mathrm{SCSSIM}(I, I_0) = \frac{1}{2} \left( \mathrm{M}(I, I_0) + \mathrm{M}(I_0, I) \right).
$$
The metric is bounded in $[0,1]$ and is explicitly designed to be invariant to non-compositional distortions while decreasing monotonically under compositional distortions [2508.05037].

The experimental examples make the intended semantics clear. On `kodim16`, a noisy image yields **SCSSIM = 0.99**, and a blurry image also yields **SCSSIM = 0.99**, whereas a 90° rotated version falls to about **0.09** [2508.05037]. Different images with somewhat similar or opposite scene structure score around **0.32** and **0.10**, respectively [2508.05037]. This is a pure structural-fidelity interpretation of SCS: appearance changes that preserve layout barely affect the score, but structural rearrangements collapse it.

A distinct geometric usage appears in face alignment. The “Structure Coherence Component” models landmark relations with a dynamic sparse graph and improves WFLW NME from **5.95%** in the baseline to **4.40%** in the full model [2006.11697]. Although no explicit scalar SCS is defined, the work treats structure coherence as the preservation of consistent geometric relations among landmarks. This reinforces the broader pattern that SCS-like constructs emerge when prediction targets are relationally constrained.

## 7. Cross-domain themes, misconceptions, and open problems

Across domains, several shared design patterns recur. First, SCS-like measures almost always decompose coherence into multiple relational components rather than relying on raw similarity. In fact verification these are length, lexical overlap, semantics, and image similarity [2303.01510]; in explanation quality they are causal completeness, understandability, consistency, and related properties [1912.09024]; in narrative assessment they include referential continuity, discourse relations, topic dynamics, and character grounding [2603.25537]. Second, many successful formulations either aggregate structured sub-scores explicitly, as in NCS or the topical coherence formula [1809.00410], or learn a coherence function from structured features, as in multi-task coherence assessment [2310.00598].

A common misconception is that coherence can be reduced to a single pairwise similarity value. Several papers argue against this directly. The FACTIFY2 system reports that replacing pre-trained encoders while still taking only one cosine score “does not improve the classification performance” because “the information contained in a score is limited” [2303.01510]. Topic-coherence work shows that pairwise associations can rate conceptually muddled word sets too highly, motivating subset-based structural coherence measures [1403.6397]. Narrative studies likewise find that apparently fluent model stories diverge from human narratives once several structural measures are considered jointly [2603.25537].

Another misconception is that global coherence can be measured reliably without local analysis. CoheSentia shows higher agreement for incremental annotation than for holistic scoring [2310.16329]. This suggests that many future SCS designs will likely be hierarchical: local coherence decisions first, followed by aggregation into a document- or story-level score.

The main unresolved issue is standardization. “Structure Coherence Score” currently names no universal formula, no shared benchmark, and no common scale across tasks. Some formulations are human-centered and questionnaire-based [1912.09024]; others are unsupervised symbolic scores [1809.00410; 1608.00758]; others are supervised learned predictors [2310.00598; 2306.06472]; still others are analytic structural similarity metrics in vision [2508.05037]. This suggests that SCS is best treated as a design pattern: a scalar or vector-valued measure derived from relations among structured units, whose exact form must be specified relative to the task, the representation, and the target notion of coherence.

Source: https://www.emergentmind.com/topics/structure-coherence-score-scs