Structure Coherence Score (SCS)

Updated 12 July 2026

Structure Coherence Score (SCS) is an umbrella metric quantifying how well structured elements align across domains, including text, images, and narratives.
It is measured using explicit formulas, classifier-learned features, or aggregated sub-scores, guiding applications from fact verification to explanation quality.
Approaches range from Likert-scale assessments in human-AI explanations to unsupervised discourse modeling and graph-based metrics, highlighting domain-specific adaptations.

Searching arXiv for recent and relevant papers on “Structure Coherence Score (SCS)” and closely related uses of the acronym. “Structure Coherence Score” (SCS) is not a single standardized metric across the literature. Instead, the term designates several distinct scoring constructions that quantify coherence or structural consistency in different domains, including multi-modal fact verification, explanation quality, discourse coherence, narrative organization, image composition, and conditional generation. In some cases SCS is an explicit scalar with a stated formula, as in the “System Causability Scale” for explanation quality (Holzinger et al., 2019); in other cases it is an interpretive label for a structured feature set, such as the “structure coherence” representation used for claim–document alignment in multi-modal fact verification (Zhang et al., 2023). The common thread is that SCS-like measures do not treat coherence as a purely local similarity signal. They instead model relations among parts—sentences, topics, entities, images, explanations, or conditioning signals—and score how consistently those parts fit together.

1. Terminological scope and competing definitions

The acronym “SCS” has multiple established meanings in arXiv literature. The most explicit usage is the System Causability Scale, a 10-item, 5-point Likert questionnaire for measuring explanation quality in human–AI interfaces (Holzinger et al., 2019). A different line of work uses “structure coherence” as the organizing principle for multi-modal fact verification, but does not define a single scalar called SCS; instead, it concatenates feature groups and lets a classifier learn the decision boundary (Zhang et al., 2023). In image generation and structural fidelity, “Scene Composition Structure (SCS)” denotes the geometric arrangement of visual elements, and the corresponding scalar metric is the SCS Similarity Index Measure (SCSSIM) (Haque et al., 7 Aug 2025).

Text coherence research supplies further variants. One unsupervised discourse metric defines paragraph coherence as

$CS = \kappa \times E(T) \times \lambda(T),$

where $E(T)$ is the conditional entropy of topics with respect to the paragraph and $\lambda(T)$ is semantic relatedness among topics (Shrivastava et al., 2018). Another tradition models document coherence through bipartite sentence–entity graphs and derives scalar scores such as $\mathtt{bipDCC}$ , $\mathtt{bipACC}$ , and especially $\mathtt{bipLC}$ , which can plausibly be treated as structure coherence scores because they are computed directly from document structure (Lioma et al., 2016). More recent work on generated-text assessment formalizes coherence as the conjunction of cohesion, consistency, and relevance, and learns coherence scores from those structural conditions rather than introducing a single closed-form formula (Maimon et al., 2023).

This multiplicity implies that “Structure Coherence Score” is best understood as an umbrella notion rather than a canonically fixed metric. A plausible implication is that SCS functions less as a universally agreed benchmark than as a family of domain-specific scoring schemes for structural alignment.

In “INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification,” structure coherence is defined as alignment between a claim and a document across text and image modalities (Zhang et al., 2023). The paper states that structure coherence is reflected in four aspects: literal text similarity, text semantic similarity, text length, and image similarity. These are not collapsed into a single named scalar. Instead, they form a feature vector used by a Random Forest classifier.

The four feature groups are specified concretely. Text length is used as a scalar feature because “the authors do not specify whether they measure length in characters or tokens; they simply call it ‘text length’ and describe it as a scalar feature” (Zhang et al., 2023). Vocabulary similarity is extracted with ROUGE, motivated by the observation that “we observe a lot of vocabulary overlap in the category of refute” (Zhang et al., 2023). Semantic similarity uses Sentence-BERT cosine similarity as a baseline semantic coherence signal, while CLIP text embeddings are processed through an MLP that outputs a three-category result used as additional features (Zhang et al., 2023). Image similarity is computed primarily with ResNet50 cosine similarity between claim and document images (Zhang et al., 2023).

The full representation is the concatenation of length, ROUGE, Sentence-BERT, CLIP-MLP, and image-similarity features, after normalization because “the order of magnitude of text length and other features varies greatly” (Zhang et al., 2023). The normalized vector is then fed into a Random Forest classifier with max_depth = 40, producing one of five FACTIFY2 labels: Support_Text, Support_Multimodal, Insufficient_Text, Insufficient_Multimodal, or Refute (Zhang et al., 2023).

Empirically, the system achieved a weighted average F1 score of 0.8079, reaching 2nd place in FACTIFY2 (Zhang et al., 2023). The ablation study shows that this coherence decomposition is not ornamental. Validation F1 drops from 0.8078 to 0.6007 without ResNet50, to 0.7709 without ROUGE plus length, to 0.7926 without Sentence-BERT, and to 0.7911 without CLIP (Zhang et al., 2023). This suggests that in this setting “structure coherence” is a learned decision function over heterogeneous structural cues rather than a handcrafted scalar score.

3. Explanation quality and the System Causability Scale

The most explicit SCS formulation appears in “Measuring the Quality of Explanations: The System Causability Scale (SCS)” (Holzinger et al., 2019). Here SCS is a psychometric instrument for evaluating whether an explanation, explainable user interface, or explanation process is suitable for its intended purpose. It is grounded in causability, defined as “the extent to which an explanation of a statement to a user achieves a specified level of causal understanding with effectiveness, efficiency and satisfaction in a specified context of use” (Holzinger et al., 2019).

The scale consists of 10 items, each answered on a 5-point Likert scale from strongly disagree to strongly agree (Holzinger et al., 2019). The items cover causal factor completeness, contextual understandability, adjustable level of detail, independence from support, causal clarity, integration with the user’s knowledge base, internal consistency, learnability, sufficiency without external references, and efficiency or timeliness (Holzinger et al., 2019). Several of these items map directly onto structural coherence in explanations, particularly contextual understandability, integration with prior knowledge, and the absence of inconsistencies.

The scoring rule is explicit:

$\text{SCS} = \sum_{i=1}^{10} \frac{\text{Rating}_i}{50}.$

Because the total raw score lies between 10 and 50, SCS is normalized to the interval $[0.2, 1.0]$ (Holzinger et al., 2019). The paper does not define threshold bands such as “good” or “poor,” but reports an example on the Framingham Risk Tool with an overall reported SCS of 0.86 (Holzinger et al., 2019). The authors also caution that Likert scales are ordinal, so interval-style statistical treatment can be problematic (Holzinger et al., 2019).

This version of SCS is distinct from the feature-engineering usage in fact verification. It is explicitly user-centered, intended to assess explanation quality rather than model-internal coherence, and closely tied to causal understanding rather than semantic similarity. A plausible implication is that the System Causability Scale operationalizes structural coherence at the human-interface level: explanations are coherent when they are complete, consistent, appropriately detailed, and aligned with the user’s mental model.

4. Text and narrative coherence as structural scoring

In discourse modeling, several papers define scalar coherence scores from structural properties of text. One unsupervised approach measures topical coherence by first extracting paragraph topics and then combining topic uncertainty with topic relatedness:

$CS = \kappa \times E(T) \times \lambda(T).$

Here

$E(T) = -\sum_{i=1}^{N} p(t_i \mid P)\log p(t_i \mid P),$

and

$E(T)$ 0

This score is derived from a topic distribution over the paragraph and a WordNet subgraph built from topic words (Shrivastava et al., 2018). The method is fully unsupervised, relying on unlabeled corpora, GloVe, DBOW, WordNet, and TransE embeddings (Shrivastava et al., 2018). On evaluation, the resulting coherence scores correlate positively with gold labels, and human evaluation on synthetic data shows 79.3% agreement (Shrivastava et al., 2018).

A different structural tradition represents a document as a bipartite graph of sentences and discourse entities, rather than projecting to a one-mode graph. Three parameter-free coherence metrics are defined on that graph: $E(T)$ 1, $E(T)$ 2, and $E(T)$ 3 (Lioma et al., 2016). Of these, $E(T)$ 4 performs best, reaching 80.9 coherence accuracy on Earthquakes and 94.0 on Accidents, with 87.5 average, beating the compared baselines (Lioma et al., 2016). This line of work is important because it treats coherence as a structural property of entity continuity and linkage rather than a sentence-pair classification problem.

Recent coherence assessment for generated texts formalizes coherence in terms of cohesion, consistency, and relevance, following Reinhart’s definition (Maimon et al., 2023). Instead of giving a closed-form SCS formula, the paper constructs proxy tasks—sentence reordering, discourse-relation recognition, NP enrichment, natural language inference, and irrelevant-sentence recognition—and shows that joint training improves coherence scoring on GCDC and CoheSentia (Maimon et al., 2023). The best joint T5-large model reaches 76.4 accuracy on GCDC and 62.3 on CoheSentia, outperforming the reported baselines (Maimon et al., 2023). This suggests that structural coherence scoring in modern NLP increasingly emerges from multi-task supervision over structurally grounded subproblems.

The benchmark “CoheSentia” pushes this decomposition further by annotating coherence both holistically and incrementally (Maimon et al., 2023). Incremental annotation yields higher agreement than holistic scoring: paragraph-level ICC is 0.968 for incremental annotation versus 0.804 for holistic annotation (Maimon et al., 2023). The total number of local errors correlates with final coherence at -0.89, while cohesion, consistency, and relevance errors correlate at -0.81, -0.63, and -0.44, respectively (Maimon et al., 2023). This supports the view that structural coherence scores should be built from local relational judgments aggregated across a text rather than from a single opaque holistic classifier.

Narrative-specific work reaches a similar conclusion. “Skeleton-based Coherence Modeling in Narratives” proposes a learned similarity over sentence skeletons and shows that a document-level score can be computed by averaging adjacent-pair similarities:

$E(T)$ 5

However, sentence-level SSN models outperform skeleton-based models on all reported metrics, including 92.9 sentence-order accuracy for SSN-3 on sentences versus 84.2 on skeletons (Asnani et al., 2 Apr 2026). This indicates that structural abstraction alone does not guarantee a better coherence score.

5. Narrative, representation, and generation-oriented coherence scores

Narrative-generation research has introduced more composite scoring designs. “SCORE: Story Coherence and Retrieval Enhancement for AI Narratives” does not define a single scalar by default, but specifies component metrics for Consistency, Coherence, Item Status, and Complex Question accuracy (Yi et al., 30 Mar 2025). It also models item states

$E(T)$ 6

and defines narrative entropy

$E(T)$ 7

These ingredients motivate a composite structure score over state consistency, episodic continuity, and functional coherence (Yi et al., 30 Mar 2025). The ablation study shows that removing Dynamic State Tracking drops Consistency from 89.7% to 72.1% and Item accuracy from 98.3% to 61.2%, while removing Context Summary drops Coherence from 91.2% to 68.7% (Yi et al., 30 Mar 2025). This indicates that structural coherence in long narratives depends heavily on explicit state tracking and summarization.

A related multimodal narrative study defines a unified Narrative Coherence Score (NCS) over five dimensions: coreference, discourse relation diversity, topic switch, character persistence, and multimodal character grounding (Ilinykh et al., 26 Mar 2026). The two aggregation rules are

$E(T)$ 8

and

$E(T)$ 9

On visually grounded stories, humans score 0.50 arithmetic and 0.36 geometric under the short prompt, while GPT-4o scores 0.46 and 0.32, and Llama-4-Scout scores 0.32 and 0.06 (Ilinykh et al., 26 Mar 2026). The geometric form is notable because it penalizes imbalance across dimensions, a property highly relevant to any general “structure coherence score.”

Representation learning work provides yet another usage. “Statistical Coherence Alignment for LLM Representation Learning Through Tensor Field Convergence” introduces a coherence-alignment loss

$\lambda(T)$ 0

and proposes structure-level normalization of this loss into a scalar

$\lambda(T)$ 1

Reported results include Accuracy improvement from 82.3% to 88.7%, Perplexity reduction from 15.6 to 12.4, and Coherence Score increase from 0.72 to 0.85 (Gale et al., 13 Feb 2025). This is an internal representation-space notion of structural coherence rather than a human-judged discourse metric.

Conditional diffusion takes a complementary perspective. “Coherence-Aware Diffusion” assumes a sample-level scalar coherence score $\lambda(T)$ 2 and conditions the denoising model on both the condition $\lambda(T)$ 3 and coherence $\lambda(T)$ 4:

$\lambda(T)$ 5

The paper interprets low coherence as effectively unconditional conditioning under a coherence-consistent embedding definition (Dufour et al., 2024). While this is not called a “Structure Coherence Score,” it offers a clean operational role for SCS-like variables: a scalar measure of structural reliability that modulates how strongly a generative model should trust its conditioning signal.

In image analysis, “Scene Composition Structure (SCS)” denotes the spatial arrangement of regions, objects, orientations, and large-scale lines of a scene (Haque et al., 7 Aug 2025). The corresponding scalar similarity metric, SCSSIM, is based on hierarchical image partitioning via SSE reduction. For an image, the normalized cumulative gain curve is

$\lambda(T)$ 6

and the comparison between two images is summarized by

$\lambda(T)$ 7

followed by the symmetric similarity

$\lambda(T)$ 8

The metric is bounded in $\lambda(T)$ 9 and is explicitly designed to be invariant to non-compositional distortions while decreasing monotonically under compositional distortions (Haque et al., 7 Aug 2025).

The experimental examples make the intended semantics clear. On kodim16, a noisy image yields SCSSIM = 0.99, and a blurry image also yields SCSSIM = 0.99, whereas a 90° rotated version falls to about 0.09 (Haque et al., 7 Aug 2025). Different images with somewhat similar or opposite scene structure score around 0.32 and 0.10, respectively (Haque et al., 7 Aug 2025). This is a pure structural-fidelity interpretation of SCS: appearance changes that preserve layout barely affect the score, but structural rearrangements collapse it.

A distinct geometric usage appears in face alignment. The “Structure Coherence Component” models landmark relations with a dynamic sparse graph and improves WFLW NME from 5.95% in the baseline to 4.40% in the full model (Zhu et al., 2020). Although no explicit scalar SCS is defined, the work treats structure coherence as the preservation of consistent geometric relations among landmarks. This reinforces the broader pattern that SCS-like constructs emerge when prediction targets are relationally constrained.

7. Cross-domain themes, misconceptions, and open problems

Across domains, several shared design patterns recur. First, SCS-like measures almost always decompose coherence into multiple relational components rather than relying on raw similarity. In fact verification these are length, lexical overlap, semantics, and image similarity (Zhang et al., 2023); in explanation quality they are causal completeness, understandability, consistency, and related properties (Holzinger et al., 2019); in narrative assessment they include referential continuity, discourse relations, topic dynamics, and character grounding (Ilinykh et al., 26 Mar 2026). Second, many successful formulations either aggregate structured sub-scores explicitly, as in NCS or the topical coherence formula (Shrivastava et al., 2018), or learn a coherence function from structured features, as in multi-task coherence assessment (Maimon et al., 2023).

A common misconception is that coherence can be reduced to a single pairwise similarity value. Several papers argue against this directly. The FACTIFY2 system reports that replacing pre-trained encoders while still taking only one cosine score “does not improve the classification performance” because “the information contained in a score is limited” (Zhang et al., 2023). Topic-coherence work shows that pairwise associations can rate conceptually muddled word sets too highly, motivating subset-based structural coherence measures (Rosner et al., 2014). Narrative studies likewise find that apparently fluent model stories diverge from human narratives once several structural measures are considered jointly (Ilinykh et al., 26 Mar 2026).

Another misconception is that global coherence can be measured reliably without local analysis. CoheSentia shows higher agreement for incremental annotation than for holistic scoring (Maimon et al., 2023). This suggests that many future SCS designs will likely be hierarchical: local coherence decisions first, followed by aggregation into a document- or story-level score.

The main unresolved issue is standardization. “Structure Coherence Score” currently names no universal formula, no shared benchmark, and no common scale across tasks. Some formulations are human-centered and questionnaire-based (Holzinger et al., 2019); others are unsupervised symbolic scores (Shrivastava et al., 2018, Lioma et al., 2016); others are supervised learned predictors (Maimon et al., 2023, Liu et al., 2023); still others are analytic structural similarity metrics in vision (Haque et al., 7 Aug 2025). This suggests that SCS is best treated as a design pattern: a scalar or vector-valued measure derived from relations among structured units, whose exact form must be specified relative to the task, the representation, and the target notion of coherence.