Effective-Information Consistency Score

• EICS is a dimensionless metric that defines consistency by comparing transformations against inherent reference variability in a given data domain.
• In remote sensing, it evaluates whether channel augmentations preserve natural spectral variability based on nearest-neighbor temporal deviations.
• In transformer circuits, it combines a normalized effective information proxy with sheaf inconsistency to assess internal coherence and causal integration.

Searching arXiv for the specified papers to ground the article and confirm citation metadata. Effective-Information Consistency Score (EICS) designates a consistency-oriented scoring concept that, in current arXiv usage, appears in two technically distinct settings. In remote sensing, the designation is applied in effect to a time-series-based score for assessing whether channel-based data augmentation preserves physically plausible spectral variation, although the paper’s primary notation is $S_{\text{noaug}}$ and $S_{\text{aug}}$. In mechanistic interpretability, EICS is explicitly defined as a dimensionless, single-pass score for Transformer Circuits that combines a normalized sheaf inconsistency term with a Gaussian effective-information proxy. Despite the shared name, the two constructions formalize different notions of “consistency”: alignment with natural temporal spectral variability in Earth-observation data, and coherence plus causal integration within an identified transformer subgraph (Burgert et al., 2024, Krasnovsky, 8 Sep 2025).

1. Terminological scope and research motivations

The remote-sensing formulation arises from an open debate about channel transformations in deep-learning pipelines for remote sensing image classification. Standard augmentation operators from computer vision—such as brightness, contrast, grayscale, solarization, Gaussian blur, posterization, sharpness, and noise—can alter pixel values that, in remote sensing, are interpreted as spectral signatures tied to land cover and acquisition physics. The central question is therefore not whether augmentation changes pixel values, but whether the induced change is larger than the deviations already observed naturally for the same location over time. The proposed score addresses this by estimating whether an augmented signature remains within the range of natural, time-dependent deviations observed in a multi-temporal image series (Burgert et al., 2024).

The Transformer-Circuit formulation is motivated by uncertainty quantification in mechanistic interpretability. There, a Transformer Circuit is a functional subgraph hypothesized to implement a task such as induction, copying, or factual recall. The paper argues that an active circuit can nonetheless be internally inconsistent or causally unreliable on a particular prompt. EICS is introduced as a white-box, single-pass score intended to indicate whether the mechanism itself is behaving coherently and integratively, rather than relying only on output-level confidence or black-box calibration (Krasnovsky, 8 Sep 2025).

These two uses of EICS share a structural intuition: both define consistency relative to a reference system internal to the object under study. In one case, the reference system is the temporal neighborhood of the same spatial location; in the other, it is the agreement structure induced by local Jacobians and the macro-versus-parts effective-information comparison. This suggests that the common term “consistency” is methodological rather than domain-specific.

2. Remote-sensing EICS as a physical-consistency score

In the remote-sensing setting, the score estimates the expected deviation of a pixel signature from the closest other signature in the same time series. A channel augmentation is treated as physically consistent when the augmented signature remains within the range of natural deviations already observed for the same location over time. The construction depends on a collection of image time series, each associated with a homogeneous $k\times k$ region assumed to be stable over time (Burgert et al., 2024).

Let

$\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$

be a set of $N$ time series, with

$$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$

Each series has a binary mask

$\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$

marking the homogeneous region. The signature-extraction operator is

$$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$

which averages the pixel values over the selected region. For a channel augmentation operator $\mathrm{aug}(\cdot)$, the augmented image is

$$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$

The per-signature deviation is defined as the normalized $S_{\text{aug}}$0-distance to the nearest other timestamp in the same time series: $S_{\text{aug}}$1 The expected deviation for unaugmented signatures is

$S_{\text{aug}}$2

and for stochastic augmentations,

$S_{\text{aug}}$3

Several design choices are central. The $S_{\text{aug}}$4 distance is divided by $S_{\text{aug}}$5, so the score is normalized per channel. Signatures are computed from a homogeneous $S_{\text{aug}}$6 area to reduce errors from residual spatial misalignment. Each signature is compared to the closest other signature within the same time series, via $S_{\text{aug}}$7, so the score measures whether an augmented signature resembles some naturally observed signature of that location over time. For stochastic operators, multiple repetitions are used to estimate $S_{\text{aug}}$8.

The interpretation is explicitly relative. Lower values indicate that augmented signatures remain close to naturally occurring signatures and are therefore more physically consistent. Higher values indicate that the augmentation introduces spectral changes exceeding natural temporal variability. The paper does not define a hard threshold in the formula itself; instead, it compares $S_{\text{aug}}$9 to the standard deviation band of $k\times k$0. If $k\times k$1 stays within that band, the augmentation is treated as physically consistent; if it exceeds the band, it is considered physically inconsistent.

3. Practical workflow, empirical validation, and limitations in remote sensing

The practical computation has three steps. First, a collection of time series is built for the same spatial locations; for optical data, cloudy images are removed. Second, a homogeneous $k\times k$2 region is manually selected in each series, with the requirement that it not be affected by land-cover change. Third, signatures are computed and nearest-neighbor temporal deviations are evaluated by comparing each unaugmented or augmented signature to the nearest unaugmented signature from a different timestamp in the same series, then averaging over all series, timestamps, and augmentation repetitions (Burgert et al., 2024).

The score is validated on the BigEarthNet-S2 dataset, using a Lithuania split for training and Ireland splits for validation and test. The training set contains 28,226 images, and the validation and test sets contain 12,013 images each. To compute the scores, 70 images were randomly selected from the training set, their corresponding time series between 2017 and 2020 were downloaded, cases with one present class were retained so that the selected region belongs to the same class, and time series of 70 to 100 cloud-free images were obtained. The included classes were arable land, broad-leaved forest, coniferous forest, inland waters, marine waters, pastures, and urban fabric.

The evaluated channel augmentations were brightness, contrast, Gaussian blur, Gaussian noise, grayscale, posterize, sharpness, and solarize. The experiments followed standard augmentation settings from the AutoAugment-style literature: augmentation probability $k\times k$3, magnitude range subdivided into 20 bins, maximum magnitude $k\times k$4, and $k\times k$5 repetitions for each augmentation magnitude. Because Sentinel-2 data are originally in uint16, but some augmentations require uint8, a preprocessing scheme was used in which values are divided by the 99th percentile per channel, clipped to $k\times k$6, multiplied by 255, and then mapped back with a pseudo-inverse to interpret scores in the original uint16 space.

A classification experiment connected physical consistency to model utility. A ResNet18 was trained on the multi-label classification task with batch size 512, AdamW, cosine annealing, learning rates between $k\times k$7 and $k\times k$8, 30 epochs, and 5 seeds, with test performance measured by macro mAP. For each augmentation and magnitude, $k\times k$9 was compared to the no-augmentation baseline $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$0.

The main findings are sharply asymmetric. Contrast and sharpness are physically consistent, with $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$1 within the standard deviation of $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$2, and they almost always improve baseline performance. Gaussian blur and Gaussian noise are also physically consistent and can improve performance, although the best magnitude is not monotonic. Posterize and solarize are physically consistent but do not improve performance. Brightness and grayscale become problematic: grayscale consistently exceeds the deviation range strongly, and brightness becomes inconsistent from about maximum magnitude 6 onward, roughly corresponding to $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$3 in the normalized setting. These augmentations do not improve the baseline. The resulting conclusion is that physical inconsistency is a strong indicator that an augmentation will not help, whereas physical consistency is not a sufficient condition for improved training performance.

The stated limitations are substantive. The method relies on manually selected homogeneous regions, assumes sufficiently rich time series to represent natural variability, is evaluated on one dataset and one task, focuses on channel rather than geometric augmentations, and is tied to the chosen distance metric and stable-area selection.

4. Transformer-circuit EICS: formal construction

In mechanistic interpretability, EICS is defined for a circuit $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$4, its observed activations $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$5, and edge Jacobians $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$6 computed at the current forward state as

$\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$7

The numerator is a normalized emergence proxy, and the denominator penalizes sheaf inconsistency. High EICS is intended to indicate a circuit that is both coherent and integrative, and therefore more likely to be trustworthy on the current input (Krasnovsky, 8 Sep 2025).

The sheaf component models the circuit as a cellular sheaf on the undirected version of the circuit graph. Each node $\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$8 has stalk

$\mathcal{D} = \{\mathbf{t}_1,\dots,\mathbf{t}_N\}$9

and each directed edge $N$0 has restriction map

$N$1

For a 0-cochain $N$2, the coboundary is

$N$3

Rather than using a quotient norm in cohomology, the paper defines a normalized inconsistency energy directly from the observed activations: $N$4 The numerator aggregates edgewise mismatch between locally predicted downstream activations and actual downstream activations; the denominator normalizes by activation magnitude. The term is dimensionless and is $N$5 when activations form a noisily consistent global section. The paper notes that the raw activations $N$6 may optionally be replaced by a least-squares projection $N$7, but the main definition uses the directly observed activations.

The effective-information component is a Gaussian, single-pass proxy derived from a local linearization. At the current forward state, the circuit is approximated by

$N$8

For this linear Gaussian channel,

$N$9

The Gaussian EI proxy is then

$$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$0

with $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$1 acting as an SNR-like scale parameter.

For a circuit $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$2, let $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$3 be the macro-Jacobian from circuit inputs to outputs, and let $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$4 be the Jacobian associated with node $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$5. The emergence signal is

$$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$6

followed by the positive part

$$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$7

and the normalized form

$$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$8

This normalization renders the emergence term dimensionless and confines it to $$\mathbf{t}_i = (\mathbf{t}_{i,1}, \dots, \mathbf{t}_{i,T}), \qquad \mathbf{t}_{i,\tau} \in \mathbb{R}^{C \times H \times W}.$$9. The full score is therefore dimensionless. The paper also gives the small-$\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$0 approximation

$\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$1

which is used for fast ranking mode.

5. Computation, interpretation, and caveats in mechanistic interpretability

The computation is explicitly designed to require one forward pass plus Jacobian-vector products, without input Monte Carlo. Given a model $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$2, an input $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$3, a circuit $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$4, and a scale $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$5, the procedure is: run the forward pass and record activations $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$6; compute edge Jacobians $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$7 using VJP/JVP; evaluate $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$8; construct $\mathbf{b}_i \in \{0,1\}^{C\times H\times W}$9 and the node Jacobians $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$0; compute $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$1; and return

$$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$2

For the sheaf term, a node-seeded JVP scheme is described, with one JVP per source node computing all outgoing terms $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$3 (Krasnovsky, 8 Sep 2025).

Two computational modes are distinguished. Fast mode uses the small-$$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$4 approximation together with Hutch++ or Lanczos log-det estimators, with suggested probe counts of 4–8 per $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$5 and 8–12 for $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$6, and is intended for ranking and screening. Exact mode is intended for smaller blocks and computes $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$7 via Cholesky or SVD. The stated complexity claims are that $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$8 can be computed with $$\mathrm{sig}(\mathbf{x}, \mathbf{b}) \coloneqq \frac{\mathbf{x}\cdot \mathbf{b}}{\|\mathbf{b}\|_1},$$9 JVPs via node-seeded evaluation, fast EI mode typically costs about 50–200 JVP/VJP products on the restricted subgraph, roughly 2–6 forward-equivalents, and exact mode may cost about 5–15 forward-equivalents.

Interpretation is direct but component-sensitive. High EICS corresponds to a large normalized emergence term and low sheaf inconsistency; it is therefore interpreted as coherent, trustworthy circuit behavior. Low EICS indicates low emergence and/or high internal disagreement, suggesting incoherent computation, weak causal integration, or possible failure or hallucination. The paper recommends inspecting the two components separately—$\mathrm{aug}(\cdot)$0 and $\mathrm{aug}(\cdot)$1—because they diagnose different failure modes.

The paper includes a toy sanity check using a 6-node feedforward circuit with linear blocks and additive Gaussian edge noise. Under the small-$\mathrm{aug}(\cdot)$2 approximation, increasing additive noise reduces EI and increases sheaf residuals, so EICS should decrease as noise increases. An appendix simulation with two parallel branches reports exactly this qualitative behavior: as node-noise $\mathrm{aug}(\cdot)$3 increases, $\mathrm{aug}(\cdot)$4 rises, $\mathrm{aug}(\cdot)$5 falls, branch alignment weakens, the emergence proxy decreases, and EICS decreases accordingly.

The limitations are explicit. EICS is only as meaningful as the chosen circuit $\mathrm{aug}(\cdot)$6; if the circuit decomposition is wrong, the score may be misleading. Both the sheaf term and the EI proxy rely on Jacobian linearization around the observed state, so the method provides a local approximation rather than a global causal guarantee. The paper also notes sensitivity to weights and connectivity, discusses the sheaf Hodge Laplacian $\mathrm{aug}(\cdot)$7 and its spectral gap $\mathrm{aug}(\cdot)$8, and recommends reporting $\mathrm{aug}(\cdot)$9, normalizing edge weights by operator norms, and possibly regularizing $$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$0 if $$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$1 is near zero. Empirical validation on LLM tasks is deferred; the paper outlines a protocol using factual-retrieval circuits, verifiable question sets, and adversarial or hallucination-inducing prompts, but does not claim those experiments have already been run.

6. Comparative interpretation and recurrent misconceptions

The two EICS constructions are mathematically distinct and should not be conflated. In remote sensing, the relevant objects are spectral signatures extracted from homogeneous spatial regions in image time series, and the principal quantities are $$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$2 and $$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$3, both based on normalized nearest-neighbor temporal $$\widetilde{\mathbf{t}_{i,\tau}} = \mathrm{aug}(\mathbf{t}_{i,\tau}).$$4 deviation. In mechanistic interpretability, the relevant objects are node activations, local Jacobians, and macro- versus part-level Jacobian maps within a Transformer Circuit, and the score is a ratio of normalized emergence to one plus sheaf inconsistency. A plausible implication is that the shared label reflects an analogous epistemic role—testing whether a transformation or mechanism remains aligned with an admissible internal reference class—rather than a shared derivation.

Several misconceptions are directly contradicted by the two papers. In the remote-sensing case, the critique is not that any channel augmentation is invalid because it changes pixel values; the operative criterion is whether the change exceeds natural variability already present in time series of the same location. Likewise, physical consistency is not presented as a sufficient condition for model improvement: posterize and solarize remain within the natural deviation range yet do not outperform the no-augmentation baseline (Burgert et al., 2024).

In the transformer-circuit case, EICS is not presented as a completed empirical benchmark for hallucination detection. It is proposed as a white-box uncertainty indicator grounded in internal mechanism, but validation on LLM tasks is explicitly deferred. Nor is the score reducible to output confidence; its construction is based on local Jacobian agreement and effective-information emergence rather than output probabilities alone (Krasnovsky, 8 Sep 2025).

Taken together, the two formulations show that “consistency” can be operationalized at very different levels of analysis. One scores whether augmented observations stay within naturally realized physical variability; the other scores whether an identified computational subgraph exhibits low internal disagreement and positive macro-level causal emergence. This suggests that EICS is best understood not as a single universal metric, but as a recurring design pattern for constructing dimensionless or normalized criteria that compare a perturbation or mechanism against a domain-specific notion of admissible structure.