GCAV: Global Concept Activation Vector

• GCAV is an interpretability framework that aggregates and aligns layer-wise concept activation vectors into a single global representation.
• It employs a three-stage process with autoencoder compression, contrastive alignment, and attention-based fusion to reconcile cross-layer inconsistencies.
• Empirical findings show that GCAV reduces variance in concept sensitivity and enhances qualitative localization compared to traditional TCAV.

Global Concept Activation Vector (GCAV) denotes a line of concept-based interpretability in which human-defined concepts are represented not merely as layer-specific concept directions, but as globally reusable or globally aggregated objects for analysis across many samples, classes, or layers. In the most explicit usage, GCAV refers to a framework that unifies independently learned layer-wise Concept Activation Vectors (CAVs) into a single semantically consistent representation and applies a TCAV-style test to the reconstructed layer-specific vectors (He et al., 28 Aug 2025). More broadly, the term also sits within the CAV/TCAV tradition, where a concept is encoded as a direction in latent space and its influence is aggregated over a dataset or class, even when the term “GCAV” is not itself used (Kim et al., 2017, Yeh et al., 2022).

1. Foundations in CAV and TCAV

GCAV is rooted in the CAV formalism introduced for concept-based explanations of neural networks. In that framework, a network is probed at a chosen hidden layer $l$, and a concept $C$ is represented by a vector $\mathbf{v}_C^l$ learned from examples of the concept and examples outside the concept. The vector is typically the normal of a linear decision boundary in activation space (Kim et al., 2017). The local quantity of interest is the directional derivative of a class score along that concept direction: $$S_{C,k,l}(\mathbf{x}) = \nabla h_{l,k}(f_l(\mathbf{x})) \cdot \mathbf{v}_C^l.$$ This measures whether moving infinitesimally in the concept direction increases the logit for class $k$ at input $\mathbf{x}$ (Kim et al., 2017).

TCAV turns that local sensitivity into a class-level statistic by aggregating signs of directional derivatives over a class-specific dataset. The standard score is

$$\mathrm{TCAV}_{C,k,l} = \frac{\left| \left\{ \mathbf{x} \in X_k : S_{C,k,l}(\mathbf{x}) > 0 \right\} \right|}{|X_k|},$$

that is, the fraction of class-$k$ examples for which the concept direction positively influences the class logit (Kim et al., 2017). This aggregation is already global in the sense that it summarizes concept influence over many examples rather than only a single instance. A later survey makes this interpretation explicit: although it does not define GCAV as a named method, it treats a GCAV-style explanation as the use of a CAV together with global aggregation such as TCAV, completeness measures, or related concept-level summaries (Yeh et al., 2022).

2. From layer-specific concept vectors to a global representation

The most explicit formulation of GCAV appears in “GCAV: A Global Concept Activation Vector Framework for Cross-Layer Consistency in Interpretability” (He et al., 28 Aug 2025). Its starting point is a limitation of standard CAV practice: CAVs are commonly computed independently at different layers, even though different layers have different dimensions and encode different abstraction levels. The paper argues that this independence produces semantic mismatch across layers, unstable layer selection, spurious concept activations, and high variance in TCAV scores (He et al., 28 Aug 2025).

GCAV addresses that problem by replacing a set of unrelated layer-wise CAVs with a single globally aligned concept representation. The framework proceeds in three stages. First, each layer-specific CAV $x_l^c$ is compressed by a layer-specific autoencoder into a shared embedding space: $$z_l^c = f_{\text{encoder}^l}(x_l^c), \qquad \tilde{x}_l^c = f_{\text{decoder}^l}(z_l^c).$$ The reconstruction objective is a cosine reconstruction loss,

$C$0

which preserves direction information while unifying dimensions (He et al., 28 Aug 2025).

Second, the method aligns same-concept embeddings across layers using contrastive learning. Positive pairs are embeddings of the same concept from different layers, and negatives are derived from random CAVs. The principal alignment term is an InfoNCE loss: $C$1 A consistency loss keeps the aligned embeddings close to their original latent semantics (He et al., 28 Aug 2025).

Third, the aligned layer-wise embeddings are fused into a global concept embedding by an attention-based cross-layer module. The paper writes the resulting global concept representation as

$C$2

and then reconstructs a layer-specific concept vector from the global embedding with

$C$3

The key idea is therefore not to discard layer specificity, but to mediate it through a global latent representation that is intended to be semantically consistent across layers (He et al., 28 Aug 2025).

3. Testing with Global Concept Activation Vectors

The evaluation analogue of GCAV is TGCAV, or Testing with Global Concept Activation Vectors (He et al., 28 Aug 2025). TGCAV preserves the directional-derivative logic of TCAV, but replaces independently learned layer-wise CAVs with vectors decoded from the global concept embedding. The score is

$C$4

Thus TGCAV remains a class-level fraction of positive directional sensitivities, but the direction being tested has been globally aligned beforehand (He et al., 28 Aug 2025).

The fusion stage is explicitly trained to reduce cross-layer inconsistency. The paper introduces a variance loss over layer-wise scores,

$C$5

together with a consistency term between decoded global vectors and decoded aligned per-layer vectors (He et al., 28 Aug 2025). Because the hard TCAV indicator is non-differentiable, the appendix uses a sigmoid-plus-straight-through-estimator approximation during optimization (He et al., 28 Aug 2025). This suggests that GCAV is best understood as a representation-learning layer on top of standard CAVs, rather than as a redefinition of concept sensitivity itself.

More generally, the term “global” in the CAV literature has at least two meanings. In TCAV-style work, it often means aggregation over many examples of a class or dataset (Kim et al., 2017, Yeh et al., 2022). In the explicit GCAV framework, it means a semantically unified representation spanning multiple layers (He et al., 28 Aug 2025). A plausible implication is that GCAV consolidates both senses: it is global both as a cross-layer representation and as an object used in dataset-level concept testing.

4. Empirical behavior and reported benefits

The explicit GCAV paper evaluates GoogleNet, ResNet50V2, and MobileNetV2, using Broden concepts with 50 images per concept and 10 random probe datasets of 50 random ImageNet images each (He et al., 28 Aug 2025). Its main quantitative claim is improved cross-layer consistency. For example, for the concept “dotted” on the class zebra in GoogleNet, the reported standard deviation falls from $C$6 under TCAV to $C$7 under TGCAV, while the coefficient of variation falls from $C$8 to $C$9 and the range ratio from $\mathbf{v}_C^l$0 to $\mathbf{v}_C^l$1 (He et al., 28 Aug 2025). Similar reductions are reported for “striped” and “zigzagged” on zebra and for “honeycombed” on honeycomb across architectures (He et al., 28 Aug 2025).

The paper states that mean concept relevance is largely preserved while variability is reduced. It also reports qualitative improvements in concept localization: Visual-TCAV maps derived from GCAV are described as more concentrated on semantically relevant object regions and less prone to background activation than ordinary TCAV maps (He et al., 28 Aug 2025). In a targeted concept attack on zebra, the attacked layer’s TCAV score for “dotted” changes from $\mathbf{v}_C^l$2 to $\mathbf{v}_C^l$3, whereas the corresponding TGCAV score changes from $\mathbf{v}_C^l$4 to $\mathbf{v}_C^l$5, indicating reduced but not eliminated vulnerability to layer-targeted perturbation (He et al., 28 Aug 2025).

Adjacent work strengthens the importance of these stability questions. A statistical analysis of CAV construction shows that CAVs are random because they depend on sampled random negatives, and that under an asymptotic regime with fixed concept set and increasing number $\mathbf{v}_C^l$6 of random negatives, the variance of CAVs decreases as $\mathbf{v}_C^l$7 (Wenkmann et al., 28 Sep 2025). A later theoretical paper treats CAVs explicitly as random vectors, derives distributions for PatternCAV, FastCAV, and ridge-based CAVs, and argues that the standard TCAV score is statistically unstable because its discontinuous indicator can induce non-decaying variance in critical regimes (Schnoor et al., 15 May 2026). These results do not define GCAV, but they bear directly on any method that treats concept directions as stable global objects.

5. Conceptual extensions and neighboring alternatives

GCAV sits within a broader effort to refine what a global concept representation should be. One line of work argues that a concept direction should reflect concept signal rather than the geometry of a discriminative separator. “Navigating Neural Space” distinguishes between classifier filters and concept patterns, and proposes pattern-based CAVs computed by regressing activations on labels; for binary concepts this reduces to the difference of class means,

$\mathbf{v}_C^l$8

The paper argues that such vectors align better with the true concept direction than separator normals, which is especially relevant if a GCAV is supposed to function as a stable semantic axis rather than merely a detector (Pahde et al., 2022).

A second line of work questions whether a concept should be one global vector at all. “Concept Activation Regions” replaces a single concept direction with a concept activation region in latent space, motivated by the possibility that concept examples are scattered across several clusters (Crabbé et al., 2022). In the linear-kernel case, the framework recovers a global vector-like behavior, but its main point is that region-based geometry can be more faithful when concepts are multimodal (Crabbé et al., 2022). Likewise, “Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces” argues that concepts often form distributions of local vectors rather than a single global direction, and proposes one local concept embedding per sample, later aggregated by clustering (Mikriukov et al., 2023). This suggests that GCAV’s globality may be useful only when a concept is coherent enough to admit cross-layer alignment without severe loss of structure.

A third line of work focuses on entanglement between concepts. “Post-Hoc Concept Disentanglement” observes that independently learned CAVs for correlated concepts can become highly non-orthogonal and proposes refining them with an orthogonality penalty

$\mathbf{v}_C^l$9

while preserving directional correctness (Erogullari et al., 7 Mar 2025). Although that paper does not name GCAV, it directly addresses a central requirement of any global concept direction: moving along the vector should correspond to changing the intended concept rather than a correlated bundle of attributes (Erogullari et al., 7 Mar 2025).

6. Ambiguities, limitations, and terminology

The term GCAV is not fully standardized. In the cross-layer interpretability paper, it means Global Concept Activation Vector (He et al., 28 Aug 2025). In a separate large-language-model control paper, the same acronym stands for Generation with Concept Activation Vector, a framework that learns layer-wise concept vectors by logistic regression on hidden states and uses them to steer activations during inference (Zhang et al., 10 Jan 2025). The two uses share the CAV intuition of a reusable concept direction, but they solve different problems. This acronym ambiguity is now part of the term’s literature.

Even in the explicit Global Concept Activation Vector sense, several limitations remain. The framework still depends on the quality of the initial layer-wise CAVs, the quality of concept and random examples, and the assumption that the concept can be meaningfully aligned across layers (He et al., 28 Aug 2025). More general critiques of CAVs also apply. Probe accuracy alone is not a reliable measure of concept alignment, because probes can achieve high accuracy by exploiting spurious correlations rather than the intended concept (Lysnæs-Larsen et al., 6 Nov 2025). A probabilistic analysis shows that concept vectors depend strongly on the non-concept distribution, so even a “global” concept direction is only global relative to a chosen contrast set (Schnoor et al., 26 Sep 2025). This suggests that GCAV should not be interpreted as a universally canonical semantic axis.

A further limitation is geometric. Concept Gradient argues that a single linear concept direction is often too restrictive, and generalizes CAV to non-linear differentiable concept functions with local concept gradients

$$S_{C,k,l}(\mathbf{x}) = \nabla h_{l,k}(f_l(\mathbf{x})) \cdot \mathbf{v}_C^l.$$0

thereby replacing one global linear direction with an input-dependent local derivative (Bai et al., 2022). From that perspective, GCAV is strongest when concepts are sufficiently linear and sufficiently coherent across layers. Where concepts are curved, multimodal, entangled, or only partially linearly encoded, GCAV should be read as a structured approximation rather than a definitive semantic representation.

Overall, Global Concept Activation Vector denotes a maturation of concept-based interpretability away from isolated, layer-specific probes toward globally reusable concept objects. In its explicit cross-layer form, GCAV is a framework for compressing, aligning, and fusing layer-wise CAVs into a single representation that supports more stable TGCAV scores, better qualitative localization, and greater robustness to single-layer perturbations (He et al., 28 Aug 2025). In the wider literature, it also names a broader ambition: to turn concept directions from fragile local artifacts into stable, semantically meaningful instruments for global analysis of neural representations (Kim et al., 2017, Yeh et al., 2022).