Transferable Concept Vectors

• Transferable Concept Vectors are concept-linked representations that maintain semantic or functional meaning across diverse settings, including layers, datasets, and modalities.
• They are constructed using methods like linear separability, pattern differences, and attention head summation to isolate true concept directions from distractor influences.
• Key applications include concept sensitivity analysis, model correction against artifacts, and cross-modal inference, highlighting their role in robust model interpretability.

Transferable Concept Vectors are concept-linked representations that are intended to preserve their semantic or functional meaning when conditions change, including layer, architecture, dataset, language, prompt format, modality, or even parameterization. In the literature, the transferable object may be a latent-space direction such as a Concept Activation Vector, a sentence-level projection direction, a steering vector in a transformer residual stream, a task vector in a vision-LLM, an ontology-defined conceptual axis, or a parameter-space capability difference. Across these formulations, the core question is whether the representation captures the concept itself rather than nuisance correlations introduced by separability objectives, prompt format, model idiosyncrasies, or dataset artifacts (Pahde et al., 2022, Lyngbaek et al., 12 Jan 2026, Opiełka et al., 5 Mar 2025, Song et al., 11 May 2026).

1. Conceptual scope and representational forms

The literature does not use a single canonical object. Instead, it operationalizes transferable concept vectors through several related constructions. In vision and medical imaging, a concept vector is typically a direction in a layer activation space. In sentence embedding work, it is a normalized centroid difference used as a projection axis. In transformer interpretability, it may be a sum of attention-head outputs selected for conceptual invariance rather than causal contribution to the next token. In multimodal models, it may be a hidden-state “task vector” that is invariant across text and image specifications. In robotics-oriented VLA finetuning, the corresponding object is a parameter-space difference that isolates capability gains from auxiliary objectives (Pahde et al., 2022, Lyngbaek et al., 12 Jan 2026, Opiełka et al., 5 Mar 2025, Luo et al., 2024, Song et al., 11 May 2026).

Setting	Representation	Transfer target
Pattern-based CAVs	$v_c^{(l)} = \mu^+ - \mu^-$ or $\mathrm{cov}[A,t]/\sigma_t^2$	Layers, architectures, datasets, preprocessing (Pahde et al., 2022)
CVP for sentiment	Unit-normalized positive-negative centroid difference	Genres, periods, languages (Lyngbaek et al., 12 Jan 2026)
LLM concept vectors	Sum of mean outputs from top RSA heads	Languages and prompt formats (Opiełka et al., 5 Mar 2025, Opiełka et al., 25 Feb 2026)
VLM task vectors	Final-delimiter hidden state at a selected layer	Text/image and LLM$\rightarrow$VLM transfer (Luo et al., 2024)
Capability vectors	$\gamma_{ao} = \theta_{ao} - \theta_{ft}$	Models, environments, embodiments (Song et al., 11 May 2026)

This variety matters because different papers define transferability differently. Some prioritize direction fidelity, some prioritize behavioral steering, some measure cross-domain correlation, and some measure geometry preservation. A transferable concept vector is therefore not merely a vector that works elsewhere; it is a vector whose underlying conceptual geometry remains stable enough for the intended downstream use.

2. Construction paradigms

A major construction family begins with linear separability. Standard CAVs are learned from latent activations $h^{(l)}(x)$ by fitting a linear classifier between concept-positive and concept-negative examples. The literature explicitly gives logistic-regression and linear-SVM objectives, and also notes ridge-style variants in which the learned weight is taken as the concept vector. The standard assumption is that the learned normal vector points from concept-negative to concept-positive regions of activation space (Pahde et al., 2022).

A central revision of that paradigm is the distinction between filters and patterns. Pattern-based CAVs replace classifier weights with a pattern that explains the variance in activations attributable to the concept label. For binary labels, the resulting closed form is the difference of layer-wise mean activations,

$h = \bar y_A^+ - \bar y_A^-,$

equivalently $h = \mathrm{cov}[A,t]/\sigma_t^2$. The stated motivation is that separability-oriented filters mix the true concept direction with distractor directions, whereas the pattern solution suppresses distractors that are independent of the concept in expectation (Pahde et al., 2022).

Several other transferable constructions also use mean differences, but in different spaces. CVP models valence as the unit-normalized difference between positive and negative sentence centroids in a multilingual sentence embedding space, with scores computed by projection and then z-score normalized per corpus (Lyngbaek et al., 12 Jan 2026). ConTrans computes layer-wise concept directions in a source LLM as averaged hidden-state differences between positive and negative prompts and then reformulates them for a target model through a least-squares affine map (Dong et al., 2024). Cross-model steering-vector work similarly uses mean differences or first principal components of contrastive hidden-state differences and then aligns them between models with ordinary least squares (Huang et al., 2 Jan 2025).

In LLM circuit-style work, concept vectors are not extracted from a single classifier at all. Instead, attention heads are ranked by Representational Similarity Analysis (RSA), using a design matrix for the concept attribute and a representational similarity matrix built from head outputs. The concept vector is then formed by summing the mean outputs of the top heads by $\Phi_{\text{concept}}$, typically the top 3 heads in the cited experiments (Opiełka et al., 5 Mar 2025, Opiełka et al., 25 Feb 2026). This construction is designed to separate invariant concept encoding from format-specific task policy, which the same papers attribute to Function Vectors.

Structured alternatives depart from single directions. Simhi and Markovitch define a conceptual space by ontology concepts and map an embedding $l$ to similarities with concept prototypes,

$$CES^{f,C,\tau}(l)=\langle sim(l,\hat c_1),\dots,sim(l,\hat c_n)\rangle^T,$$

with a linear realization $\mathrm{cov}[A,t]/\sigma_t^2$0 when vectors are normalized and cosine similarity is used (Simhi et al., 2022). Zhu and Yu represent semantic slots as ordered tuples of atomic concepts and factor prediction across dimensions, allowing transfer through shared atomic substructure rather than a monolithic slot label (Zhu et al., 2017). “Gaussian Concept Subspace” extends probe-derived vectors to a distribution over probe weights, fitting a Gaussian over multiple bootstrap probes to capture uncertainty and multi-faceted structure rather than committing to one unstable direction (Zhao et al., 2024).

3. Criteria for transferability

Transferability is measured with different criteria in different subfields. In controlled CAV studies, the most direct criterion is alignment with a “true” concept direction constructed from paired samples that differ only in the concept,

$\mathrm{cov}[A,t]/\sigma_t^2$1

and measured by cosine similarity against the estimated vector. The same work defines transferability operationally through direction stability across layers, architecture robustness across VGG16, ResNet18, and EfficientNet-B0, dataset generalization across ISIC2019, Pediatric Bone Age, and FunnyBirds, and preprocessing invariance to centering and scaling (Pahde et al., 2022).

In chest X-ray autoencoders, transferability is evaluated by cosine similarity between independently fitted latent-space CAVs across datasets and by IoU between attribution maps and ground-truth bounding boxes. The emphasis is not only whether a concept direction exists, but whether averaging across batches yields a direction that remains stable under dataset shift (Maksudov et al., 4 Jun 2025).

For sentiment CVP, the primary criterion is Spearman correlation between projected scores and human annotations under train-on-A, test-on-B transfer. This makes portability a predictive notion: a concept vector is transferable if a vector trained on one corpus yields only minimal performance loss on another corpus spanning different genres, historical periods, or languages (Lyngbaek et al., 12 Jan 2026).

LLM work separates causal efficacy from invariance. RSA-based concept vectors are judged by within-concept clustering across formats and languages, by lower question-type RSA than Function Vectors, by cosine similarity between concept vectors extracted under different prompt conditions, and by KL divergence between steered token distributions under in-distribution and out-of-distribution extraction formats (Opiełka et al., 5 Mar 2025, Opiełka et al., 25 Feb 2026). VLM task-vector work measures exact-match accuracy after patching a hidden-state vector derived in one modality into the computation for another modality, and also compares cosine similarity between task vectors from a base LLM and its finetuned VLM counterpart (Luo et al., 2024).

Recent geometry-centered work broadens the criterion further. vec2vec defines translation operators into and out of a universal latent space and trains them with adversarial alignment, reconstruction, cycle consistency, and Vector Space Preservation to maintain pairwise geometry, reporting cosine similarity up to 0.92 across model pairs (Jha et al., 18 May 2025). A subsequent line models concepts as point-cloud manifolds and context effects as vector fields, comparing displacement structure across models with CKA, Grassmann distance, magnitude rank correlation, and residual cosine similarity (Hu et al., 5 Jul 2026). These papers shift the emphasis from single vectors to transferable relational geometry.

4. Empirical evidence across domains

In convolutional vision models, pattern-based CAVs consistently align better with the controlled ground-truth concept direction than filter-based CAVs across all convolutional layers of VGG16 and across ResNet18 and EfficientNet-B0 blocks, even though filter-based CAVs show higher separability AUC. The same study reports that RR-ClArC correction with a pattern-based artifact vector on ISIC2019 reduces band-aid artifact relevance in VGG16 input heatmaps from 0.51 to 0.31, reduces $\mathrm{cov}[A,t]/\sigma_t^2$2 from 0.10 to 0.03, raises biased test accuracy from 0.75 to 0.79, and keeps clean accuracy at approximately 0.82 (Pahde et al., 2022).

In chest X-ray autoencoders, averaged latent-space concept vectors are highly stable within dataset and measurably stable across datasets. For Atelectasis, random comparisons are near zero, intra-dataset mean alignment is $\mathrm{cov}[A,t]/\sigma_t^2$3 for NIH and $\mathrm{cov}[A,t]/\sigma_t^2$4 for CheXpert, and cross-dataset mean alignment is $\mathrm{cov}[A,t]/\sigma_t^2$5. Localization results are mixed: averaged CAV traversal generally underperforms Latent Shift on small, spatially variable findings, but NIH Cardiomegaly reaches IoU 0.3630 versus 0.2864 for Latent Shift (Maksudov et al., 4 Jun 2025).

For sentiment, CVP shows that a single centroid-difference direction transfers with only small losses across corpora. Reported valence Spearman correlations are 0.66 for Fiction4, 0.70 for EmoBank, and 0.68 for Facebook in-domain, while cross-corpus transfer losses are typically between $\mathrm{cov}[A,t]/\sigma_t^2$6 and $\mathrm{cov}[A,t]/\sigma_t^2$7. The same work also finds that arousal and dominance are detectable but less stable, and that the linearity assumption is only approximate because neutral content introduces a second “neutral-component” direction, producing a banana-shaped manifold (Lyngbaek et al., 12 Jan 2026).

In LLMs, the strongest evidence concerns verbal concepts rather than abstract relational ones. For Llama 3.1, mean similarity across languages for the same verbal concept is approximately 0.8 and mean similarity across formats is approximately 0.7, but abstract “Previous” and “Next” do not yield invariant linear concept vectors, with $\mathrm{cov}[A,t]/\sigma_t^2$8 compared with $\mathrm{cov}[A,t]/\sigma_t^2$9 (Opiełka et al., 5 Mar 2025). Across Llama and Qwen models, Function Vectors extracted from different formats are nearly orthogonal, while RSA-selected Concept Vectors yield lower KL divergence and more stable out-of-distribution steering across open-ended versus multiple-choice prompts and across languages (Opiełka et al., 25 Feb 2026).

Cross-modal and cross-model results extend the same theme. In VLMs, text-derived task vectors patched into image computations outperform text ICL in-context baselines on image queries, with reported averages of 0.32 versus 0.18 for LLaVA-v1.5, 0.31 versus 0.09 for Mantis-Fuyu, and 0.51 versus 0.18 for Idefics2. Task-vector cosine similarity is also high between base LLMs and their VLM descendants, reported as 0.95 for Vicuna$\rightarrow$0LLaVA-v1.5 and 0.89 for Mistral$\rightarrow$1Idefics2 (Luo et al., 2024). In cross-model steering among decoder-only LLMs, ordinary least-squares alignment can transfer harmfulness vectors so that target-model harmful-output rates rise from 0.0% to 96.0% or 98.0% in selected source-target pairs (Huang et al., 2 Jan 2025).

Parameter-space capability transfer exhibits a related pattern. CapVector extracts

$\rightarrow$2

from two finetuned VLA models and merges it into pretrained parameters. On RoboTwin 2.0, a capability vector extracted from LIBERO-Spatial raises OpenVLA-OFT average success from 6.7% to 31.8%, close to 33.1% for Spatial Forcing, and external cross-embodiment deployment improves a four-stage test-tube transfer task from 0.36 to 0.53 out of the box (Song et al., 11 May 2026).

5. Principal applications

One major application is concept sensitivity analysis. In TCAV-style use, the directional derivative

$\rightarrow$3

measures whether moving along the concept direction increases or decreases the class logit, and the TCAV score is the fraction of concept-positive inputs with positive derivative. The pattern-CAV work argues that this application requires direction fidelity rather than mere separability, because a misaligned vector yields unreliable sensitivity maps and unstable TCAV values under distractor rotations (Pahde et al., 2022).

A second application is model correction against shortcuts or artifacts. RR-ClArC penalizes gradient sensitivity along an artifact vector through

$\rightarrow$4

thereby attempting to suppress the shortcut without damaging the true signal. The efficacy of this intervention depends directly on the fidelity of the transferred artifact direction (Pahde et al., 2022).

Generative counterfactuals provide another use case. In chest X-ray autoencoders, a concept vector estimated post hoc in latent space is traversed as

$\rightarrow$5

to exaggerate or curtail a clinical concept. The same study computes explanation maps from the maximum absolute deviation across a discrete traversal schedule and uses pixel-wise differences between exaggerated and curtailed reconstructions to highlight clinically relevant regions (Maksudov et al., 4 Jun 2025).

In transformers, concept vectors are used for inference-time steering. RSA-selected LLM concept vectors and task vectors in VLMs are added to hidden states or residual streams,

$\rightarrow$6

to bias the model toward a concept or task without parameter updates (Opiełka et al., 5 Mar 2025, Opiełka et al., 25 Feb 2026, Luo et al., 2024). Cross-model steering-vector work and ConTrans extend this by first aligning the vector between models with least-squares or affine transport and then injecting it into the target residual stream (Huang et al., 2 Jan 2025, Dong et al., 2024).

Structured concept representations support broader analytical tasks. Ontology-based conceptualization creates reusable concept axes for comparing models, tracing layers, and auditing bias by measuring differences along curated concept dimensions (Simhi et al., 2022). Atomic-concept decompositions support adaptive LU under slot-definition mismatch by transferring supervision through shared substructures such as value types and role atoms (Zhu et al., 2017). In information retrieval, concept vectors built from UMLS terms allow words and concepts to inhabit the same embedding space, supporting conceptual indexing and inter-concept similarity computation (Abdulahhad, 2020).

6. Limits, controversies, and current extensions

A recurrent limitation is that vectors optimized for one objective may not represent the concept itself. Pattern-based CAV work explicitly identifies directional divergence: classifier weights optimize separability and can rotate toward distractor directions. The same paper notes additional caveats for correlated concepts, layer dependence, and dataset bias in the concept-positive and concept-negative sets (Pahde et al., 2022).

A second limitation is that linearity is often approximate rather than exact. The sentiment CVP study finds that valence is largely captured by one direction, but neutrality deviates off-axis and produces banana-shaped curvature. The implication given in that work is not that linear projection fails completely, but that single-direction models leave systematic residual structure unmodeled (Lyngbaek et al., 12 Jan 2026). The LLM concept-vector literature reaches a related conclusion from another angle: verbal concepts such as translation or antonymy admit invariant linear representations, whereas more abstract relational notions such as “previous” and “next” do not (Opiełka et al., 5 Mar 2025, Opiełka et al., 25 Feb 2026).

A third controversy concerns whether concept vectors are transferable at all in some parameterizations. The visual soft-prompt study finds that single-word concept embeddings trained for Stable Diffusion, OWL-v2, and DFN are largely model-specific and non-transferable. Performant embeddings can be found within small $\rightarrow$7-balls around unrelated anchors, but when these embeddings are linearly mapped into a new model they lose the fine-tuned effect and collapse toward the nearest known token neighborhood. Relative transfer can be as low as the reported $\rightarrow$8 range for many cross-task settings, although classification$\rightarrow$9generation reaches up to 84% for some common PASCAL categories (Trabucco et al., 2024). This is a direct counterexample to the stronger claim that any learned concept direction should port across models.

A fourth issue is instability of single directions. Probe-derived concept vectors can vary with subsamples and seeds, which motivates Gaussian Concept Subspace and related subspace-oriented models rather than one fixed direction (Zhao et al., 2024). More recent work also moves beyond stationary vectors by modeling concepts as point-cloud manifolds and contextual effects as vector fields. That literature reports that the variance in contextual displacements is semantically organized by lexical density and concreteness, and that this displacement structure can be transported across models to predict held-out displacements above chance (Hu et al., 5 Jul 2026). In parallel, vec2vec proposes a universal latent space with non-linear adapters and geometry-preserving losses, recommending that concepts be computed in the universal space and then decoded into target embedding spaces rather than mapped as raw source-space directions (Jha et al., 18 May 2025).

These extensions indicate a broader shift in the field. When the application depends on the direction itself, the literature increasingly favors constructions that prioritize invariance, geometry preservation, or uncertainty over pure separability. When the application depends on task execution or in-distribution control, richer vectors that encode policy or format-specific detail may still be preferable. The resulting picture is not that one transferable concept vector formalism has replaced all others, but that transferability has become a diagnostic property whose adequacy depends on the geometry of the concept and the demands of the downstream intervention.