Anchored Feature Representations

• Anchored feature representations are learning schemes that stabilize latent spaces by referencing fixed or slowly updated anchors, preventing collapse and drift.
• They employ diverse forms—such as projection domains, semantic anchors, and spatial or anatomical anchors—to maintain modality-specific structure and improve feature matching.
• Empirical studies report enhanced performance in areas like speech recognition, medical imaging, and visual navigation, highlighting anchoring's practical benefits.

Anchored feature representations are representation-learning schemes in which latent variables are organized relative to a stable reference rather than being shaped only by a task objective and the model’s incidental internal geometry. In recent work, the anchor has taken the form of a lightweight shared projection domain, a class or label embedding, a token embedding from a model vocabulary, an exponential-moving-average reference branch, an anatomy mask, a frozen human motion manifold, a set of reliable correspondence seeds, or the camera pose from which a landmark was first observed. Across multimodal recommendation, self-supervised speech, few-shot relation extraction, emotion recognition, implicit neural representations, medical imaging, robotics, interpretability, feature matching, financial NLP, and visual-inertial navigation, the shared purpose is to prevent collapse or drift, preserve domain structure, improve separability or transfer, and expose more controllable or consistent coordinates (Jeong et al., 13 Mar 2026, Ioannides et al., 30 Jan 2026, Ge et al., 2023, Zhang et al., 26 Jun 2026, Cohen et al., 17 Jun 2026).

1. Motivation and problem setting

The immediate motivation for anchoring is that unconstrained or overly self-referential representation learning often produces unstable geometry. In multimodal recommender systems, aggressive unification of item ID, image, and text embeddings can cause positional collapse, ID dominance, and the blurring of modality-specific structures (Jeong et al., 13 Mar 2026). In JEPA-style self-supervised speech learning, an EMA teacher alone may not provide a sufficiently stable acoustic reference, and the reported failure mode is outright representation collapse, with Pure JEPA yielding 100.00% WER on ASR (Ioannides et al., 30 Jan 2026). In long-tailed chest X-ray classification, the problem is formulated as feature drift under long-tailed imbalance, where majority-class gradients repeatedly overwrite minority-sensitive features (Khuong et al., 4 May 2026). In prototype-based representation learning, class centers derived from current features inherit the noise, bias, and long-tail skew of those same features, producing an EM-like feedback loop (Ge et al., 2023).

A second motivation is the mismatch between useful structure in the data and the geometry induced by standard architectures. Global MLP-based implicit neural representations are sensitive to absolute image location and struggle with high-frequency detail, which motivates explicitly local, image-space-anchored latent vectors (Luijmes et al., 19 Mar 2025). In feature matching, all-to-all self- and cross-attention over primal features is computationally redundant and susceptible to outliers, motivating a small set of reliable anchor correspondences as a message bottleneck (Jiang et al., 2023). In visual-inertial navigation, representing landmarks directly in a global frame makes the unobservable subspace depend on the estimated landmark state, which harms EKF consistency under relinearization; anchoring landmarks to an observation pose removes that landmark-state dependence (Cohen et al., 17 Jun 2026).

Taken together, these settings treat anchoring not as a cosmetic regularizer but as a response to specific pathologies: collapse, drift, prototype bias, coordinate incompatibility, excessive sensitivity to global position, and inconsistency under state estimation.

2. Forms of anchoring

One major family uses a shared coordinate system or projection domain. In AnchorRec, modalities are aligned indirectly in a lightweight projection domain rather than forced into a single native space, so alignment is decoupled from representation learning (Jeong et al., 13 Mar 2026). In Anchor Coordinate Space (ACS), each hidden state is represented by its cosine similarities to a fixed prompt pool, yielding a common $\mathbb{R}^N$ space in which behavioral directions from different LLM families can be projected, averaged, and reconstructed (Kim et al., 11 May 2026). FinAnchor chooses one frozen encoder as an anchor space and learns linear ridge maps from other encoders into that space before aggregation (He et al., 24 Feb 2026).

A second family uses semantic anchors. In SaCon, sentences and relation labels are treated as complementary views of the same relation, and both sentence-anchored and label-anchored contrastive losses are optimized symmetrically (Luo et al., 2023). In EACL, textual emotion labels are encoded into emotion anchors that serve as semantic attractors for utterance representations, with an auxiliary angular loss to force similar emotions apart (Yu et al., 2024). Semantic Anchor Regularization (SAR) replaces feature-derived prototypes with predefined class anchors serving as feature centroids, then makes them classifier-aware through an auxiliary cross-entropy objective and EMA-smoothed semantic embedding (Ge et al., 2023). VASAE anchors SAE decoder columns toward the model’s own token embeddings, so each feature acquires an intrinsic token name (Zhang et al., 26 Jun 2026). Sparse Concept Anchoring assigns selected concepts to fixed directions or axis-aligned subspaces on a hypersphere, while leaving the remainder of latent space free to self-organize (Fraser et al., 13 Dec 2025).

A third family uses spatial, temporal, anatomical, or physical anchors. ARC places latent vectors directly in image coordinate space and retrieves only the nearest anchored latents for a query coordinate (Luijmes et al., 19 Mar 2025). ANAUS shifts ultrasound SSL from frame-level or generic region-level objectives to anatomically delineated masks, using anatomy as the unit of representation and alignment (Zhu et al., 25 May 2026). The momentum-anchored chest X-ray model introduces an EMA-updated copy of the final expansion block as a slow-moving reference branch (Khuong et al., 4 May 2026). PHASOR freezes a human-pretrained phase manifold and aligns multiple humanoid robots to that same manifold through lightweight adapters and semantic losses (Kim et al., 1 Jun 2026). In VINS, the anchor is the camera or pose of first observation, relative to which inverse-depth landmark parameters are defined (Cohen et al., 17 Jun 2026).

These variants differ in what is anchored and in what is used as the reference, but they share the same structural move: the representation is constrained relative to an external or stabilized frame of reference.

3. Mathematical archetypes

A common formal pattern is projection into anchor coordinates. In ACS, if $A_m[i]=a_m(z_i;L_m)$ denotes anchor activations for model $m$, then the centered, normalized anchor matrix $\widehat{A}_m[i]=\operatorname{norm}(A_m[i]-\mu_m)$ defines the projection

$$\pi_m^{\mathrm{pt}}(h)=\widehat{A}_m \cdot \operatorname{norm}(h-\mu_m)\in\mathbb{R}^{N},$$

and a direction projects as

$$\pi_m^{\mathrm{dir}}(v)=\widehat{A}_m \cdot \operatorname{norm}(v).$$

Projected directions are averaged into a canonical ACS direction $c_a$, which is then back-projected to a new model by

$$v_{a,\mathrm{recon}}^{(m_u)}=\widehat{A}_{m_u}^{\top} c_a.$$

FinAnchor implements a related but supervised alignment: for each source encoder $m\neq a$, the map $\mathbf{W}_m$ is learned by ridge regression into the anchor encoder’s space (Kim et al., 11 May 2026, He et al., 24 Feb 2026).

A second pattern is anchor-guided contrastive or compactness loss. SaCon defines

$A_m[i]=a_m(z_i;L_m)$0

with sentence-anchored and label-anchored terms operating in opposite directions across the same paired instance-label structure (Luo et al., 2023). EACL augments supervised contrastive learning with an anchor-angle loss

$A_m[i]=a_m(z_i;L_m)$1

which explicitly maximizes the smallest inter-anchor angle (Yu et al., 2024). SAR pulls features toward semantic anchors via

$A_m[i]=a_m(z_i;L_m)$2

while Sparse Concept Anchoring uses either a directional penalty,

$A_m[i]=a_m(z_i;L_m)$3

or a subspace penalty,

$A_m[i]=a_m(z_i;L_m)$4

These objectives all impose geometry relative to predefined or stabilized reference features (Ge et al., 2023, Fraser et al., 13 Dec 2025).

A third pattern is slow reference dynamics or anchored parameterization. In the momentum-anchored chest X-ray model, the anchor branch is updated by

$A_m[i]=a_m(z_i;L_m)$5

with $A_m[i]=a_m(z_i;L_m)$6, so the reference evolves slowly and resists gradient-induced drift (Khuong et al., 4 May 2026). In anchored VINS, inverse-depth coordinates are defined in an anchoring frame by

$A_m[i]=a_m(z_i;L_m)$7

so the landmark state is local to an anchor pose rather than global (Cohen et al., 17 Jun 2026).

4. Domain-specific realizations

In multimodal systems, anchoring is often a way to preserve native modality geometry while still enforcing comparability. AnchorRec is explicit on this point: image, text, and ID encoders are learned separately; only their projected forms are aligned in a lightweight shared domain; and the recommendation objective is optimized jointly with that auxiliary alignment so that the original modality spaces are not turned into a bottleneck (Jeong et al., 13 Mar 2026). FinAnchor addresses a related incompatibility at the model level rather than the modality level: frozen LLM embeddings for long financial documents are mapped into one anchor space before averaging, so complementary model-specific evidence becomes usable without fine-tuning (He et al., 24 Feb 2026).

In speech and language, anchoring is frequently tied to semantic or acoustic grounding. GMM-Anchored JEPA fits a diagonal-covariance GMM once on log-mel spectrograms and uses frozen soft posteriors as auxiliary targets throughout training, with a linearly decaying $A_m[i]=a_m(z_i;L_m)$8 from 1.0 to 0.01 so that acoustic grounding dominates early and persists residually later (Ioannides et al., 30 Jan 2026). SaCon treats sentence instances and label descriptions as incomplete yet complementary semantic views, and EACL converts label texts into stable emotion anchors that later become nearest-anchor classifiers after a second adaptation stage (Luo et al., 2023, Yu et al., 2024). VASAE pushes SAE dictionary directions toward the vocabulary geometry of the underlying transformer, so feature naming is intrinsic rather than purely post hoc (Zhang et al., 26 Jun 2026).

In vision and medical imaging, the anchor is often spatial or anatomical. ARC converts an image into a latent point cloud whose anchor positions are fixed after initialization and sampled by the image gradient norm, so locality is built into the INR representation itself (Luijmes et al., 19 Mar 2025). ANAUS first produces annotation-free anatomical masks with LP-SAM and a latent prompt engine, then aligns same-anatomy regions across views while separating distinct structures and reconstructing corrupted core anatomy (Zhu et al., 25 May 2026). The long-tailed chest X-ray model uses a temporally anchored EMA branch together with multi-scale spatial fusion, so the stabilized representation is also scale-aware (Khuong et al., 4 May 2026).

In geometry-sensitive settings, anchors become structural constraints on information flow or state parameterization. AMatFormer selects top-$A_m[i]=a_m(z_i;L_m)$9 reliable tentative matches as anchor pairs, performs self- and cross-attention on them, and then uses anchor-primary attention to broadcast refined consensus information back to all primal features (Jiang et al., 2023). PHASOR fixes a human-pretrained phase manifold as a shared anchor for robots, so new embodiments are aligned to a stable motion geometry instead of learning embodiment-specific action spaces (Kim et al., 1 Jun 2026). Anchored VINS parameterizes landmarks relative to the first observing pose, which changes the estimator’s observability structure in a favorable way (Cohen et al., 17 Jun 2026).

5. Reported empirical behavior

Across domains, the reported gains associated with anchoring are heterogeneous in metric but recurrent in character: reduced collapse, improved transfer, better minority or similar-class discrimination, stronger robustness to coordinate mismatch, and greater interpretability. Representative results are summarized below.

Setting	Reported result	Anchoring effect
GMM-Anchored JEPA (Ioannides et al., 30 Jan 2026)	28.68% WER vs. 33.22% baseline; 98% entropy vs. 31%	Collapse prevention and more uniform cluster utilization
ARC (Luijmes et al., 19 Mar 2025)	FMNIST padded: 80.42 / 79.36 / 73.57 at 28 / 100 / 1024; shifted test 79.36 → 47.61 vs. 74.50 → 13.00	High-resolution stability and better translation robustness
Momentum-anchored chest X-ray (Khuong et al., 4 May 2026)	0.8682 average AUC; baseline 0.8304; momentum-only 0.8670	Reduced feature drift under long-tail imbalance
EACL (Yu et al., 2024)	Weighted-F1: 70.41 / 67.12 / 40.24; excited/happy cosine 0.77 → 0.08	Better separation of similar emotions
ACS across LLM families (Kim et al., 11 May 2026)	LQMP same-axis cosine about +0.79; 0.83 ten-way accuracy; 0.95 mean binary AUROC	Cross-family direction alignment and transfer
FinAnchor (He et al., 24 Feb 2026)	Conference Call 0.787 / 0.741; FOMC 0.643 / 0.656; concatenation on FOMC 0.613 Acc / 0.619 F1	Aligned multi-model fusion outperforms naive concatenation
ANAUS (Zhu et al., 25 May 2026)	96.2% POCUS accuracy; 87.1% BUSI accuracy; 85.9% Dice and 92.7% PPV on UDIAT-B	Anatomy-grounded invariance and segmentation transfer
VASAE (Zhang et al., 26 Jun 2026)	GPT-2-small layers L0–L10 align about 89%–94% at cutoff 0.8; Llama shallow layer 92.8%	Vocabulary-grounded dictionary features without reconstruction loss degradation

Other results fit the same pattern. AnchorRec reports competitive top-N recommendation accuracy on four Amazon datasets, together with qualitative evidence of improved multimodal expressiveness and coherence (Jeong et al., 13 Mar 2026). In VINS, real-world TUM-VI experiments show that anchored feature parameterizations alone can yield performance comparable to consistency-improved estimators using global feature representations (Cohen et al., 17 Jun 2026). In SaCon, the bidirectional anchored objective improves downstream few-shot, domain-shift, and zero-shot relation extraction over baselines (Luo et al., 2023).

6. Misconceptions, limitations, and unresolved questions

A recurrent misconception is that anchoring simply means forcing everything into one common space. The literature repeatedly argues against that interpretation. AnchorRec is framed precisely as an alternative to direct full-space unification, because such unification can suppress modality-specific information (Jeong et al., 13 Mar 2026). AMatFormer uses anchors as a bottleneck for message passing rather than as a universal replacement for all features (Jiang et al., 2023). VASAE shows that soft anchoring and hard tying are not interchangeable: fixing the decoder to the token embedding matrix produces much worse reconstruction than allowing learnable features that are merely pulled toward vocabulary directions (Zhang et al., 26 Jun 2026).

A second misconception is that anchoring guarantees invariance or universality. ARC improves translation robustness but is not fully translation invariant, and downstream classifier choice matters materially (Luijmes et al., 19 Mar 2025). In ACS, cross-family universality is qualified: the Llama–Qwen–Mistral–Phi cluster aligns tightly, while Gemma is an outlier with much lower similarity and some anti-aligned axes (Kim et al., 11 May 2026). In VASAE, shallow and middle layers are strongly vocabulary-aligned, but the representative final-layer dictionary in Llama-3.1-8B shows limited or unstable alignment (Zhang et al., 26 Jun 2026). In anchored VINS, the unobservable subspace becomes independent of the landmark state but still depends on the navigation state, so additional FEJ- or RI-style consistency machinery can still help (Cohen et al., 17 Jun 2026). In GMM-Anchored JEPA, setting $m$0 sharply degrades entropy, used clusters, and WER, indicating that the anchor cannot be treated as a one-time initialization device (Ioannides et al., 30 Jan 2026).

These limitations suggest several broader research questions. The papers collectively indicate that anchor quality, anchor granularity, update schedule, and where anchoring is applied are first-class design variables. This suggests that future progress may depend less on the generic idea of “using anchors” than on how carefully the anchor is chosen, how strongly it constrains the latent space, and whether the anchor should be fixed, softly updated, or adapted in a decoupled second stage (Khuong et al., 4 May 2026, Zhu et al., 25 May 2026, Fraser et al., 13 Dec 2025). A plausible implication is that the most effective anchored systems will continue to be those that preserve native representational capacity while using anchors only where collapse, drift, incompatibility, or observability defects actually arise.