ACE: Anisotropy-Controllable Embedding

• ACE is a geometry-aware embedding method that adjusts anisotropic LLM-generated item representations for effective sequential recommendation.
• It implements a linear autoencoder with L2-regularization to perform controlled singular value shrinkage, preserving semantic structure while reducing dominant directions.
• Empirical results show that ACE improves metrics such as Recall@20 and NDCG@20 across various recommendation backbones and datasets.

Anisotropy-Controllable Embedding (ACE) is a geometry-aware embedding reshaping method for LLM-enhanced sequential recommendation (SR). It is proposed for the LLM-as-Extractor setting, where item textual metadata is first encoded by an LLM into dense vectors and then transferred into SR backbones. The central claim is that such LLM-generated item embeddings are semantically rich but often strongly anisotropic: most vectors concentrate in similar directions, and variance is dominated by a few principal components. ACE addresses this by applying a linear autoencoder (LAE) with an $L_2$-regularized reconstruction objective that reshapes the spectrum of the embedding space while preserving semantic structure, thereby balancing geometric uniformity and semantic fidelity (Lee et al., 28 May 2026).

1. Definition and problem setting

ACE is introduced as an initialization module for LLM-enhanced SR rather than as a standalone recommender. In the underlying pipeline, item text metadata is encoded into an embedding matrix $\mathbf{E}\in\mathbb{R}^{n\times d}$, projected into a lower-dimensional SR embedding space, and then used to initialize the item embedding table of a downstream SR backbone such as SASRec, GRU4Rec, or BERT4Rec (Lee et al., 28 May 2026).

The method is motivated by a specific failure mode of LLM-generated item representations. According to the paper, these embeddings are often highly anisotropic: vectors cluster in a narrow cone, and the covariance spectrum is extremely skewed. Writing the covariance as

$$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$

the paper states that a few eigenvalues dominate, indicating a geometric imbalance that makes it difficult to adapt the embeddings to collaborative signals during SR fine-tuning (Lee et al., 28 May 2026).

This framing places ACE in the broader class of methods that attempt to reconcile semantic priors from foundation models with interaction-driven adaptation in recommender systems. Its specific intervention point is the geometry of the initial item space.

2. Geometric motivation: anisotropy, semantic retention, and controlled correction

The paper characterizes anisotropy as a mismatch between semantic richness and optimization suitability. LLM embeddings may encode meaningful item semantics from metadata such as title, category, and brand, but their concentration in a small number of dominant directions reduces representational diversity during downstream adaptation (Lee et al., 28 May 2026).

ACE is positioned against whitening-based approaches such as WhitenRec. The paper argues that whitening can force perfect isotropy but is too rigid and can destroy semantic structure by flattening the spectrum too aggressively. ACE is designed to occupy an intermediate regime: less anisotropic than raw LLM embeddings, yet less destructive than full whitening. In the paper’s own interpretation, it operates as a “generalized whitening” mechanism that smooths the spectrum without collapsing the semantic hierarchy encoded in the singular values (Lee et al., 28 May 2026).

The method therefore treats embedding geometry as a controllable property rather than as a byproduct of the upstream encoder. A plausible implication is that ACE targets a conditioning problem in transfer learning for recommendation: it seeks a geometry that remains semantically informative while being more amenable to collaborative fine-tuning.

3. LAE formulation and spectral mechanism

ACE uses a linear autoencoder to learn an item–item operator $\mathbf{B}_\text{ACE}$ that reconstructs the original embedding matrix from itself. The main optimization objective is

$$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$

Here, the reconstruction term preserves semantic structure, while the $L_2$-regularization term is the anisotropy-control mechanism. The scalar $\lambda$ is the key control knob (Lee et al., 28 May 2026).

The paper gives the closed-form solution as

$$\hat{\mathbf{B}}_{\text{ACE}} = (\mathbf{E}\mathbf{E}^{\top}+\lambda I)^{-1}\mathbf{E}\mathbf{E}^{\top}.$$

It then analyzes the method through the singular value decomposition

$\mathbf{E}=\mathbf{U}\mathbf{S}\mathbf{V}^{\top}.$

The reshaped embedding is constructed through singular-value shrinkage: $$\mathbf{E}_\text{ACE}=\mathbf{U}\,g_\lambda(\mathbf{S}),$$ with

$\mathbf{E}\in\mathbb{R}^{n\times d}$0

The associated linear operator is

$\mathbf{E}\in\mathbb{R}^{n\times d}$1

The paper identifies this spectral form as the core of ACE: dominant singular directions are shrunk more strongly, weaker directions are relatively less distorted, and the overall effect is a controlled flattening of the spectrum (Lee et al., 28 May 2026).

For the final reduced embedding used in recommendation, the paper keeps the top-$\mathbf{E}\in\mathbb{R}^{n\times d}$2 singular directions and rescales them: $\mathbf{E}\in\mathbb{R}^{n\times d}$3 Here $\mathbf{E}\in\mathbb{R}^{n\times d}$4 is a scaling factor used to restore embedding magnitude after shrinkage (Lee et al., 28 May 2026).

4. Theoretical interpretation and integration into SR backbones

The paper’s theoretical interpretation assigns distinct roles to the two terms of the LAE objective. The reconstruction term preserves the semantic basis learned by the LLM; in the SVD view, this means retaining the principal directions $\mathbf{E}\in\mathbb{R}^{n\times d}$5. The $\mathbf{E}\in\mathbb{R}^{n\times d}$6-regularizer limits the dominance of large singular values, thereby reducing anisotropy (Lee et al., 28 May 2026).

Two limiting behaviors are explicitly noted. When $\mathbf{E}\in\mathbb{R}^{n\times d}$7, all singular components are effectively retained equally, and the reshaped embedding becomes maximally normalized in spectral terms. When $\mathbf{E}\in\mathbb{R}^{n\times d}$8 becomes very large, the shrinkage becomes weak relative to the original spectrum, so the representation moves closer to the original anisotropic geometry. On the paper’s account, ACE therefore offers continuous anisotropy control rather than a binary choice between raw embeddings and fully whitened embeddings (Lee et al., 28 May 2026).

Operationally, integration into SR is simple. The pipeline is: encode each item’s textual metadata with an LLM; apply ACE to reshape the item embedding matrix; use the resulting $\mathbf{E}\in\mathbb{R}^{n\times d}$9-dimensional embeddings to initialize the item embedding table in an SR model; and fine-tune the SR backbone on interaction sequences. The paper emphasizes that ACE changes the embedding initialization, not the downstream architecture itself (Lee et al., 28 May 2026).

This architecture-agnostic insertion point is central to its scope. ACE is intended to be compatible with established SR backbones rather than to replace them.

5. Experimental protocol and empirical results

The experiments follow a standard leave-one-out SR protocol on four benchmark datasets: Beauty and Toys from Amazon Review 2014, Yelp 2018, and ML-20M. Users and items with fewer than five interactions are filtered. Evaluation uses full-sort ranking and reports Recall@10, NDCG@10, Recall@20, and NDCG@20. Implementation is based on RecBole, with embedding dimension 128, maximum sequence length 50, batch size 256, Adam optimization, early stopping patience 10, learning rate tuned from $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$0 to $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$1, $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$2 searched over $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$3, and scaling $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$4 chosen from $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$5. Results are averaged over four runs (Lee et al., 28 May 2026).

For item initialization, the main encoder is OpenAI text-embedding-3-large. The paper also evaluates F2LLM-4B, Qwen3-Embedding-8B, and KaLM-Embedding-Gemma3-12B-2511. Baselines are five LLM-as-Extractor methods: LLM2X, WhitenRec+, LLMEmb, AlphaRec, and AlphaFuse (Lee et al., 28 May 2026).

The empirical findings are summarized below.

Setting	Exact result
SASRec / Beauty	$$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$6, $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$7
SASRec / Toys	$$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$8, $$\mathrm{Cov}(\mathbf{E})=\frac{1}{n}(\mathbf{E}-\mu)^\top(\mathbf{E}-\mu),$$9
SASRec / Yelp	$\mathbf{B}_\text{ACE}$0, $\mathbf{B}_\text{ACE}$1
SASRec / ML-20M	$\mathbf{B}_\text{ACE}$2, $\mathbf{B}_\text{ACE}$3
GRU4Rec / Beauty	$\mathbf{B}_\text{ACE}$4, $\mathbf{B}_\text{ACE}$5
BERT4Rec / Beauty	$\mathbf{B}_\text{ACE}$6, $\mathbf{B}_\text{ACE}$7
GRU4Rec / Yelp	$\mathbf{B}_\text{ACE}$8, $\mathbf{B}_\text{ACE}$9
BERT4Rec / Yelp	$$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$0, $$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$1

On SASRec, the paper reports that ACE yields up to 12.4% improvement in Recall@20 and 11.8% improvement in NDCG@20 over the strongest LLM-as-Extractor baseline. It also states that ACE consistently outperforms all baselines across datasets and backbones. Under alternative item encoders, ACE remains best on Recall@20, improving by up to 2.94% over the best baseline under each encoder (Lee et al., 28 May 2026).

The hyperparameter sweep over $$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$2 supports the paper’s central thesis. Too small a $$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$3 over-flattens the spectrum and harms semantic hierarchy; too large a $$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$4 leaves anisotropy insufficiently corrected; moderate $$\mathcal{L}(\mathbf{B}_\text{ACE}) = \left\| \mathbf{E}^\top-\mathbf{E}^\top\mathbf{B}_\text{ACE} \right\|_F^2 + \lambda\|\mathbf{B}_\text{ACE}\|_F^2.$$5 provides the best balance between uniformity and semantic preservation (Lee et al., 28 May 2026).

6. Relation to adjacent methods and acronym ambiguity

ACE belongs to a family of methods that modify transferred semantic embeddings before or during downstream recommendation. Within the paper’s comparison set, LLM2X is described as PCA-based projection, WhitenRec+ as whitening-based, LLMEmb as MLP-based semantic adaptation, AlphaRec as adaptation-oriented embedding learning, and AlphaFuse as preserving the semantic subspace while reinitializing others (Lee et al., 28 May 2026). The paper uses these comparisons to argue that full whitening can underperform even simpler PCA-based projection, which it interprets as evidence that aggressive isotropization may distort useful structure.

A common source of confusion is the acronym itself. “ACE” is also used by the unrelated paper “ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling,” which concerns generative cross-modal retrieval for text-to-image, text-to-audio, and text-to-video tasks rather than anisotropy control in embeddings (Fang et al., 2024). In the sequential recommendation context, ACE specifically denotes Anisotropy-Controllable Embedding (Lee et al., 28 May 2026).

Taken together, the paper frames ACE as a method for conditioning the geometry of LLM-derived item spaces before collaborative learning. This suggests that its contribution is not merely better initialization in a narrow empirical sense, but a more explicit treatment of spectral geometry as an adjustable interface between semantic extraction and recommendation optimization.