Semantic Field Subspace (SFS)

• SFS is a context-aware representation that models collections of embeddings as linear subspaces, preserving local semantic structures.
• It employs Grassmannian geometry and algebraic set operations (union, intersection, complement) to capture internal variability and compositional semantics.
• SFS enhances NLP tasks like sentence similarity and set retrieval through efficient basis construction and interpretable, context-driven embeddings.

A Semantic Field Subspace (SFS) is a geometry-preserving, context-aware representation that captures the local semantic structure of sets or groups of tokens within a high-dimensional embedding space. Unlike conventional single-vector representations of words or sentences, SFS models collections of embedding vectors as linear subspaces, capturing both the internal variability and the combinatorial logic (union, intersection, complement) inherent to sets of semantic content. The formalism leverages algebraic and geometric methods—particularly from the theory of Grassmannians and subspace lattice logic—to provide a principled structure for semantic analysis, compositionality, interpretability, and efficient computation across modalities (Sun et al., 30 Nov 2025, Ishibashi et al., 2022, Wang et al., 2020, Manin et al., 2016).

1. Mathematical Definition and Construction

Given a finite set of vectors (such as the word embeddings $\{w_1, ..., w_n\}$ for a sentence or topic set), the associated Semantic Field Subspace $S_A$ is defined as their span:

$$S_A = \mathrm{span}\{\vec{w}_1, \dots, \vec{w}_n\} \subset \mathbb{R}^d$$

where $\vec{w}_i \in \mathbb{R}^d$ is the pre-trained embedding of token $w_i$ (Ishibashi et al., 2022). Computationally, $S_A$ is represented by constructing an $n \times d$ matrix $W_A$ whose rows are the word vectors, followed by an orthonormalization (using QR or SVD) to produce a matrix $U_A$ whose rows form a basis for the subspace. The projection operator onto $S_A$ is $P_A = U_A^\top U_A$, with $P_A^2 = P_A$ and $P_A^\top = P_A$ (Ishibashi et al., 2022).

Cluster-based SFS construction (as used in S3E) begins by partitioning the word embedding vocabulary into $K$ clusters $\{G_1, ..., G_K\}$ using weighted K-means. Each group $G_i$ is associated with a centroid $g_i \in \mathbb{R}^d$, and words are assigned residuals against their group centroids. For a new sentence, intra-group descriptors $v_i$ capture the projection of the sentence onto each SFS, and the inter-group covariance matrix $C$ models higher-order interactions among these fields (Wang et al., 2020).

Geometrically, an SFS of dimension $k$ is a point in the Grassmannian $Gr(k, V)$, the manifold of $k$-dimensional subspaces of an ambient vector space $V$ (Manin et al., 2016).

2. Subspace Set Operations and Membership

The subspace representation endows SFS with an algebra akin to classical set theory but realized in the linear algebraic structure of $\mathbb{R}^d$ (Ishibashi et al., 2022):

• Union ($S_1 \cup S_2$): The smallest subspace containing both $S_1$ and $S_2$; constructed as the span of their combined basis vectors. The projection operator is $P_{S_1+S_2} = P_1 + P_2 - P_1 P_2$.
• Intersection ($S_1 \cap S_2$): The subspace of vectors common to both; projector is $P_{S_1 \cap S_2} = P_1 P_2$.
• Complement: The orthogonal complement $S_A^\perp$, with projector $P_{S_A^{\perp}} = I - P_A$.
• Soft Membership: For a candidate vector $w$, its degree of membership in $S_A$ is $\mu_A(w) = \|P_A w\|^2 / \|w\|^2$. This generalizes set membership to $\mu_A(w) \in [0,1]$, representing the squared cosine of the minimal angle between $w$ and $S_A$.

The subspace lattice structure guarantees compatibility with modular and De Morgan laws, allowing compositional semantics and logical operations to be performed directly on sets of embeddings (Ishibashi et al., 2022).

3. Embedding Families of Subspaces: Grassmannians and Plücker Embedding

SFSs naturally inhabit Grassmannian manifolds $Gr(k, V)$, the parameter space of all $k$-dimensional linear subspaces of $V$. This geometric perspective provides a rigorous foundation for comparing, aligning, and interpolating semantic subspaces (Manin et al., 2016).

• Plücker Embedding: Each $k$-plane $W$ is mapped to a point in projective space $P(\wedge^k V)$ via $w_1 \wedge ... \wedge w_k$, with coordinates (Plücker coordinates) representing $k \times k$ minors.
• Subspace Flow: Latent Semantic Analysis (LSA) and related factorization methods can be viewed as a flow on $Gr(k, V)$, converging to dominant semantic directions via iterative updates on the projector $P(t)$.

The structure of Grassmannians enables the formulation of semantic alignment tasks—such as Gärdenfors's "meeting of minds"—as geometric barycenter or clustering problems among subspaces (Manin et al., 2016).

4. Application: Soft Similarity, Set Retrieval, and Sentence Modeling

SFS representations facilitate a range of NLP tasks by capturing the nuanced geometry of sets of tokens or concepts (Ishibashi et al., 2022, Wang et al., 2020):

• Sentence Similarity: Sentences or texts are embedded as SFS, enabling the comparison of complex semantic content. The analogues of recall ($R$), precision ($P$), and $F_1$ score are defined directly from soft membership averages with respect to subspace projectors.
• Set Retrieval and Expansion: The union and intersection operators allow retrieval of concept sets with controlled semantic specificity or generality.
• Sentence Embedding (S3E): The intra-group and inter-group covariance descriptors model both the content within semantic fields and their interrelations. The final embedding is constructed from the vectorization of the upper-triangular part of the covariance matrix, followed by $\ell_2$ normalization.

Empirical evidence shows that SFS-based set operations outperform vector-based aggregation for both sentence similarity (STS Benchmark: Spearman $\rho$ raised from 0.506 to 0.526 unweighted; WMT18: F-score $\tau$ from 0.365 to 0.372) and topic-set retrieval (subspace-based R@100 = 35.7% vs. fuzzy set 30.9%; median rank 246 vs. 320) (Ishibashi et al., 2022).

5. Computational and Algorithmic Aspects

SFS operations are amenable to efficient implementation:

• Basis Construction: Orthonormalization by QR or SVD for $n$-word sets in $d$-space costs $O(nd^2)$ or $O(n^2d)$.
• Soft Membership and Projection: For established basis $U_A$ ($r \ll d$), projection costs $O(rd)$.
• Union/Intersection: Union formed by concatenating and orthonormalizing bases ($O((r_1+r_2)^2 d)$); intersection via principal angles from SVD ($O(r_1 r_2 \min(r_1, r_2))$).
• S3E Complexity: The per-sentence cost is $O(Nd + K^2d)$, dominated by covariance computation, with typical K and d yielding millisecond inference (Wang et al., 2020).

This computational efficiency enables practical deployment for large-scale retrieval, classification, and similarity evaluation tasks.

6. Interpretability, Generalization, and Theoretical Implications

Modeling semantic content as subspaces enhances interpretability by:

• Preserving the internal structure and compositionality of sets (e.g., words, tokens, topics).
• Allowing the explicit modeling of semantic hierarchies and local neighborhoods.
• Providing field-level analysis (intra-group projection) and cross-field relations (inter-group covariance) (Wang et al., 2020).

The framework generalizes seamlessly to multilingual embeddings, contextualized token embeddings, and even cross-modal data, provided the mapping from instances to embedding vectors is well-defined (Wang et al., 2020, Ishibashi et al., 2022).

A plausible implication is that SFS can serve as a unifying mathematical structure for both symbolic and distributional semantics, bridging the gap between logical set-theory and continuous vector space models.

7. Empirical Performance and Adoption

Empirical evaluation across multiple text and image datasets demonstrates that SFS-based approaches robustly outperform traditional classifiers in tasks requiring nuanced semantic differentiation—not only in classification but also in fine-grained applications such as political bias detection and multilingual sentence similarity (Sun et al., 30 Nov 2025, Ishibashi et al., 2022).

The ability of SFS frameworks to expose semantic hierarchies, support scalable set operations, and provide interpretable similarity metrics has motivated their adoption for structuring and analyzing embedding spaces in state-of-the-art NLP pipelines.