Self-Organizing Maps with Optimized Latent Positions

Abstract: Self-Organizing Maps (SOM) are a classical method for unsupervised learning, vector quantization, and topographic mapping of high-dimensional data. However, existing SOM formulations often involve a trade-off between computational efficiency and a clearly defined optimization objective. Objective-based variants such as Soft Topographic Vector Quantization (STVQ) provide a principled formulation, but their neighborhood-coupled computations become expensive as the number of latent nodes increases. In this paper, we propose Self-Organizing Maps with Optimized Latent Positions (SOM-OLP), an objective-based topographic mapping method that introduces a continuous latent position for each data point. Starting from the neighborhood distortion of STVQ, we construct a separable surrogate local cost based on its local quadratic structure and formulate an entropy-regularized objective based on it. This yields a simple block coordinate descent scheme with closed-form updates for assignment probabilities, latent positions, and reference vectors, while guaranteeing monotonic non-increase of the objective and retaining linear per-iteration complexity in the numbers of data points and latent nodes. Experiments on a synthetic saddle manifold, scalability studies on the Digits and MNIST datasets, and 16 benchmark datasets show that SOM-OLP achieves competitive neighborhood preservation and quantization performance, favorable scalability for large numbers of latent nodes and large datasets, and the best average rank among the compared methods on the benchmark datasets.

• The paper introduces SOM-OLP, which optimizes latent positions via block-coordinate descent to achieve a monotonic decrease in a unified objective.
• It combines entropy-regularized soft assignments with continuous latent representations, overcoming grid rigidity and reducing computational cost.
• Empirical evaluations show superior neighborhood preservation, quantization fidelity, and scalability on synthetic and real-world datasets.

Self-Organizing Maps with Optimized Latent Positions: Technical Analysis

Motivation and Objectives

Self-Organizing Maps (SOMs) are established for vector quantization and topographic mapping, but classical approaches such as sequential SOM and Batch SOM (BSOM) lack a unified smooth optimization objective and are constrained to discrete latent representations tied to predefined node coordinates. Objective-driven approaches, such as Soft Topographic Vector Quantization (STVQ), introduce explicit neighborhood-preserving objectives with entropy regularization but suffer from $O(NM^2)$ cost due to neighborhood coupling among nodes. Generative Topographic Mapping (GTM) provides continuous latent spaces through probabilistic modeling but is computationally intensive and grid-constrained.

This paper introduces SOM-OLP (Self-Organizing Maps with Optimized Latent Positions), which formulates an objective-based topographic mapping with continuous latent positions per data point, constructed from a surrogate local cost inspired by the quadratic structure of STVQ neighborhood distortion and entropy-regularized assignment. SOM-OLP achieves monotonic non-increase in the objective under block coordinate descent (BCD), closed-form updates, and $O(NM)$ per-iteration complexity, avoiding explicit node-node coupling and grid rigidity.

Methodological Framework

SOM-OLP introduces a continuous latent position $\bm{v}_i$ for each data point $\bm{x}_i$, minimizing a locally quadratic surrogate of the neighborhood distortion. The objective incorporates two terms: (1) Data-space distortion (quantization error), and (2) latent-space proximity, regularized by an assignment entropy:

$$J_{\text{SOM-OLP}}=\sum_{i=1}^N\sum_{j=1}^M p_{ij} \left(\|\bm{x}_i-\bm{w}_j\|^2 + \gamma\|\bm{v}_i-\bm{r}_j\|^2\right) + \lambda \sum_{i=1}^N \sum_{j=1}^M p_{ij}\ln p_{ij}$$

where $\bm{w}_j$ are reference vectors, $\bm{r}_j$ are latent node coordinates, $p_{ij}$ are assignment probabilities, $\lambda$ is the entropy regularization parameter, and $\gamma$ balances proximity versus distortion.

Block coordinate updates admit closed forms:

• Assignment probabilities ($O(NM)$0) via softmax over combined distortion and proximity;
• Latent positions ($O(NM)$1) as weighted averages of node coordinates;
• Reference vectors ($O(NM)$2) as weighted centroids of assigned data.

This architecture encompasses entropy-regularized fuzzy $O(NM)$3-means ($O(NM)$4) and $O(NM)$5-means ($O(NM)$6) as degenerate cases.

Empirical Evaluation

Numerical studies spanned synthetic and real-world datasets. Key evaluation metrics included Trustworthiness (TW), Continuity (CN), and Quantization Error (QE), quantifying neighborhood preservation and distortion.

Saddle Dataset: Topographic Preservation

SOM-OLP demonstrated superior TW and CN, slightly trailing STVQf in QE, with competitive iteration and per-iteration runtime.

Figure 1: Comparison of BSOM, STVQf, GTM, and SOM-OLP on the saddle dataset showing topology preservation and latent representation.

Digits Dataset: Flexibility and Scalability

SOM-OLP outperformed in CN and QE versus BSOM, STVQf, and GTM, delivering flexible latent embeddings that adapt to local data density.

Figure 2: Latent representations of the Digits dataset highlighting SOM-OLP’s adaptive continuous latent positions.

Scalability tests showed SOM-OLP’s ability to scale up to $O(NM)$7 latent nodes without out-of-memory errors, exceeding classical and objective-based counterparts in efficiency for large $O(NM)$8.

MNIST: Large-Scale Mapping

SOM-OLP mapped $O(NM)$9 MNIST images on a $\bm{v}_i$0 grid, generating cluster-structured latent spaces at practical computational cost, indicative of its robustness for high-dimensional and high-cardinality regimes.

Figure 3: MNIST latent positions via SOM-OLP, colored by digit class, showing spatial organization across classes.

Comprehensive Benchmarking

On 16 UCI and real-world datasets, SOM-OLP achieved the best average rank in neighborhood preservation (TW+CN)/2, outperforming PCA, BSOM, STVQf, and GTM in aggregate. The critical-difference statistical analysis affirmed SOM-OLP’s consistent superiority.

Figure 4: Critical-difference diagram for average ranks of neighborhood preservation measures across benchmark datasets.

Theoretical and Practical Implications

SOM-OLP’s design, which eliminates explicit node-node coupling and grid-only representations, provides enhanced flexibility and scalability. The block-separable objective and linear complexity facilitate applications to large-scale, high-dimensional settings previously inaccessible to STVQ or GTM due to prohibitive costs.

Graph-Laplacian-style regularization inherent in the objective promotes topographically consistent assignments in data and latent spaces, positioning SOM-OLP as a robust reference-vector framework with combined neighborhood and quantization preservation, without recourse to deep architectures or generative modeling.

SOM-OLP is theoretically significant for bridging vector quantization and manifold learning, offering an interpretable, shallow alternative to recent deep variants (SOM-VAE, Topological Autoencoders) and generative approaches. Its block-coordinate closed-form updates and monotonic convergence guarantee adaptivity and stability.

Future Directions

Future work should rigorously compare SOM-OLP to global and local manifold learning algorithms (e.g., t-SNE, UMAP, Isomap), extend theoretical analyses of surrogate cost and latent position optimization, and develop principled hyperparameter selection strategies. Integration with deep architectures or persistent-homology may augment representation power or provide richer regularization.

Conclusion

SOM-OLP advances objective-based topographic mapping by unifying continuous latent positions, entropy-regularized soft assignments, and scalable block-coordinate optimization. Empirical results across synthetic and real-world datasets validate SOM-OLP’s superior neighborhood preservation, quantization fidelity, and scalability, with closed-form objective guarantees and interpretability. Its framework is promising for high-throughput, high-dimensional clustering, visualization, and manifold analysis applications in computational biology, pattern recognition, and machine learning.