Robust Orthogonal NMF with Label Propagation for Image Clustering (2504.21472v1)

Abstract: Non-negative matrix factorization (NMF) is a popular unsupervised learning approach widely used in image clustering. However, in real-world clustering scenarios, most existing NMF methods are highly sensitive to noise corruption and are unable to effectively leverage limited supervised information. To overcome these drawbacks, we propose a unified non-convex framework with label propagation called robust orthogonal nonnegative matrix factorization (RONMF). This method not only considers the graph Laplacian and label propagation as regularization terms but also introduces a more effective non-convex structure to measure the reconstruction error and imposes orthogonal constraints on the basis matrix to reduce the noise corruption, thereby achieving higher robustness. To solve RONMF, we develop an alternating direction method of multipliers (ADMM)-based optimization algorithm. In particular, all subproblems have closed-form solutions, which ensures its efficiency. Experimental evaluations on eight public image datasets demonstrate that the proposed RONMF outperforms state-of-the-art NMF methods across various standard metrics and shows excellent robustness. The code will be available at https://github.com/slinda-liu.

• The paper presents a novel robust orthogonal NMF framework that integrates non-convex loss functions and orthogonal constraints, enhancing noise resilience in image clustering.
• It employs label propagation alongside graph Laplacian regularization to effectively utilize both labeled and unlabeled data, boosting clustering performance.
• Experimental results demonstrate that RONMF consistently outperforms state-of-the-art methods, with significant improvements in metrics like accuracy, F1-score, and NMI on noisy datasets.

Robust Orthogonal NMF with Label Propagation for Image Clustering

Introduction

The paper "Robust Orthogonal NMF with Label Propagation for Image Clustering" (2504.21472) proposes an innovative approach to improving non-negative matrix factorization (NMF) methods in the context of image clustering. Traditional NMF methods often struggle with noise sensitivity and fail to effectively utilize limited labeled information. This research introduces a new framework, robust orthogonal NMF (RONMF), which leverages label propagation and structured non-convex optimization to enhance clustering robustness and performance. The method integrates the use of graph Laplacians, orthogonal constraints, and non-convex loss functions to deliver superior image clustering results.

Methodology

The proposed RONMF framework builds upon existing NMF techniques by incorporating several key innovations:

1. Non-Convex Structure: Unlike classical NMF which often employs the Frobenius norm, the RONMF method uses non-convex functions such as MCP, SCAD, or ETP to measure reconstruction error. This approach enhances the model's resilience to noise and outliers by providing more flexible feature selection.
2. Orthogonal Constraints: Imposing orthogonal constraints on the basis matrix improves feature discriminability and robustness by reducing redundancy among basis vectors. This is particularly effective in high-dimensional spaces, where it aids in preserving the intrinsic geometric structure of the data.
3. Label Propagation: The integration of label propagation into the NMF framework allows the method to utilize limited supervised information more effectively. This is achieved by considering both labeled and unlabeled data, allowing the model to predict and propagate labels based on data geometry.
4. Graph Laplacians as Regularization: The use of graph Laplacian regularization helps in capturing the geometric relationships between data points, further enhancing clustering accuracy.

Optimization Algorithm

The paper employs an alternating direction method of multipliers (ADMM)-based optimization algorithm to solve the RONMF problem. Each subproblem within this framework has a closed-form solution, ensuring computational efficiency. The algorithm iteratively updates variables by minimizing the loss function while adhering to the imposed constraints.

(Figure 1)

Figure 1: A comparison of the learning behavior of RONMF variants, showing effective learning and robust final scores across different datasets.

Experimental Results

Experimental evaluations were conducted on several benchmark image datasets, demonstrating the effectiveness of the RONMF method. The results show that RONMF consistently outperforms state-of-the-art methods across various metrics, including accuracy (ACC), F1-score, normalized mutual information (NMI), and purity (PUR).

• On datasets with high noise levels, RONMF maintained robust performance, confirming its noise-resilience attributes.
• Across all tested datasets, RONMF variants exhibited superior clustering results, with particularly notable improvements in environments with limited labeled data.

  Figure 2: All metrics for different clustering numbers $k$ on Type-I datasets, highlighting the consistent performance improvements across various scenarios.

Conclusions

The RONMF method represents a significant advancement in the field of image clustering by effectively addressing the limitations of traditional NMF techniques. Its integration of non-convex loss functions, orthogonal constraints, and label propagation facilitates superior image clustering under noisy conditions and with scarce labeled data. As indicated by experimental results, RONMF offers a robust and efficient solution for image clustering tasks.

Future work could explore the integration of deep learning architectures to further enhance the method's capabilities, particularly in handling increasingly complex and large-scale image datasets. Additionally, investigating the application of RONMF in real-time or online clustering scenarios could be an interesting avenue for research development.