Incremental Affinity Propagation based on Cluster Consolidation and Stratification (2401.14439v1)

Abstract: Modern data mining applications require to perform incremental clustering over dynamic datasets by tracing temporal changes over the resulting clusters. In this paper, we propose A-Posteriori affinity Propagation (APP), an incremental extension of Affinity Propagation (AP) based on cluster consolidation and cluster stratification to achieve faithfulness and forgetfulness. APP enforces incremental clustering where i) new arriving objects are dynamically consolidated into previous clusters without the need to re-execute clustering over the entire dataset of objects, and ii) a faithful sequence of clustering results is produced and maintained over time, while allowing to forget obsolete clusters with decremental learning functionalities. Four popular labeled datasets are used to test the performance of APP with respect to benchmark clustering performances obtained by conventional AP and Incremental Affinity Propagation based on Nearest neighbor Assignment (IAPNA) algorithms. Experimental results show that APP achieves comparable clustering performance while enforcing scalability at the same time.

• The paper introduces the APP algorithm, which consolidates past clusters via centroid reduction to efficiently integrate new data points.
• It employs cluster stratification to balance stability and plasticity, tracking cluster evolution without full recomputation.
• Performance tests show APP's scalability and accuracy, with applications in semantic shift detection in computational linguistics.

Introduction

Advancements in data mining necessitate methodologies capable of handling dynamically changing datasets. One crucial area where this is evident is in incremental clustering, where objects are introduced over time, and cluster formations need to reflect these temporal changes. Affinity Propagation (AP) has been a cornerstone algorithm for static datasets, but the challenge lies in extending AP to incremental scenarios. This paper posits that conventional extension strategies are inadequate as they fail to efficiently assimilate newly arriving data points into existing cluster structures, resulting in a re-computation burden.

In response, the authors put forward an innovative approach termed A-Posteriori affinity Propagation (APP) which builds upon AP, designed specifically for dynamic datasets and aimed at resolving scalability issues while upholding the quality of clustering through strategies of cluster consolidation and stratification.

Related Work

Incremental clustering has received considerable attention in literature, modifying algorithms like k-means and AP to adapt to data streams and dynamically evolving data. These efforts often focus on lower computational costs by updating clusters with new data points instead of global recomputation. However, incremental algorithms also need to strike a delicate balance between stability, to avoid dramatic changes with each new data point, and plasticity, to integrate new information. The authors review various existing approaches, emphasizing that none adequately support both the accurate tracing of cluster evolution over time and efficient scalability, setting the stage for the introduction of APP.

A-Posteriori Affinity Propagation

APP stands out by dynamically collapsing each t-1 time step cluster into a centroid, using this simplified cluster representation to quickly assimilate new data points at time t. Cluster stratification plays a critical role—new clusters emerge, existing ones are enriched, or old clusters merge based on incoming data. Importantly, APP places emphasis on "group evolution," wherein a novel data point is more likely to augment an existing cluster than pioneer a new one, unless a significant number of similar points warrants such creation.

The authors also address the need to forget obsolete clusters, incorporating decremental learning functionalities—an aspect critical to maintaining high performance and relevancy over time. This aspect of forgetfulness introduces a practical mechanism for discarding outdated information, thereby conserving memory and enhancing scalability.

Performance Evaluation

The efficacy of APP is tested against AP and other benchmarks on several established datasets, demonstrating comparable performance while markedly improving scalability. Results indicate APP's enhanced ability to manage dynamic datasets with frequently introduced data points, producing accurate and coherent cluster histories. This is attributed to APP's innovative consolidation and stratification mechanisms.

An application to semantic shift detection in computational linguistics further underscores APP's relevance, showing promise in uncovering word meaning evolution in diachronic corpora—a pressing need in the age of digital humanities and LLMs.

Conclusion

Through the proposed framework, APP looks to revamp incremental clustering within dynamic datasets. It is a significant stride towards a scalable solution that preserves both the historic integrity of clusters and the flexibility to accrete or prune as dictated by data evolution. The algorithms' potential in semantic shift detection indicates its wider applicability across computational linguistics, opening avenues in natural language processing where data is inherently dynamic and evolving.