Authors:
Diogo Silva
1
;
Helena Aidos
2
and
Ana Fred
2
Affiliations:
1
Portuguese Air Force Academy, Portugal
;
2
Instituto Superior Técnico, Portugal
Keyword(s):
Clustering Methods, EAC, K-Means, MST, GPGPU, CUDA, Sparse Matrices, Single-Link.
Related
Ontology
Subjects/Areas/Topics:
Clustering
;
Ensemble Methods
;
Pattern Recognition
;
Sparsity
;
Theory and Methods
Abstract:
The unprecedented collection and storage of data in electronic format has given rise to an interest in automated analysis for generation of knowledge and new insights. Cluster analysis is a good candidate since it makes as few assumptions about the data as possible. A vast body of work on clustering methods exist, yet, typically, no single method is able to respond to the specificities of all kinds of data. Evidence Accumulation Clustering (EAC) is a robust state of the art ensemble algorithm that has shown good results. However, this robustness comes with higher computational cost. Currently, its application is slow or restricted to small datasets. The objective of the present work is to scale EAC, allowing its applicability to big datasets, with technology available at a typical workstation. Three approaches for different parts of EAC are presented: a parallel GPU K-Means implementation, a novel strategy to build a sparse CSR matrix specialized to EAC and Single-Link based on Minim
um Spanning Trees using an external memory sorting algorithm. Combining these approaches, the application of EAC to much larger datasets than before was accomplished.
(More)