Efficient Approaches for Solving the Large-Scale k-medoids Problem

Alessio Martino, Antonello Rizzi, Fabio Massimo Frattale Mascioli

2017

Abstract

In this paper, we propose a novel implementation for solving the large-scale k-medoids clustering problem. Conversely to the most famous k-means, k-medoids suffers from a computationally intensive phase for medoids evaluation, whose complexity is quadratic in space and time; thus solving this task for large datasets and, specifically, for large clusters might be unfeasible. In order to overcome this problem, we propose two alternatives for medoids update, one exact method and one approximate method: the former based on solving, in a distributed fashion, the quadratic medoid update problem; the latter based on a scan and replacement procedure. We implemented and tested our approach using the Apache Spark framework for parallel and distributed processing on several datasets of increasing dimensions, both in terms of patterns and dimensionality, and computational results show that both approaches are efficient and effective, able to converge to the same solutions provided by state-of-the-art k-medoids implementations and, at the same time, able to scale very well as the dataset size and/or number of working units increase.

Paper Citation

in Harvard Style

Martino A., Rizzi A. and Frattale Mascioli F. (2017). Efficient Approaches for Solving the Large-Scale k-medoids Problem.In Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI, ISBN 978-989-758-274-5, pages 338-347. DOI: 10.5220/0006515003380347

in Bibtex Style

@conference{ijcci17,
author={Alessio Martino and Antonello Rizzi and Fabio Massimo Frattale Mascioli},
title={Efficient Approaches for Solving the Large-Scale k-medoids Problem},
booktitle={Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,},
year={2017},
pages={338-347},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006515003380347},
isbn={978-989-758-274-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 9th International Joint Conference on Computational Intelligence - Volume 1: IJCCI,
TI - Efficient Approaches for Solving the Large-Scale k-medoids Problem
SN - 978-989-758-274-5
AU - Martino A.
AU - Rizzi A.
AU - Frattale Mascioli F.
PY - 2017
SP - 338
EP - 347
DO - 10.5220/0006515003380347