loading
Documents

Research.Publish.Connect.

Paper

Authors: François Waeselynck and Benoit Pelletier

Affiliation: Atos, 1 rue de Provence, Echirolles and France

ISBN: 978-989-758-365-0

ISSN: 2184-5042

Keyword(s): CloudDBAppliance, Bullsequana S, Scale-up, Spark, NUMA.

Abstract: Within the scope of the CloudDBAppliance project, we investigate how Apache SparkTM can leverage a many cores and large memory platform, with a scale up approach as opposed to the commonly used scale out one: that is, the approach is to deploy a spark cluster to few large servers with many cores (up to several hundreds) and large memory (up to several tera-byte), rather than spreading it on many vanilla servers, and to stack several Spark executor processes per cluster node when running a job. It requires to cope with the non-uniform memory access within such servers, so we inculcate NUMA awareness to Spark, that provides a smart and application transparent placement of executor processes. We experiment it on a BullSequanaTM S series platform with the Intel HiBench suite benchmark and compare performance where NUMA awareness is off or on.

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 34.201.9.19

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Waeselynck, F. and Pelletier, B. (2019). A NUMA Aware SparkTM on Many-cores and Large Memory Servers.In Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: ADITCA, ISBN 978-989-758-365-0, ISSN 2184-5042, pages 648-653. DOI: 10.5220/0007905506480653

@conference{aditca19,
author={Fran\c{C}ois Waeselynck. and Benoit Pelletier.},
title={A NUMA Aware SparkTM on Many-cores and Large Memory Servers},
booktitle={Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: ADITCA,},
year={2019},
pages={648-653},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007905506480653},
isbn={978-989-758-365-0},
}

TY - CONF

JO - Proceedings of the 9th International Conference on Cloud Computing and Services Science - Volume 1: ADITCA,
TI - A NUMA Aware SparkTM on Many-cores and Large Memory Servers
SN - 978-989-758-365-0
AU - Waeselynck, F.
AU - Pelletier, B.
PY - 2019
SP - 648
EP - 653
DO - 10.5220/0007905506480653

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.