loading
Papers

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Rui C. Gonçalves 1 ; José Pereira 1 and Ricardo Jimenez-Peris 2

Affiliations: 1 INESC TEC & U. Minho, Portugal ; 2 LeanXcale, Spain

ISBN: 978-989-758-182-3

Keyword(s): Shuffling, Analytical Processing, Middleware, RDMA.

Abstract: A key component in a distributed parallel analytical processing engine is shuffling, the distribution of data to multiple nodes such that the computation can be done in parallel. In this paper we describe the initial design of a communication middleware to support asynchronous shuffling of data among multiple processes on a distributed memory environment. The proposed middleware relies on RDMA (Remote Direct Memory Access) operations to transfer data, and provides basic operations to send and queue data on remote machines, and to retrieve this queued data. Preliminary results show that the RDMA-based middleware can provide a 75% reduction on communication costs, when compared with a traditional sockets implementation.

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.233.226.151

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Gonçalves, R.; Pereira, J. and Jimenez-Peris, R. (2016). Design of an RDMA Communication Middleware for Asynchronous Shuffling in Analytical Processing.In Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: DataDiversityConvergence, (CLOSER 2016) ISBN 978-989-758-182-3, pages 348-351. DOI: 10.5220/0005923703480351

@conference{datadiversityconvergence16,
author={Rui C. Gon\c{C}alves. and José Pereira. and Ricardo Jimenez{-}Peris.},
title={Design of an RDMA Communication Middleware for Asynchronous Shuffling in Analytical Processing},
booktitle={Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: DataDiversityConvergence, (CLOSER 2016)},
year={2016},
pages={348-351},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005923703480351},
isbn={978-989-758-182-3},
}

TY - CONF

JO - Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: DataDiversityConvergence, (CLOSER 2016)
TI - Design of an RDMA Communication Middleware for Asynchronous Shuffling in Analytical Processing
SN - 978-989-758-182-3
AU - Gonçalves, R.
AU - Pereira, J.
AU - Jimenez-Peris, R.
PY - 2016
SP - 348
EP - 351
DO - 10.5220/0005923703480351

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.