Authors: Mohamad Jaber ; Mohamed Nassar ; Wael Al Rahal Al Orabi ; Bilal Abi Farraj ; Mohamad Omar Kayali and Chadi Helwe

Affiliation: American University of Beirut, Lebanon

ISBN: 978-989-758-243-1

Keyword(s): Big Data, Spark, Hadoop, Quality of Service, Cloud.

Abstract: The contribution of this paper is two-fold. First, we propose a Domain Specific Language (DSL) to easily reconfigure and compose Spark applications. For each Spark application we define its input and output interfaces. Then, given a set of connections that map outputs of some Spark applications to free inputs of other Spark applications, we automatically embed Spark applications with the required synchronization and communication to properly run them according to the user-defined mapping. Second, we present an adaptive quality management/selection method for Spark applications. The method takes as input a pipeline of parameterized Spark applications, where the execution time of each Spark application is an unknown increasing function of quality level parameters. The method builds a controller that automatically computes adequate quality for each Spark application to meet a user-defined deadline. Consequently, users can submit a pipeline of Spark applications and a deadline, our method automatically runs all the Spark applications with the maximum quality while respecting the deadline specified by the user. We present experimental results showing the effectiveness of our method. (More)

PDF ImageFull Text

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Jaber, M.; Nassar, M.; Al Rahal Al Orabi, W.; Abi Farraj, B.; Kayali, M. and Helwe, C. (2017). Reconfigurable and Adaptive Spark Applications.In Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-243-1, pages 112-119. DOI: 10.5220/0006289901120119

author={Mohamad Jaber. and Mohamed Nassar. and Wael Al Rahal Al Orabi. and Bilal Abi Farraj. and Mohamad Omar Kayali. and Chadi Helwe.},
title={Reconfigurable and Adaptive Spark Applications},
booktitle={Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},


JO - Proceedings of the 7th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - Reconfigurable and Adaptive Spark Applications
SN - 978-989-758-243-1
AU - Jaber, M.
AU - Nassar, M.
AU - Al Rahal Al Orabi, W.
AU - Abi Farraj, B.
AU - Kayali, M.
AU - Helwe, C.
PY - 2017
SP - 112
EP - 119
DO - 10.5220/0006289901120119

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.