loading
Documents

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Marco Cavallo ; Lorenzo Cusma' ; Giuseppe Di Modica ; Carmelo Polito and Orazio Tomarchio

Affiliation: University of Catania, Italy

ISBN: 978-989-758-182-3

Keyword(s): Big Data, Mapreduce, Hierarchical Hadoop, Context Awareness, Integer Partitioning.

Related Ontology Subjects/Areas/Topics: Big Data Cloud Services ; Cloud Computing ; Cloud Computing Architecture ; Fundamentals ; Platforms and Applications

Abstract: In many application fields such as social networks, e-commerce and content delivery networks there is a constant production of big amounts of data in geographically distributed sites that need to be timely elaborated. Distributed computing frameworks such as Hadoop (based on the MapReduce paradigm) have been used to process big data by exploiting the computing power of many cluster nodes interconnected through high speed links. Unfortunately, Hadoop was proved to perform very poorly in the just mentioned scenario. We designed and developed a Hadoop framework that is capable of scheduling and distributing hadoop tasks among geographically distant sites in a way that optimizes the overall job performance. We propose a hierarchical approach where a top-level entity, by exploiting the information concerning the data location, is capable of producing a smart schedule of low-level, independent MapReduce sub-jobs. A software prototype of the framework was developed. Tests run on the prototyp e showed that the job scheduler makes good forecasts of the expected job’s execution time. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.87.61.215

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Cavallo, M.; Cusma', L.; Cusma', L.; Di Modica, G.; Di Modica, G.; Polito, C.; Polito, C. and Tomarchio, O. (2016). A Hadoop based Framework to Process Geo-distributed Big Data.In Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER, ISBN 978-989-758-182-3, pages 178-185. DOI: 10.5220/0005806101780185

@conference{closer16,
author={Marco Cavallo. and Lorenzo Cusma'. and Lorenzo Cusma'. and Giuseppe Di Modica. and Giuseppe Di Modica. and Carmelo Polito. and Carmelo Polito. and Orazio Tomarchio.},
title={A Hadoop based Framework to Process Geo-distributed Big Data},
booktitle={Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,},
year={2016},
pages={178-185},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005806101780185},
isbn={978-989-758-182-3},
}

TY - CONF

JO - Proceedings of the 6th International Conference on Cloud Computing and Services Science - Volume 1: CLOSER,
TI - A Hadoop based Framework to Process Geo-distributed Big Data
SN - 978-989-758-182-3
AU - Cavallo, M.
AU - Cusma', L.
AU - Cusma', L.
AU - Di Modica, G.
AU - Di Modica, G.
AU - Polito, C.
AU - Polito, C.
AU - Tomarchio, O.
PY - 2016
SP - 178
EP - 185
DO - 10.5220/0005806101780185

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.