loading
Documents

Research.Publish.Connect.

Paper

Authors: Jiangtao Zhang 1 ; Lingmin Zhang 2 ; Hejiao Huang 2 ; Zeo L. Jiang 3 and Xuan Wang 3

Affiliations: 1 Harbin Institute of Technology Shenzhen Graduate School and Public Service Platform of Mobile Internet Application Security Industry, China ; 2 Harbin Institute of Technology Shenzhen Graduate School and Shenzhen Key Laboratory of Internet of Information Collaboration, China ; 3 Harbin Institute of Technology Shenzhen Graduate School and Shenzhen Applied Technology Engineering Laboratory for Internet Multimedia Application, China

ISBN: 978-989-758-183-0

Keyword(s): Reducer Placement, Resource Provision, Hadoop Across Data Centers, Distributed Cloud.

Abstract: Due to the distribution characteristic of the data source, such as astronomy and sales, or the legal prohibition, it is not always practical to store the world-wide data in only one data center (DC). Hadoop is a commonly accepted framework for big data analytics. But it can only deal with data within one DC. The distribution of data necessitates the study of Hadoop across DCs. In this situation, though we can place mapper in the local DCs, where to place reducers is a great challenge, since each reducer almost needs to process all map output across all involved DCs. Aiming to reduce costs, a key based scheme is proposed which can respect the locality principle of traditional Hadoop as much as possible while realizing deployment of reducers with lower cost. Considering both data center level and server level resource provision, a bi-level programming is used to formalize the problem and it is solved by a tailored two level group genetic algorithm (TLGGA). Extensive simulations demonstr ate the effectiveness of TLGGA. It can outperform both the baseline and the state-of-the-art mechanisms by 49% and 40%, respectively. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.80.208.105

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Zhang J., Zhang L., Huang H., Jiang Z. and Wang X. (2016). Key based Reducer Placement for Data Analytics across Data Centers Considering Bi-level Resource Provision in Cloud Computing.In Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD, ISBN 978-989-758-183-0, pages 243-254. DOI: 10.5220/0005894202430254

@conference{iotbd16,
author={Jiangtao Zhang and Lingmin Zhang and Hejiao Huang and Zeo L. Jiang and Xuan Wang},
title={Key based Reducer Placement for Data Analytics across Data Centers Considering Bi-level Resource Provision in Cloud Computing},
booktitle={Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD,},
year={2016},
pages={243-254},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005894202430254},
isbn={978-989-758-183-0},
}

TY - CONF

JO - Proceedings of the International Conference on Internet of Things and Big Data - Volume 1: IoTBD,
TI - Key based Reducer Placement for Data Analytics across Data Centers Considering Bi-level Resource Provision in Cloud Computing
SN - 978-989-758-183-0
AU - Zhang J.
AU - Zhang L.
AU - Huang H.
AU - Jiang Z.
AU - Wang X.
PY - 2016
SP - 243
EP - 254
DO - 10.5220/0005894202430254

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.