loading
Documents

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Mikolaj Baranowski 1 ; Adam Belloum 1 and Marian Bubak 2

Affiliations: 1 University of Amsterdam, Netherlands ; 2 University of Amsterdam and AGH University of Science and Technology Krakow PL / University of Amsterdam NL, Netherlands

ISBN: 978-989-8565-69-3

Keyword(s): Scientific Workflow, Ruby, MapReduce, Hadoop, Parallel Processing.

Related Ontology Subjects/Areas/Topics: Domain-Specific Tools ; Simulation and Modeling ; Simulation Tools and Platforms

Abstract: MapReduce frameworks proved to be a good solution for storing and processing large amounts of data. Thanks to data parallelism, they allow to move computations very close to the storage and therefore to reduce an influence of “I/O bottleneck”. Workflow Management Systems, in turn, are widely used for modeling of scientific applications. Users that are willing to use MapReduce frameworks in their workflows have to run separate environment to develop Map/Reduce operations. In this paper we propose an approach that will allow to extend existing application models by MapReduce routines. Our solution bases on DSL constructed on top of Ruby programming language. It follows examples of Sawzall and Pig Latin languages and allows to define Map/Reduce operations in minimalist way. Moreover, because the language is based on Ruby, the model allows to use user defined routines and existing Ruby libraries. A particular model of the workflow management system can be extended with our DSL letting use rs to use one environment for developing the workflow and MapReduce application. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.198.27.243

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Baranowski M., Belloum A. and Bubak M. (2013). Enhancing VLAM Workflow Model with MapReduce Operations.In Proceedings of the 3rd International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH, ISBN 978-989-8565-69-3, pages 179-185. DOI: 10.5220/0004488401790185

@conference{simultech13,
author={Mikolaj Baranowski and Adam Belloum and Marian Bubak},
title={Enhancing VLAM Workflow Model with MapReduce Operations},
booktitle={Proceedings of the 3rd International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH,},
year={2013},
pages={179-185},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0004488401790185},
isbn={978-989-8565-69-3},
}

TY - CONF

JO - Proceedings of the 3rd International Conference on Simulation and Modeling Methodologies, Technologies and Applications - Volume 1: SIMULTECH,
TI - Enhancing VLAM Workflow Model with MapReduce Operations
SN - 978-989-8565-69-3
AU - Baranowski M.
AU - Belloum A.
AU - Bubak M.
PY - 2013
SP - 179
EP - 185
DO - 10.5220/0004488401790185

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.