loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Emmanouil Pavlidakis ; Stelios Mavridis ; Giorgos Saloustros and Angelos Bilas

Affiliation: Foundation for Research and Technology – Hellas (FORTH), Greece

Keyword(s): Distributed File Systems, NoSQL Data Stores, Key-value Stores, HBase, HDFS.

Abstract: Recently, NoSQL stores, such as HBase, have gained acceptance and popularity due to their ability to scale-out and perform queries over large amounts of data. NoSQL stores typically arrange data in tables of (key,value) pairs and support few simple operations: get, insert, delete, and scan. Despite its simplicity, this API has proven to be extremely powerful. Nowadays most data analytics frameworks utilize distributed file systems (DFS) for storing and accessing data. HDFS has emerged as the most popular choice due to its scalability. In this paper we explore how popular NoSQL stores, such as HBase, can provide an HDFS scale-out file system abstraction. We show how we can design an HDFS compliant filesystem on top a key-value store. We implement our design as a user-space library (KVFS) providing an HDFS filesystem over an HBase key-value store. KVFS is designed to run Hadoop style analytics such as MapReduce, Hive, Pig and Mahout over NoSQL stores without the use of HDFS. We perform a preliminary evaluation of KVFS against a native HDFS setup using DFSIO with varying number of threads. Our results show that the approach of providing a filesystem API over a key-value store is a promising direction: Read and write throughput of KVFS and HDFS, for big and small datasets, is identical. Both HDFS and KVFS throughput is limited by the network for small datasets and from the device I/O for bigger datasets. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.236.226.100

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Pavlidakis, E.; Mavridis, S.; Saloustros, G. and Bilas, A. (2016). KVFS: An HDFS Library over NoSQL Databases. In Proceedings of the 6th International Conference on Cloud Computing and Services Science (CLOSER 2016) - Volume 1: DataDiversityConvergence; ISBN 978-989-758-182-3; ISSN 2184-5042, SciTePress, pages 360-367. DOI: 10.5220/0005924003600367

@conference{datadiversityconvergence16,
author={Emmanouil Pavlidakis. and Stelios Mavridis. and Giorgos Saloustros. and Angelos Bilas.},
title={KVFS: An HDFS Library over NoSQL Databases},
booktitle={Proceedings of the 6th International Conference on Cloud Computing and Services Science (CLOSER 2016) - Volume 1: DataDiversityConvergence},
year={2016},
pages={360-367},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005924003600367},
isbn={978-989-758-182-3},
issn={2184-5042},
}

TY - CONF

JO - Proceedings of the 6th International Conference on Cloud Computing and Services Science (CLOSER 2016) - Volume 1: DataDiversityConvergence
TI - KVFS: An HDFS Library over NoSQL Databases
SN - 978-989-758-182-3
IS - 2184-5042
AU - Pavlidakis, E.
AU - Mavridis, S.
AU - Saloustros, G.
AU - Bilas, A.
PY - 2016
SP - 360
EP - 367
DO - 10.5220/0005924003600367
PB - SciTePress