loading
Papers

Research.Publish.Connect.

Paper

Authors: Diego García-Gil ; Alejandro Alcalde-Barros ; Julián Luengo ; Salvador García and Francisco Herrera

Affiliation: Departamento de Ciencias de la Computación e Inteligencia Artificial, Universidad de Granada, Granada, 18071 and Spain

ISBN: 978-989-758-369-8

Keyword(s): Big Data, Apache Spark, Data Preprocessing, Smart Data, Imbalanced, Classification.

Abstract: With the advent of Big Data, terabytes of data are generated and stored every second. This raw data is far from being perfect, it contains many imperfections (noise, missing values, etc.) and is not suitable for analysis, as it will led to wrong conclusions. Data preprocessing is the set of techniques devoted to polish, clean, fix, and improve that raw data. With this preprocessed data, we would be able to find more patterns in it, and to better explain the underlaying distribution of the data. This is what is called Smart Data, raw data that has been preprocessed and is ready for being analyzed, data that contains valuable information that will led to knowledge. In this work, we present two Big Data libraries for achieving Smart Data from Big Data, BigDaPSpark and BigDaPFlink. They are built on top of two Big Data frameworks, Apache Spark and Apache Flink. Both libraries contain a series of algorithms for Big Data preprocessing, ranging from noise cleaning, to discretization, or data reduction, among many others. Additionally, we ilustrate the usage of the libraries with two cases of use. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 35.172.150.239

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
García-Gil, D.; Alcalde-Barros, A.; Luengo, J.; García, S. and Herrera, F. (2019). Big Data Preprocessing as the Bridge between Big Data and Smart Data: BigDaPSpark and BigDaPFlink Libraries.In Proceedings of the 4th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS, ISBN 978-989-758-369-8, pages 324-331. DOI: 10.5220/0007738503240331

@conference{iotbds19,
author={Diego García{-}Gil. and Alejandro Alcalde{-}Barros. and Julián Luengo. and Salvador García. and Francisco Herrera.},
title={Big Data Preprocessing as the Bridge between Big Data and Smart Data: BigDaPSpark and BigDaPFlink Libraries},
booktitle={Proceedings of the 4th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS,},
year={2019},
pages={324-331},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007738503240331},
isbn={978-989-758-369-8},
}

TY - CONF

JO - Proceedings of the 4th International Conference on Internet of Things, Big Data and Security - Volume 1: IoTBDS,
TI - Big Data Preprocessing as the Bridge between Big Data and Smart Data: BigDaPSpark and BigDaPFlink Libraries
SN - 978-989-758-369-8
AU - García-Gil, D.
AU - Alcalde-Barros, A.
AU - Luengo, J.
AU - García, S.
AU - Herrera, F.
PY - 2019
SP - 324
EP - 331
DO - 10.5220/0007738503240331

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.