loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Thi Thu Trang Ngo 1 ; David Sarramia 2 ; Myoung-Ah Kang 1 and François Pinet 3

Affiliations: 1 Université Clermont Auvergne, ISIMA, LIMOS-UMR CNRS 6158, Aubière, France ; 2 Université Clermont Auvergne, CNRS/IN2P3, LPC, Clermont-Ferrand, France ; 3 Université Clermont Auvergne, INRAE, UR TSCF, Clermont-Ferrand, France

Keyword(s): ELK Stack, Elasticsearch, Spatial Data Warehouse, Georeferenced Sensor Data, ETL, Streaming Data, NoSQL, Data Lake, Data Integration.

Abstract: In the context of the French CAP 2025 I-Site project, an environmental data lake called CEBA is built at an Auvergne regional level. Its goal is to integrate data from heterogeneous sensors, provide end users tools to query and analyse georeferenced environmental data, and open data. The sensors collect different environmental measures according to their location (air and soil temperature, water quality, etc.). The measures are used by different research laboratories to analyse the environment. The main component for data shipping and storing is the ELK stack. Data are collected from sensors through Beats and streamed by Logstash to Elasticsearch. Scientists can query the data through Kibana. In this paper, we propose a data warehouse frontend to CEBA based on the ELK stack. We as well propose an additional component to the ELK stack that operates streaming ETL which allows integrating and aggregating streaming data from different sensors and sources given the user configuration in o rder to provide end users more analytical capabilities on the data. We show the architecture of this system, we present the functionalities of the data lake through examples, and finally, we present an example dashboard of the data on Kibana. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.97.9.169

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Ngo, T. ; Sarramia, D. ; Kang, M. and Pinet, F. (2021). An Analytical Tool for Georeferenced Sensor Data based on ELK Stack. In Proceedings of the 7th International Conference on Geographical Information Systems Theory, Applications and Management - GISTAM; ISBN 978-989-758-503-6; ISSN 2184-500X, SciTePress, pages 82-89. DOI: 10.5220/0010439200820089

@conference{gistam21,
author={Thi Thu Trang Ngo and David Sarramia and Myoung{-}Ah Kang and Fran\c{c}ois Pinet},
title={An Analytical Tool for Georeferenced Sensor Data based on ELK Stack},
booktitle={Proceedings of the 7th International Conference on Geographical Information Systems Theory, Applications and Management - GISTAM},
year={2021},
pages={82-89},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010439200820089},
isbn={978-989-758-503-6},
issn={2184-500X},
}

TY - CONF

JO - Proceedings of the 7th International Conference on Geographical Information Systems Theory, Applications and Management - GISTAM
TI - An Analytical Tool for Georeferenced Sensor Data based on ELK Stack
SN - 978-989-758-503-6
IS - 2184-500X
AU - Ngo, T.
AU - Sarramia, D.
AU - Kang, M.
AU - Pinet, F.
PY - 2021
SP - 82
EP - 89
DO - 10.5220/0010439200820089
PB - SciTePress