Authors:
Rui Lima
1
and
Estrela Ferreira Cruz
2
Affiliations:
1
ARC4DigiT - Applied Research Centre for Digital Transformation, Instituto Politécnico de Viana do Castelo and Portugal
;
2
ARC4DigiT - Applied Research Centre for Digital Transformation, Instituto Politécnico de Viana do Castelo, Portugal, ALGORITMI Research Centre, Escola de Engenharia, Universidade do Minho, Guimarães and Portugal
Keyword(s):
Data Mining, Data Wrangling, Web Scraping, Data Warehouse, Parsing, Business Intelligence.
Related
Ontology
Subjects/Areas/Topics:
Coupling and Integrating Heterogeneous Data Sources
;
Data Warehouses and OLAP
;
Databases and Information Systems Integration
;
Enterprise Information Systems
Abstract:
This paper proposes an approach to detect and extract data from unstructured data source (about the subject to be studied) available online and spread by several Web pages and aggregate and store the data in a Data Warehouse properly designed for it. The Data Warehouse repository will serve as basis for the Business Intelligence and Data Mining analysis. The extracted data may be complemented with information provided by other sources in order to enrich the information to enhance the analysis and draw new and more interesting conclusions. The proposed process is then applied to a case study composed by results of athletics events realized in Portugal in the last 12 years. The files, about competition results, are available online, spread by the websites of the several athletics associations. Almost all files are published in portable document format (PDF) and each association provides files with its own different internal format. The case study also proposes an integrating mechanism
between results of athletics events with their geographic location and atmospheric conditions of the events allowing to assess and analyze how the atmospheric and geographical conditions interfere in the results achieved by the athletes.
(More)