Authors:
Nibel Nadjeh
;
Sabrina Abdellaoui
and
Fahima Nader
Affiliation:
Laboratoire des Méthodes de Conception de Systèmes (LMCS), Ecole Nationale Supérieure d’Informatique (ESI), BP, 68M Oued-Smar, 16270 Alger, Algeria
Keyword(s):
Data Quality, Data Consistency, Data Cleaning, Constraint Satisfaction Problems.
Abstract:
In this paper, we present CSP-DC, a data cleaning system that integrates a new intelligent solution into the cleaning process to improve data accuracy, consistency, and minimize user involvement. We address three main challenges: (1) Consistency: Most repairing algorithms introduce new violations when repairing data, especially when constraints have overlapping attributes. (2) Automaticity: User intervention is time-consuming, we seek to minimize their efforts. (3) Accuracy: Most automatic approaches compute minimal repairs and apply unverified modifications to repair ambiguous cases, which may introduce more noise. To address these challenges, we propose to formulate this problem as a constraint satisfaction problem (CSP) allowing updates that always maintain data consistency. To achieve high performances, we perform a first cleaning phase to automatically repair violations that are easily handled by existing repair algorithms. We handle violations with multiple possible repairs wit
h a CSP solving algorithm, which selects from possible fixes, values that respect all constraints. To reduce the problem’s complexity, we propose a new variables ordering technique and pruning strategies, allowing to optimize the repair search and find a solution quickly. Our experiments show that CSP-DC provides consistent and accurate repairs in a linear time, while also minimizing user intervention.
(More)