A Reference Process for Judging Reliability of Classification Results in Predictive Analytics

Simon Staudinger, Christoph Schuetz, Michael Schrefl

2021

Abstract

Organizations employ data mining to discover patterns in historic data. The models that are learned from the data allow analysts to make predictions about future events of interest. Different global measures, e.g., accuracy, sensitivity, and specificity, are employed to evaluate a predictive model. In order to properly assess the reliability of an individual prediction for a specific input case, global measures may not suffice. In this paper, we propose a reference process for the development of predictive analytics applications that allow analysts to better judge the reliability of individual classification results. The proposed reference process is aligned with the CRISP-DM stages and complements each stage with a number of tasks required for reliability checking. We further explain two generic approaches that assist analysts with the assessment of reliability of individual predictions, namely perturbation and local quality measures.

Paper Citation

in Harvard Style

Staudinger S., Schuetz C. and Schrefl M. (2021). A Reference Process for Judging Reliability of Classification Results in Predictive Analytics. In Proceedings of the 10th International Conference on Data Science, Technology and Applications - Volume 1: DATA, ISBN 978-989-758-521-0, pages 124-134. DOI: 10.5220/0010620501240134

in Bibtex Style

@conference{data21,
author={Simon Staudinger and Christoph Schuetz and Michael Schrefl},
title={A Reference Process for Judging Reliability of Classification Results in Predictive Analytics},
booktitle={Proceedings of the 10th International Conference on Data Science, Technology and Applications - Volume 1: DATA,},
year={2021},
pages={124-134},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010620501240134},
isbn={978-989-758-521-0},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 10th International Conference on Data Science, Technology and Applications - Volume 1: DATA,
TI - A Reference Process for Judging Reliability of Classification Results in Predictive Analytics
SN - 978-989-758-521-0
AU - Staudinger S.
AU - Schuetz C.
AU - Schrefl M.
PY - 2021
SP - 124
EP - 134
DO - 10.5220/0010620501240134