Implementing the Perturbation Approach for Reliability Assessment: A Case Study in the Context of Flight Delay Prediction

Simon Staudinger, Christoph Großauer, Pascal Badzura, Christoph G. Schuetz, Michael Schrefl

2025

Abstract

Organizations employ prediction models as a foundation for decision-making. A prediction model learned from training data is often only evaluated using global quality indicators, e.g., accuracy and precision. These global indicators, however, do not provide guidance regarding the reliability of the prediction for a specific input case. In this paper, we instantiate a generic reference process for implementing reliability assessment methods for specific input cases on the real-world use case of flight delay prediction. We specifically implement the perturbation approach to reliability assessment for this use case and then describe the steps that were taken to train the prediction model, with an emphasis on the activities required to implement the perturbation approach. The perturbation approach consists of slightly altering feature values for an individual input case, e.g., within the margins of error of a sensed value, and observe whether the prediction of the model changes, which would render the prediction unreliable. The implementation of the perturbation approach requires decisions and documentations along the various stages of the data mining process. A generic tool can be used to document and perform reliability assessment using the perturbation approach.

Paper Citation

in Harvard Style

Staudinger S., Großauer C., Badzura P., Schuetz C. and Schrefl M. (2025). Implementing the Perturbation Approach for Reliability Assessment: A Case Study in the Context of Flight Delay Prediction. In Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-749-8, SciTePress, pages 75-86. DOI: 10.5220/0013299700003929

in Bibtex Style

@conference{iceis25,
author={Simon Staudinger and Christoph Großauer and Pascal Badzura and Christoph Schuetz and Michael Schrefl},
title={Implementing the Perturbation Approach for Reliability Assessment: A Case Study in the Context of Flight Delay Prediction},
booktitle={Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS},
year={2025},
pages={75-86},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013299700003929},
isbn={978-989-758-749-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 27th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - Implementing the Perturbation Approach for Reliability Assessment: A Case Study in the Context of Flight Delay Prediction
SN - 978-989-758-749-8
AU - Staudinger S.
AU - Großauer C.
AU - Badzura P.
AU - Schuetz C.
AU - Schrefl M.
PY - 2025
SP - 75
EP - 86
DO - 10.5220/0013299700003929
PB - SciTePress