Highly Automated Corner Cases Extraction: Using Gradient Boost Quantile Regression for AI Quality Assurance

Niels Heller, Namrata Gurung

2022

Abstract

This work introduces a method for Quality Assurance of Artificial Intelligence (AI) Systems, which identifies and characterizes “corner cases”. Here, corner cases are intuitively defined as “inputs yielding an unexpectedly bad AI performance”. While relying on automated methods for corner case selection, the method relies also on human work. Specifically, the method structures the work of data scientists in an iterative process which formalizes the expectations towards an AI under test. The method is applied in a use case in Autonomous Driving, and validation experiments, which point at a general effectiveness of the method, are reported on. Besides allowing insights on the AI under test, the method seems to be particularly suited to structure a constructive critique of the quality of a test dataset. As this work reports on a first application of the method, a special focus lies on limitations and possible extensions of the method.

Paper Citation

in Harvard Style

Heller N. and Gurung N. (2022). Highly Automated Corner Cases Extraction: Using Gradient Boost Quantile Regression for AI Quality Assurance. In Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA, ISBN 978-989-758-583-8, pages 62-73. DOI: 10.5220/0011152200003269

in Bibtex Style

@conference{data22,
author={Niels Heller and Namrata Gurung},
title={Highly Automated Corner Cases Extraction: Using Gradient Boost Quantile Regression for AI Quality Assurance},
booktitle={Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA,},
year={2022},
pages={62-73},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011152200003269},
isbn={978-989-758-583-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Data Science, Technology and Applications - Volume 1: DATA,
TI - Highly Automated Corner Cases Extraction: Using Gradient Boost Quantile Regression for AI Quality Assurance
SN - 978-989-758-583-8
AU - Heller N.
AU - Gurung N.
PY - 2022
SP - 62
EP - 73
DO - 10.5220/0011152200003269