loading
Papers

Research.Publish.Connect.

Paper

Authors: Jasper van der Waa ; Jurriaan van Diggelen ; Mark Neerincx and Stephan Raaijmakers

Affiliation: TNO, Netherlands

ISBN: 978-989-758-275-2

Keyword(s): Machine learning, Trust, Certainty, Uncertainty, Explainable Artificial Intelligence.

Related Ontology Subjects/Areas/Topics: Artificial Intelligence ; Computational Intelligence ; Evolutionary Computing ; Industrial Applications of AI ; Knowledge Discovery and Information Retrieval ; Knowledge-Based Systems ; Machine Learning ; Soft Computing ; Symbolic Systems ; Uncertainty in AI

Abstract: End-users of machine learning-based systems benefit from measures that quantify the trustworthiness of the underlying models. Measures like accuracy provide for a general sense of model performance, but offer no detailed information on specific model outputs. Probabilistic outputs, on the other hand, express such details, but they are not available for all types of machine learning, and can be heavily influenced by bias and lack of representative training data. Further, they are often difficult to understand for non-experts. This study proposes an intuitive certainty measure (ICM) that produces an accurate estimate of how certain a machine learning model is for a specific output, based on errors it made in the past. It is designed to be easily explainable to non-experts and to act in a predictable, reproducible way. ICM was tested on four synthetic tasks solved by support vector machines, and a real-world task solved by a deep neural network. Our results show that ICM is both more accurate and intuitive than related approaches. Moreover, ICM is neutral with respect to the chosen machine learning model, making it widely applicable. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 34.237.51.159

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
van der Waa, J.; van Diggelen, J.; Neerincx, M. and Raaijmakers, S. (2018). ICM: An Intuitive Model Independent and Accurate Certainty Measure for Machine Learning.In Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-275-2, pages 314-321. DOI: 10.5220/0006542603140321

@conference{icaart18,
author={Jasper van der Waa. and Jurriaan van Diggelen. and Mark Neerincx. and Stephan Raaijmakers.},
title={ICM: An Intuitive Model Independent and Accurate Certainty Measure for Machine Learning},
booktitle={Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2018},
pages={314-321},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006542603140321},
isbn={978-989-758-275-2},
}

TY - CONF

JO - Proceedings of the 10th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - ICM: An Intuitive Model Independent and Accurate Certainty Measure for Machine Learning
SN - 978-989-758-275-2
AU - van der Waa, J.
AU - van Diggelen, J.
AU - Neerincx, M.
AU - Raaijmakers, S.
PY - 2018
SP - 314
EP - 321
DO - 10.5220/0006542603140321

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.