Feature-level Approach for the Evaluation of Text Classification Models

Vanessa Bracamonte, Seira Hidano, Toru Nakamura, Shinsaku Kiyomoto

2022

Abstract

Visualization of explanations of text classification models is important for their evaluation. The evaluation of these models is mostly based on visualization techniques that apply to a datapoint level. Although a feature-level evaluation is possible with current visualization libraries, existing approaches do not yet implement ways for an evaluator to visualize how a text classification model behaves for features of interest for the whole data or a subset of it. In this paper, we describe and evaluate a simple feature-level approach that leverages existing interpretability methods and visualization techniques to provide evaluators information on the importance of specific features in the behavior of a text classification model. We conduct case studies of two types of text classification models: a movie review sentiment classification model and a comment toxicity model. The results show that a feature-level explanation visualization approach can help identify problems with the models.

Download


Paper Citation


in Harvard Style

Bracamonte V., Hidano S., Nakamura T. and Kiyomoto S. (2022). Feature-level Approach for the Evaluation of Text Classification Models. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP, ISBN 978-989-758-555-5, pages 164-170. DOI: 10.5220/0010846900003124


in Bibtex Style

@conference{ivapp22,
author={Vanessa Bracamonte and Seira Hidano and Toru Nakamura and Shinsaku Kiyomoto},
title={Feature-level Approach for the Evaluation of Text Classification Models},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP,},
year={2022},
pages={164-170},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010846900003124},
isbn={978-989-758-555-5},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 3: IVAPP,
TI - Feature-level Approach for the Evaluation of Text Classification Models
SN - 978-989-758-555-5
AU - Bracamonte V.
AU - Hidano S.
AU - Nakamura T.
AU - Kiyomoto S.
PY - 2022
SP - 164
EP - 170
DO - 10.5220/0010846900003124