Language Identification for Short Medical Texts

Erick Godinez, Zoltán Szlávik, Selene Santamaría, Robert-Jan Sips

Abstract

Language identification remains a challenge for short texts originating from social media. Moreover, domain-specific terminology, which is frequent in the medical domain, may not change cross-linguistically, making language identification even more difficult. We conducted language identification on four datasets, two of them with general language, and two of them containing medical language. We evaluated the impact of two embedding representations and a set of linguistic features based on graphotactics. The proposed linguistic features reflect the graphotactics of the languages included in the test dataset. For classification, we implemented two algorithms: random forest and SVM. Our findings show that, when classifying general language, linguistic-based features perform close to the embedding representations of fastText and BERT. However, when classifying text with technical terms, the linguistic features outperform embedding representations. The combination of embeddings with linguistic features had a positive impact on the classification task under both settings. Therefore, our results suggest that these linguistic features could be applied for big and small datasets keeping the good performances in both general and medical languages. As future work, we want to test the linguistic features for a more significant set of languages.

Download


Paper Citation


in Harvard Style

Godinez E., Szlávik Z., Santamaría S. and Sips R. (2020). Language Identification for Short Medical Texts.In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, ISBN 978-989-758-398-8, pages 399-406. DOI: 10.5220/0008950903990406


in Bibtex Style

@conference{healthinf20,
author={Erick Godinez and Zoltán Szlávik and Selene Santamaría and Robert-Jan Sips},
title={Language Identification for Short Medical Texts},
booktitle={Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF,},
year={2020},
pages={399-406},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008950903990406},
isbn={978-989-758-398-8},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF,
TI - Language Identification for Short Medical Texts
SN - 978-989-758-398-8
AU - Godinez E.
AU - Szlávik Z.
AU - Santamaría S.
AU - Sips R.
PY - 2020
SP - 399
EP - 406
DO - 10.5220/0008950903990406