Unsupervised Evaluation of Human Translation Quality

Yi Zhou, Danushka Bollegala

2019

Abstract

Even though machine translation (MT) systems have reached impressive performances in cross-lingual translation tasks, the quality of MT is still far behind professional human translations (HTs) due to the complexity in natural languages, especially for terminologies in different domains. Therefore, HTs are still widely demanded in practice. However, the quality of HT is also imperfect and vary significantly depending on the experience and knowledge of the translators. Evaluating the quality of HT in an automatic manner has faced many challenges. Although bilingual speakers are able to assess the translation quality, manually checking the accuracy of translations is expensive and time-consuming. In this paper, we propose an unsupervised method to evaluate the quality of HT without requiring any labelled data. We compare a range of methods for automatically grading HTs and observe the Bidirectional Minimum Word Mover’s distance (BiMWMD) to produce gradings that correlate well with humans.

Paper Citation

in Harvard Style

Zhou Y. and Bollegala D. (2019). Unsupervised Evaluation of Human Translation Quality. In Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR; ISBN 978-989-758-382-7, SciTePress, pages 55-64. DOI: 10.5220/0008064500550064

in Bibtex Style

@conference{kdir19,
author={Yi Zhou and Danushka Bollegala},
title={Unsupervised Evaluation of Human Translation Quality},
booktitle={Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR},
year={2019},
pages={55-64},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0008064500550064},
isbn={978-989-758-382-7},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2019) - Volume 1: KDIR
TI - Unsupervised Evaluation of Human Translation Quality
SN - 978-989-758-382-7
AU - Zhou Y.
AU - Bollegala D.
PY - 2019
SP - 55
EP - 64
DO - 10.5220/0008064500550064
PB - SciTePress