Authors:
Kazuhito Tamura
1
;
Ikumi Suzuki
2
and
Kazuo Hara
3
Affiliations:
1
Graduate School of Science and Engineering, Yamagata University, Yonezawa-shi, Yamagata, Japan
;
2
School of Information and Data Sciences, Nagasaki University, Nagasaki-shi, Nagasaki, Japan
;
3
Faculty of Science, Yamagata University, Yamagata-shi, Yamagata, Japan
Keyword(s):
Neural Language Model, Target Evaluation, Japanese Case Frame, LSTM.
Abstract:
Automatic text generation are widely used in various type of natural language processing systems. It is crucial to capture correct grammar for these systems to work. According to the recent studies, neural language models successfully acquire English grammar. However, it’s not thoroughly investigated why the neural language models work. Therefore, fine-grained grammatical or syntactic analysis is important to assess neural language models. In this paper, we constructed grammatical evaluation methods to assess Japanese grammatical ability in neural language models by adopting a target evaluation approach. We especially focus on case marker and verb match in Japanese case grammar. In experiments, we report the grammatical ability of neural language model by comparing n-gram models. Neural language model performed better even some information lacks, while n-gram performs poorly. Also, Neural language model exhibited more robust performance for low frequency terms.