loading
Papers

Research.Publish.Connect.

Paper

Author: Daiga Deksne

Affiliation: Tilde, Vienibas Gatve 75a, Riga and Latvia

ISBN: 978-989-758-350-6

Keyword(s): Non-Canonical Language, Language Normalisation, Intelligent Virtual Assistants, Intent Detection.

Abstract: This paper reports on the development of a chat language normalisation module for the Latvian language. The model is trained using a random forest classifier algorithm that learns to rate normalisation candidates for every word. Candidates are generated using pre-trained word embeddings, N-gram lists, a spelling checker module and some other modules. The use of different means in generation of the normalisation candidates allows covering a wide spectre of errors. We are planning to use this normalisation module in the development of intelligent virtual assistants. We have performed tests to detect if the results of the intent detection module improve when text is pre-processed with the normalisation module.

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.95.131.208

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Deksne, D. (2019). Chat Language Normalisation using Machine Learning Methods.In Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI, ISBN 978-989-758-350-6, pages 965-972. DOI: 10.5220/0007693509650972

@conference{nlpinai19,
author={Daiga Deksne.},
title={Chat Language Normalisation using Machine Learning Methods},
booktitle={Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI,},
year={2019},
pages={965-972},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0007693509650972},
isbn={978-989-758-350-6},
}

TY - CONF

JO - Proceedings of the 11th International Conference on Agents and Artificial Intelligence - Volume 2: NLPinAI,
TI - Chat Language Normalisation using Machine Learning Methods
SN - 978-989-758-350-6
AU - Deksne, D.
PY - 2019
SP - 965
EP - 972
DO - 10.5220/0007693509650972

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.