Authors:
Tolgahan Cakaloglu
1
;
2
;
Xiaowei Xu
1
and
Roshith Raghavan
3
Affiliations:
1
University of Arkansas, Little Rock, Arkansas, U.S.A.
;
2
Walmart Labs, Dallas, Texas, U.S.A.
;
3
Walmart Labs, Bentonville, Arkansas, U.S.A.
Keyword(s):
Natural Language Processing, Information Retrieval, Deep Learning, Learning Representations, Text Matching.
Abstract:
Learning hierarchical representation has been vital in natural language processing and information retrieval. With recent advances, the importance of learning the context of words has been underscored. In this paper we propose EmBoost i.e. Embedding Boosting of word or document vector representations that have been learned from multiple embedding models. The advantage of this approach is that this higher order word embedding represents documents at multiple levels of abstraction. The performance gain from this approach has been demonstrated by comparing with various existing text embedding strategies on retrieval and semantic similarity tasks using Stanford Question Answering Dataset (SQuAD), and Question Answering by Search And Reading (QUASAR). The multilevel abstract word embedding is consistently superior to existing solo strategies including Glove, FastText, ELMo and BERT-based models. Our study shows that further gains can be made when a deep residual neural model is specifical
ly trained for document retrieval.
(More)