Authors:
Anum Afzal
1
;
Juraj Vladika
1
;
Daniel Braun
2
and
Florian Matthes
1
Affiliations:
1
Department of Computer Science, Technical University of Munich, Boltzmannstrasse 3, 85748 Garching bei Muenchen, Germany
;
2
Department of High-tech Business and Entrepreneurship, University of Twente, Hallenweg 17, 7522NH Enschede, The Netherlands
Keyword(s):
Text Summarization, Natural Language Processing, Efficient Transformers, Model Hallucination, Natural Language Generation Evaluation, Domain-Adaptation of Language Models.
Abstract:
Large Language Models work quite well with general-purpose data and many tasks in Natural Language Processing. However, they show several limitations when used for a task such as domain-specific abstractive text summarization. This paper identifies three of those limitations as research problems in the context of abstractive text summarization: 1) Quadratic complexity of transformer-based models with respect to the input text length; 2) Model Hallucination, which is a model’s ability to generate factually incorrect text; and 3) Domain Shift, which happens when the distribution of the model’s training and test corpus is not the same. Along with a discussion of the open research questions, this paper also provides an assessment of existing state-of-the-art techniques relevant to domain-specific text summarization to address the research gaps.