Authors:
Marcello Barbella
;
Michele Risi
;
Genoveffa Tortora
and
Alessia Auriemma Citarella
Affiliation:
Department of Computer Science, University of Salerno, Fisciano (SA), Italy
Keyword(s):
Automatic Text Summarization Algorithms, Extractive, Abstractive, ROUGE, BLEU, METEOR.
Abstract:
Automatic Text Summarization is the result of more than 50 years of research. Several methods for creating a summary from a single document or a group of related documents have been proposed over time, all of which have shown very efficient results. Artificial intelligence has enabled advancement in generating summaries that include other words compared to the original text. Instead, the issue is identifying how a summary may be regarded as ideal compared to a reference summary, which is still a topic of research that is open to new answers. How can the outcomes of the numerous new algorithms that appear year after year be assessed? This research aims to see if the ROUGE metric, widely used in the literature to evaluate the results of Text Summarization algorithms, helps deal with these new issues, mainly when the original reference dataset is limited to a small field of interest. Furthermore, an in-depth experiment is conducted by comparing the results of the ROUGE metric with other
metrics. In conclusion, determining an appropriate metric to evaluate the summaries produced by a machine is still a long way off.
(More)