loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Marcello Barbella ; Michele Risi and Genoveffa Tortora

Affiliation: Department of Computer Science, University of Salerno, Fisciano (SA), Italy

Keyword(s): Automatic Text Summarization Algorithms, Extractive, Abstractive, ROUGE Metric, Bert.

Abstract: Automatic Text Summarization techniques aim to extract key information from one or more input texts automatically, producing summaries and preserving the meaning of content. These techniques are divided into two main families: Extractive and Abstractive, which differ for their operating mode. The former picks up sentences directly from the document text, whilst the latter produces a summary by interpreting the text and rephrases sentences by incorporating information. Therefore, there is the need to evaluate and verify how close a summary is to original text. The research question is: how to evaluate the quality of the summaries produced by these techniques? Different metrics and scores have been proposed in the literature (e.g., ROUGE) for the evaluation of text summarization. Thus, the main purpose of this paper is to deeply estimate the behaviour of the ROUGE metric. In particular, we performed a first experiment to compare the metric efficiency for the evaluation of the Abstracti ve versus Extractive Text Summarization algorithms while, in a second one, we compared the obtained score for two different summary approaches: the simple execution of a summarization algorithm versus the multiple execution of different algorithms on the same text. The conclusions lead to the following interesting results: ROUGE does not achieve excellent results, because it has similar performance on both the Abstractive and Extractive algorithms; multiple execution works better than single one most of the time. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.209.209.28

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Barbella, M.; Risi, M. and Tortora, G. (2021). A Comparison of Methods for the Evaluation of Text Summarization Techniques. In Proceedings of the 10th International Conference on Data Science, Technology and Applications - DATA; ISBN 978-989-758-521-0; ISSN 2184-285X, SciTePress, pages 200-207. DOI: 10.5220/0010523002000207

@conference{data21,
author={Marcello Barbella. and Michele Risi. and Genoveffa Tortora.},
title={A Comparison of Methods for the Evaluation of Text Summarization Techniques},
booktitle={Proceedings of the 10th International Conference on Data Science, Technology and Applications - DATA},
year={2021},
pages={200-207},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010523002000207},
isbn={978-989-758-521-0},
issn={2184-285X},
}

TY - CONF

JO - Proceedings of the 10th International Conference on Data Science, Technology and Applications - DATA
TI - A Comparison of Methods for the Evaluation of Text Summarization Techniques
SN - 978-989-758-521-0
IS - 2184-285X
AU - Barbella, M.
AU - Risi, M.
AU - Tortora, G.
PY - 2021
SP - 200
EP - 207
DO - 10.5220/0010523002000207
PB - SciTePress