SiameseBERT: A Bert-Based Siamese Network Enhanced with a Soft Attention Mechanism for Arabic Semantic Textual Similarity

Rakia Saidi, Fethi Jarray, Fethi Jarray, Mohammed Alsuhaibani

2023

Abstract

The assessment of semantic textual similarity (STS) is a challenging task in natural language processing. It is crucial for many applications, including question answering, plagiarism detection, machine translation, information retrieval, and word sense disambiguation. The STS task evaluates the similarity of data pairs of text. For high high-resource languages (e.g. English), several approaches for STS have been proposed. In this paper, we are interested in measuring the semantic similarity of texts for Arabic, a low-resource language. A standard approach for STS is based on vector embedding of the input text and application of similarity metric on space embedding. In this contribution, we propose a BERT-based Siamese Network (SiameseBERT) and investigate the most available Arabic BERT models to embed the input sentences. We validate our approach via Arabic STS datasets. The araBERT-based Siamese Network model achieves a Pearson correlation of 0.925. The results obtained demonstrate the superiority of integrating the BERT embedding, the attention mechanism, and the Siamese neural network for the semantic textual similarity task.

Paper Citation

in Harvard Style

Saidi R., Jarray F. and Alsuhaibani M. (2023). SiameseBERT: A Bert-Based Siamese Network Enhanced with a Soft Attention Mechanism for Arabic Semantic Textual Similarity. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 146-151. DOI: 10.5220/0011624800003393

in Bibtex Style

@conference{icaart23,
author={Rakia Saidi and Fethi Jarray and Mohammed Alsuhaibani},
title={SiameseBERT: A Bert-Based Siamese Network Enhanced with a Soft Attention Mechanism for Arabic Semantic Textual Similarity},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={146-151},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011624800003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - SiameseBERT: A Bert-Based Siamese Network Enhanced with a Soft Attention Mechanism for Arabic Semantic Textual Similarity
SN - 978-989-758-623-1
AU - Saidi R.
AU - Jarray F.
AU - Alsuhaibani M.
PY - 2023
SP - 146
EP - 151
DO - 10.5220/0011624800003393