FactTrace: Designing a News Fact-Checking Tool with Large Language Models

Andy Wahyu Mardiansyah, Tiyas Yulita, Susila Windarta, Rahmat Purwoko, I Gede Maha Putra

2025

Abstract

The development of information technology has driven digitalization, which has had a positive impact on the dissemination of information. However, this has also contributed to the increase in hoaxes. The spread of hoaxes in Indonesia has increased year by year. This can cause division among Indonesian society. The main problem is the lack of tools to automatically verify news validity. This study developed a news fact-checking tool using large language models (LLMs) such as GPT-3.5 Turbo 0125, GPT-4o, Llama 3.1-8B, and DeepSeek-R1-7B, integrated with retrieval-augmented generation (RAG). The approach used to improve RAG is a hybrid retrieval and reranking method. The news validity verification tool (FactTrace) was designed using the Waterfall model of the System Development Lifecycle (SDLC), developed with the n8n platform. Testing was conducted using an evaluation matrix to assess the performance of the four LLMs used based on accuracy, precision, recall, and F1 score. Based on the test results, FactTrace was implemented into a web chatbot using LLM GPT-4o, which had an accuracy value of 0.989, precision of 0.991, recall of 0.987, and F1 Score of 0.989.

Paper Citation

in Harvard Style

Mardiansyah A., Yulita T., Windarta S., Purwoko R. and Putra I. (2025). FactTrace: Designing a News Fact-Checking Tool with Large Language Models. In Proceedings of the 1st International Conference on Research and Innovations in Information and Engineering Technology - Volume 1: RITECH; ISBN 978-989-758-784-9, SciTePress, pages 203-208. DOI: 10.5220/0014270800004928

in Bibtex Style

@conference{ritech25,
author={Andy Wahyu Mardiansyah and Tiyas Yulita and Susila Windarta and Rahmat Purwoko and I Gede Maha Putra},
title={FactTrace: Designing a News Fact-Checking Tool with Large Language Models},
booktitle={Proceedings of the 1st International Conference on Research and Innovations in Information and Engineering Technology - Volume 1: RITECH},
year={2025},
pages={203-208},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0014270800004928},
isbn={978-989-758-784-9},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Research and Innovations in Information and Engineering Technology - Volume 1: RITECH
TI - FactTrace: Designing a News Fact-Checking Tool with Large Language Models
SN - 978-989-758-784-9
AU - Mardiansyah A.
AU - Yulita T.
AU - Windarta S.
AU - Purwoko R.
AU - Putra I.
PY - 2025
SP - 203
EP - 208
DO - 10.5220/0014270800004928
PB - SciTePress