Performance Evaluation of Open Source LLMs for Legal Document Summarization and Chatbot Integration for Legal Queries

Abhinaya Danda, Mrudula Gotmare, Gomati Iyer, Charusheela Nehete

2025

Abstract

This paper presents a comprehensive evaluation of an AI-driven platform that combines cutting-edge technologies to enhance legal workflows. The platform integrates two primary features: legal document summarization using OCR-based techniques and a chatbot assistant for legal queries. The summarization module utilizes three advanced open-source LLMs-LegalBERT, Llama 2, and DeepSeek-V3-to process and condense complex legal documents into accessible summaries, focusing on obligations, risks, and key insights. The chatbot assistant provides instant conversational support for navigating legal processes, addressing queries, and ensuring better user engagement. By integrating OCR, NLP, and advanced AI models, the system simplifies legal document understanding for non-experts while offering reliable and efficient query resolution. A comparative performance analysis of the three LLMs highlights their strengths and limitations in handling diverse legal document types and chatbot interactions.Initial evaluations reveal significant advancements in summarization accuracy, query response quality, and overall user satisfaction. This study underscores the potential of AI in making legal support accessible, reducing time, effort, and costs for individuals and businesses.

Paper Citation

in Harvard Style

Danda A., Gotmare M., Iyer G. and Nehete C. (2025). Performance Evaluation of Open Source LLMs for Legal Document Summarization and Chatbot Integration for Legal Queries. In Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT; ISBN 978-989-758-763-4, SciTePress, pages 114-119. DOI: 10.5220/0013609600004664

in Bibtex Style

@conference{incoft25,
author={Abhinaya Danda and Mrudula Gotmare and Gomati Iyer and Charusheela Nehete},
title={Performance Evaluation of Open Source LLMs for Legal Document Summarization and Chatbot Integration for Legal Queries},
booktitle={Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT},
year={2025},
pages={114-119},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013609600004664},
isbn={978-989-758-763-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT
TI - Performance Evaluation of Open Source LLMs for Legal Document Summarization and Chatbot Integration for Legal Queries
SN - 978-989-758-763-4
AU - Danda A.
AU - Gotmare M.
AU - Iyer G.
AU - Nehete C.
PY - 2025
SP - 114
EP - 119
DO - 10.5220/0013609600004664
PB - SciTePress