User-Driven RAG System with LlamaIndex Multi-Agent Architectures and Qdrant

M. Devika, Shatakshi R., Madhuvanthi S., Tarunya V. V.

2025

Abstract

Retrieval-Augmented Generation (RAG) systems have achieved substantial success in the field of information retrieval and text generation by augmenting state-of-the-art large language models (LLMs) with external knowledge. Nevertheless, current RAG architectures fall short on three main fronts: user adaptability, as they need a lot of code changes to modify important retrieval settings (for example: chunking methods, embedding models, and reranking methods); We present a user-controlled RAG system powered by the multi-agent architecture of LlamaIndex and the vector search capabilities of Qdrant for real-time customized RAG. To increase system adaptability, we propose a new Adaptive Retrieval Feedback Loop (ARFL) where users are able to iteratively adjust queries given the confidence and relevance of generated responses. The ARFL system can automatically modify retrieval parameters according to user feedback or retrieval outputs with low confidence scores while limiting the amount of time to query again which causes more manual effort with possibly higher retrieval precision. Our system enables users to change retrieval configurations easily with its conversational interface, without the requirement of going into any codebase. This advances the frontiers of user-centric generative models by componentizing the RAG systems using pipeline-based architectures in order to make RAG systems more intuitive, flexible and user preference-centric.

Paper Citation

in Harvard Style

Devika M., R. S., S. M. and V. T. (2025). User-Driven RAG System with LlamaIndex Multi-Agent Architectures and Qdrant. In Proceedings of the 1st International Conference on Research and Development in Information, Communication, and Computing Technologies - ICRDICCT`25; ISBN 978-989-758-777-1, SciTePress, pages 754-761. DOI: 10.5220/0013920300004919

in Bibtex Style

@conference{icrdicct`2525,
author={M. Devika and Shatakshi R. and Madhuvanthi S. and Tarunya V.},
title={User-Driven RAG System with LlamaIndex Multi-Agent Architectures and Qdrant},
booktitle={Proceedings of the 1st International Conference on Research and Development in Information, Communication, and Computing Technologies - ICRDICCT`25},
year={2025},
pages={754-761},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013920300004919},
isbn={978-989-758-777-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 1st International Conference on Research and Development in Information, Communication, and Computing Technologies - ICRDICCT`25
TI - User-Driven RAG System with LlamaIndex Multi-Agent Architectures and Qdrant
SN - 978-989-758-777-1
AU - Devika M.
AU - R. S.
AU - S. M.
AU - V. T.
PY - 2025
SP - 754
EP - 761
DO - 10.5220/0013920300004919
PB - SciTePress