Papers Papers/2022 Papers Papers/2022



Authors: Eduardo Nascimento 1 ; 2 ; Grettel García 1 ; Lucas Feijó 1 ; Wendy Victorio 1 ; 2 ; Yenier Izquierdo 1 ; Aiko R. de Oliveira 2 ; Gustavo Coelho 1 ; Melissa Lemos 2 ; 1 ; Robinson Garcia 3 ; Luiz Leme 4 and Marco Casanova 2 ; 1

Affiliations: 1 Instituto Tecgraf, PUC-Rio, Rio de Janeiro, 22451-900, RJ, Brazil ; 2 Departamento de Informática, PUC-Rio, Rio de Janeiro, 22451-900, RJ, Brazil ; 3 Petrobras, Rio de Janeiro, 20031-912, RJ, Brazil ; 4 Instituto de Computação, UFF, Niterói, 24210-310, RJ, Brazil

Keyword(s): Text-to-SQL, GPT, Large Language Models, Industrial Databases.

Abstract: Text-to-SQL refers to the task defined as “ given a relational database D and a natural language sentence S that describes a question on D, generate an SQL query Q over D that expresses S”. Numerous tools have addressed this task with relative success over well-known benchmarks. Recently, several LLM-based text-to-SQL tools, that is, text-to-SQL tools that explore Large Language Models (LLMs), emerged that outperformed previous approaches. When adopted for industrial-size databases, with a large number of tables, columns, and foreign keys, the performance of LLM-based text-to-SQL tools is, however, significantly less than that reported for the benchmarks. This paper then investigates how a selected set of LLM-based text-to-SQL tools perform over two challenging databases, an openly available database, Mondial, and a proprietary industrial database. The paper also proposes a new LLM-based text-to-SQL tool that combines features from tools that performed well over the Spider and BIRD b enchmarks. Then, the paper describes how the selected tools and the proposed tool, running under GPT-3.5 and GPT-4, perform over the Mondial and the industrial databases over a suite of 100 carefully defined natural language questions that are closely related to those observed in practice. It concludes with a discussion of the results obtained. (More)


Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Nascimento, E.; García, G.; Feijó, L.; Victorio, W.; Izquierdo, Y.; R. de Oliveira, A.; Coelho, G.; Lemos, M.; Garcia, R.; Leme, L. and Casanova, M. (2024). Text-to-SQL Meets the Real-World. In Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS; ISBN 978-989-758-692-7; ISSN 2184-4992, SciTePress, pages 61-72. DOI: 10.5220/0012555200003690

author={Eduardo Nascimento. and Grettel García. and Lucas Feijó. and Wendy Victorio. and Yenier Izquierdo. and Aiko {R. de Oliveira}. and Gustavo Coelho. and Melissa Lemos. and Robinson Garcia. and Luiz Leme. and Marco Casanova.},
title={Text-to-SQL Meets the Real-World},
booktitle={Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS},


JO - Proceedings of the 26th International Conference on Enterprise Information Systems - Volume 1: ICEIS
TI - Text-to-SQL Meets the Real-World
SN - 978-989-758-692-7
IS - 2184-4992
AU - Nascimento, E.
AU - García, G.
AU - Feijó, L.
AU - Victorio, W.
AU - Izquierdo, Y.
AU - R. de Oliveira, A.
AU - Coelho, G.
AU - Lemos, M.
AU - Garcia, R.
AU - Leme, L.
AU - Casanova, M.
PY - 2024
SP - 61
EP - 72
DO - 10.5220/0012555200003690
PB - SciTePress