loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Sayed Arefin ; Tasnia Heya ; Hasan Al-Qudah ; Ynes Ineza and Abdul Serwadda

Affiliation: Texas Tech University, Lubbock, Texas, U.S.A.

Keyword(s): Large Language Models, ChatGPT, Code Smells, Algorithms.

Abstract: We conduct an extensive analysis of ChatGPT, a standout Large Language Model (LLM), particularly in coding within the Python language, focusing on data structures and algorithms. We assess ChatGPT’s ability to accurately solve coding problems, its code quality, and the nature of run-time errors. Additionally, we examine how ChatGPT’s code performs when it executes but doesn’t solve the problem, identifying error patterns. We also explore whether ChatGPT has memorized training data through a structured experiment. Comparing with human performance where possible, our study encompasses both GPT-3.5 and GPT-4 models, various subtopics within the main areas, and problems of different complexities.

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 18.221.188.161

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Arefin, S.; Heya, T.; Al-Qudah, H.; Ineza, Y. and Serwadda, A. (2024). Unmasking the Giant: A Comprehensive Evaluation of ChatGPT’s Proficiency in Coding Algorithms and Data Structures. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-680-4; ISSN 2184-433X, SciTePress, pages 412-419. DOI: 10.5220/0012467100003636

@conference{icaart24,
author={Sayed Arefin. and Tasnia Heya. and Hasan Al{-}Qudah. and Ynes Ineza. and Abdul Serwadda.},
title={Unmasking the Giant: A Comprehensive Evaluation of ChatGPT’s Proficiency in Coding Algorithms and Data Structures},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2024},
pages={412-419},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012467100003636},
isbn={978-989-758-680-4},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - Unmasking the Giant: A Comprehensive Evaluation of ChatGPT’s Proficiency in Coding Algorithms and Data Structures
SN - 978-989-758-680-4
IS - 2184-433X
AU - Arefin, S.
AU - Heya, T.
AU - Al-Qudah, H.
AU - Ineza, Y.
AU - Serwadda, A.
PY - 2024
SP - 412
EP - 419
DO - 10.5220/0012467100003636
PB - SciTePress