participants' performance across all levels of Bloom's
taxonomy post-intervention, indicating an improved
understanding and application of programming
concepts crucial for data analysis.
The substantial effect sizes reported in the t-test
results underscore the profound impact of the
intervention on learners' ability to grasp and apply
programming principles within the context of data
analysis. Additionally, task completion rates after
structured prompt training with Generative AI
suggest the significant influence of the intervention
on participants' ability to understand and execute data
analysis tasks. This progress goes beyond mere
memorization, indicating a shift towards the
comprehension of the underlying principles.
Furthermore, the System Usability Scale (SUS)
survey results, indicating a positive reception of the
Generative AI tool's usability, complement the
study's findings. A user-friendly and effective tool is
crucial in an educational setting, as it can significantly
reduce the cognitive load on learners, allowing them
to concentrate on understanding and applying the
concepts rather than navigating the tool itself.
This study sheds light on the potential of
Generative AI and structured prompt engineering to
transform educational methods. The results of this
research suggest that Generative AI can play a crucial
role in helping learners understand complex subjects
like programming and data analysis. Moreover, the
usability of structured prompts has been instrumental
in providing students with clear, actionable guidance
through intricate learning tasks, enhancing their
engagement and help them to master skills.
However, the study acknowledges its limitations,
including the absence of log data analysis and
qualitative data like interviews which could provide
deeper insights into the behavioral patterns of high
and low performers. The relatively small sample size
also restricts the generalizability of the findings.
7 FUTURE WORK
Future research on integrating Generative AI and
structured prompt engineering in education,
especially in programming and data analysis, is set to
deepen our understanding of its effects on learning.
Planned comparative studies can examine the
learning outcomes of groups with varying levels of
access to ChatGPT and prompt training, aiming to
understand the role of Generative AI in learner
engagement and educational processes. These studies
can expand participant diversity and employ methods
like structured interviews and task analysis to capture
detailed learner interactions and perceptions. A key
focus will be evaluating the quality of participants'
prompts to enhance critical thinking and refine
training methods. Expected to enrich learning
theories and Human-Computer Interaction
frameworks, this research will help explore how
Generative AI can innovate pedagogy and create
personalized, accessible educational experiences
worldwide.
REFERENCES
Bahrini, A. et al. (2023). ChatGPT: Applications,
opportunities, and threats. 2023 Systems and
Information Engineering Design Symposium (SIEDS),
Charlottesville, VA, USA, 274-279.
https://doi.org/10.1109/SIEDS58326.2023.10137850
Brooke, J. (1995, November 30). SUS: A quick and dirty
usability scale. Usability Evaluation in Industry, 189.
Dai, Y., et al. (2023). Reconceptualizing ChatGPT and
generative AI as a student-driven innovation in higher
education. Procedia CIRP, 119, 84-90.
https://doi.org/10.1016/j.procir.2023.05.002
De Silva, D., Mills, N., El-Ayoubi, M., Manic, M., &
Alahakoon, D. (2023). ChatGPT and generative AI
guidelines for addressing academic integrity and
augmenting pre-existing chatbots. Proceedings of the
IEEE International Conference on Industrial
Technology, 2023-April.
https://doi.org/10.1109/ICIT58465.2023.10143123
Denny, P., Kumar, V., & Giacaman, N. (2023). Conversing
with Copilot: Exploring prompt engineering for solving
CS1 problems using natural language. In Proceedings
of the 54th ACM Technical Symposium on Computer
Science Education (SIGCSE 2023) (pp. 1-7). ACM.
https://doi.org/10.1145/3545945.356982
Dhoni, P. (2023, August 29). Exploring the synergy
between generative AI, data, and analytics in the
modern age. TechRxiv.
https://doi.org/10.36227/techrxiv.24045792.v1
Feng, Y., Vanam, S., Cherukupally, M., Zheng, W., Qiu,
M., & Chen, H. (2023). Investigating code generation
performance of ChatGPT with crowdsourcing social
data. In 2023 IEEE 47th Annual Computers, Software,
and Applications Conference (COMPSAC) (pp. TBD).
IEEE.
Finnie-Ansley, J., Denny, P., Becker, B.A., Luxton-Reilly,
A., & Prather, J. (2022). The robots are coming:
Exploring the implications of OpenAI Codex on
introductory programming. In Proceedings of the 24th
Australasian Computing Education Conference (pp.
TBD). Virtual Event, February 14–18, 2022.
Firaina, R., & Sulisworo, D. (2023). Exploring the usage of
ChatGPT in higher education: Frequency and impact on
productivity. Buletin Edukasi Indonesia, 2(01), 39–46.
https://doi.org/10.56741/bei.v2i01.310