Poetry Generation Using Transformer Based Model GPT-Neo

Pranav Bhat, Karthik K P, Shrishail Golappanavar, Rohit Mendigeri, Uday Kulkarni, Shashank Hegde

2025

Abstract

Poetry generation is an exciting and evolving area of creative AI, where artificial intelligence is applied to the art of writing. In this work, we explore the use of a fine-tuned GPT-Neo model for generating poetry. A customized poem dataset is employed in the training process to capture the unique features of this creative form. The dataset is enriched, tokenized, and optimized to streamline the integration with the model. We also adopt a mixed-precision approach to fine-tuning, enhancing resource efficiency, and use top-k and temperature-reaching strategies to generate more coherent outputs. Our model demonstrates creative flow and thematic richness, making it useful for both generative and exploratory purposes in poetry. Evaluation of six generated limericks revealed semantic coherence scores ranging from 0.47 to 0.58, with an average score of 0.53. Compared to GPT-4, which averaged a semantic coherence score of 0.47, our model shows a 12.77 percent improvement. Our results, shown in Table 1, reveal that the poems generated by our fine-tuned GPT-Neo model outperform those generated by GPT-4 in terms of semantic coherence. The evaluation metrics, including token generation, entropy, coherence, and perplexity, suggest that our model produces more thematically cohesive and contextually consistent poetry. This research contributes to the growing field of AI in the arts, where the potential of artificial intelligence in creative domains is being continually explored. The improved performance of our model in semantic coherence signifies a meaningful advancement in AI-assisted poetry generation.

Paper Citation

in Harvard Style

Bhat P., K P K., Golappanavar S., Mendigeri R., Kulkarni U. and Hegde S. (2025). Poetry Generation Using Transformer Based Model GPT-Neo. In Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT; ISBN 978-989-758-763-4, SciTePress, pages 189-196. DOI: 10.5220/0013611800004664

in Bibtex Style

@conference{incoft25,
author={Pranav Bhat and Karthik K P and Shrishail Golappanavar and Rohit Mendigeri and Uday Kulkarni and Shashank Hegde},
title={Poetry Generation Using Transformer Based Model GPT-Neo},
booktitle={Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT},
year={2025},
pages={189-196},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013611800004664},
isbn={978-989-758-763-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Futuristic Technology - Volume 3: INCOFT
TI - Poetry Generation Using Transformer Based Model GPT-Neo
SN - 978-989-758-763-4
AU - Bhat P.
AU - K P K.
AU - Golappanavar S.
AU - Mendigeri R.
AU - Kulkarni U.
AU - Hegde S.
PY - 2025
SP - 189
EP - 196
DO - 10.5220/0013611800004664
PB - SciTePress