loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Federico Galatolo ; Mario Cimino and Gigliola Vaglini

Affiliation: Department of Information Engineering, University of Pisa, Largo L. Lazzarino 1, Pisa, Italy

Keyword(s): Deep Learning, Natural Language Processing, Generative Pre-trained Transformers, Zero-shot Learning, Mathematical Problem Solving.

Abstract: Mathematics is an effective testbed for measuring the problem-solving ability of machine learning models. The current benchmark for deep learning-based solutions is grade school math problems: given a natural language description of a problem, the task is to analyse the problem, exploit heuristics generated from a very large set of solved examples, and then generate an answer. In this paper, a descendant of the third generation of Generative Pre-trained Transformer Networks (GPT-3) is used to develop a zero-shot learning approach, to solve this problem. The proposed approach shows that coding based problem-solving is more effective than the natural language reasoning based one. Specifically, the architectural solution is built upon OpenAI Codex, a descendant of GPT-3 for programming tasks, trained on public GitHub repositories, the world’s largest source code hosting service. Experimental results clearly show the potential of the approach: by exploiting the Python as programming lang uage, proposed pipeline achieves the 18.63% solve rate against the 6.82% of GPT-3. Finally, by using a fine-tuned verifier, the correctness of the answer can be ranked at runtime, and then improved by generating a predefined number of trials. With this approach, for 10 trials and an ideal verifier, the proposed pipeline achieves 54.20% solve rate. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.238.118.80

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Galatolo, F.; Cimino, M. and Vaglini, G. (2022). Zero-shot Mathematical Problem Solving via Generative Pre-trained Transformers. In Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-758-569-2; ISSN 2184-4992, pages 479-483. DOI: 10.5220/0011032400003179

@conference{iceis22,
author={Federico Galatolo. and Mario Cimino. and Gigliola Vaglini.},
title={Zero-shot Mathematical Problem Solving via Generative Pre-trained Transformers},
booktitle={Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2022},
pages={479-483},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011032400003179},
isbn={978-989-758-569-2},
issn={2184-4992},
}

TY - CONF

JO - Proceedings of the 24th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - Zero-shot Mathematical Problem Solving via Generative Pre-trained Transformers
SN - 978-989-758-569-2
IS - 2184-4992
AU - Galatolo, F.
AU - Cimino, M.
AU - Vaglini, G.
PY - 2022
SP - 479
EP - 483
DO - 10.5220/0011032400003179