Assessing Grade Levels of Texts via Local Search over Fine-Tuned LLMs

Changfeng Yu, Jie Wang

2025

Abstract

The leading method for determining the grade level of a written work involves training an SVC model on hundreds of linguistic features (LFs) and a predicted grade generated by a fine-tuned large language model (FT-LLM). When applied to a diverse dataset of materials for grades 3 through 12 spanning 33 genres, however, this approach yields a poor accuracy of less than 51%. To address this issue, we devise a novel local-search algorithm called LS-LLM independent of LFs. LS-LLM employs different FT-LLMs to identify a genre, predict a genre-aware grade, and compare readability of the text to a randomly selected set of annotated works from the same genre and grade level. We demonstrate that LS-LLM significantly improves accuracy, exceeding 65%, and achieves over 92% accuracy within a one-grade error margin, making it viable for certain practical applications. To further validate its robustness, we show that LS-LLM also enhances the performance of the leading method on the WeeBit dataset used in prior research.

Paper Citation

in Harvard Style

Yu C. and Wang J. (2025). Assessing Grade Levels of Texts via Local Search over Fine-Tuned LLMs. In Proceedings of the 17th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR; ISBN , SciTePress, pages 224-231. DOI: 10.5220/0013674400004000

in Bibtex Style

@conference{kdir25,
author={Changfeng Yu and Jie Wang},
title={Assessing Grade Levels of Texts via Local Search over Fine-Tuned LLMs},
booktitle={Proceedings of the 17th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR},
year={2025},
pages={224-231},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013674400004000},
isbn={},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1: KDIR
TI - Assessing Grade Levels of Texts via Local Search over Fine-Tuned LLMs
SN -
AU - Yu C.
AU - Wang J.
PY - 2025
SP - 224
EP - 231
DO - 10.5220/0013674400004000
PB - SciTePress