Authors:
Christoph Brosch
;
Sian Brumm
;
Rolf Krieger
and
Jonas Scheffler
Affiliation:
Institute for Software Systems, University of Applied Sciences Trier, Birkenfeld, Germany
Keyword(s):
Web Scraping, Information Extraction, Large Language Models, Schema-Constrained Output, Product Pages, Pydantic Models, Code Generation.
Abstract:
Generative AI and large language models (LLMs) offer significant potential for automating the extraction of structured information from web pages. In this work, we focus on food product pages from online retailers and explore schema-constrained extraction approaches to retrieve key product attributes, such as ingredient lists and nutrition tables. We compare two LLM-based approaches, direct extraction and indirect extraction via generated functions, evaluating them in terms of accuracy, efficiency, and cost on a curated dataset of 3,000 food product pages from three different online shops. Our results show that although the indirect approach achieves slightly lower accuracy (96.48%, −2.27% compared to direct extraction), it reduces the number of required LLM calls by 95.82%, leading to substantial efficiency gains and lower operational costs. These findings suggest that indirect extraction approaches can provide scalable and cost-effective solutions for large-scale information extrac
tion tasks from template-based web pages using LLMs.
(More)