Effectiveness of Whisper’s Fine-Tuning for Domain-Specific Use Cases in the Industry

Daniel Pawlowicz, Jule Weber, Claudia Dukino

2025

Abstract

The integration of Speech-to-Text (STT) technology has the potential to enhance the efficiency of industrial workflows. However, standard speech models demonstrate suboptimal performance in domain-specific use cases. In order to gain user trust, it is essential to ensure accurate transcription, which can be achieved through the fine-tuning of the model to the specific domain. OpenAI’s Whisper was selected as the initial model and subsequently fine-tuned with domain-specific real-world recordings. The fine-tuned model outperforms the initial model in terms of transcription of technical jargon, as evidenced by the results of the study. The fine-tuned model achieved a validation loss of 1.75 and a Word Error Rate (WER) of 1. In addition to improving accuracy, this approach addresses the challenges of noisy environments and speaker variability that are common in real-world industrial environments. The present study demonstrates the efficacy of fine-tuning the Whisper model to new vocabulary with technical jargon, thereby underscoring the value of model adaptation for domain-specific use cases.

Paper Citation

in Harvard Style

Pawlowicz D., Weber J. and Dukino C. (2025). Effectiveness of Whisper’s Fine-Tuning for Domain-Specific Use Cases in the Industry. In Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-737-5, SciTePress, pages 1396-1403. DOI: 10.5220/0013378100003890

in Bibtex Style

@conference{icaart25,
author={Daniel Pawlowicz and Jule Weber and Claudia Dukino},
title={Effectiveness of Whisper’s Fine-Tuning for Domain-Specific Use Cases in the Industry},
booktitle={Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2025},
pages={1396-1403},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013378100003890},
isbn={978-989-758-737-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - Effectiveness of Whisper’s Fine-Tuning for Domain-Specific Use Cases in the Industry
SN - 978-989-758-737-5
AU - Pawlowicz D.
AU - Weber J.
AU - Dukino C.
PY - 2025
SP - 1396
EP - 1403
DO - 10.5220/0013378100003890
PB - SciTePress