Solving Job Shop Problems with Neural Monte Carlo Tree Search

Marco Kemmerling, Anas Abdelrazeq, Robert Schmitt

2024

Abstract

Job shop scheduling is a common NP-hard problem that finds many applications in manufacturing and beyond. A variety of methods to solve job shop problems exist to address different requirements arising from individual use cases. Recently, model-free reinforcement learning is increasingly receiving attention as a method to train agents capable of scheduling. In contrast, model-based reinforcement learning is less well studied in job scheduling. However, it may be able to improve upon its model-free counterpart by dynamically spending additional planning budget to refine solutions according to the available scheduling time at any given moment. Neural Monte Carlo tree search, a family of model-based algorithms including AlphaZero is especially suitable for discrete problems such as the job shop problem. Our aim is to find suitable designs of neural Monte Carlo tree search agents for the job shop problem by systematically varying certain parameters and design components. We find that different choices for the evaluation phase of the tree search have the biggest impact on performance and conclude that agents with a combination of node value initialization using learned value functions and roll-out based evaluation lead to the most favorable performance.

Paper Citation

in Harvard Style

Kemmerling M., Abdelrazeq A. and Schmitt R. (2024). Solving Job Shop Problems with Neural Monte Carlo Tree Search. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 149-158. DOI: 10.5220/0012311700003636

in Bibtex Style

@conference{icaart24,
author={Marco Kemmerling and Anas Abdelrazeq and Robert Schmitt},
title={Solving Job Shop Problems with Neural Monte Carlo Tree Search},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2024},
pages={149-158},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012311700003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - Solving Job Shop Problems with Neural Monte Carlo Tree Search
SN - 978-989-758-680-4
AU - Kemmerling M.
AU - Abdelrazeq A.
AU - Schmitt R.
PY - 2024
SP - 149
EP - 158
DO - 10.5220/0012311700003636
PB - SciTePress