A Unified Conceptual Framework Integrating UML and RL for Efficient Reconfiguration Design

Amen Ben Hadj Ali, Samir Ben Ahmed

2024

Abstract

The problem of early exploration of various design choices to anticipate potential runtime changes at design time for complex and highly-dynamic Reconfigurable Control Systems (RCS), is still a real challenge for designers. This paper proposes a novel conceptual framework that integrates the benefits of UML-based modeling with Reinforcement Learning (RL) to overcome this difficulty. Our proposal exploits UML diagrams enriched with OCL constraints to describe the reconfiguration controller structure and dynamics using predefined reconfiguration knowledge. On the other hand, the reconfiguration controller is designed as a RL agent (Reinforcement Learning Reconfiguration Agent or RLRA) able to improve its knowledge through online exploration while running a Q-Learning algorithm. The design process we propose starts with an abstract UML-based specification of RCS. Then, a RL-based framework in Python language will be generated from UML/OCL models by applying a generation algorithm. Finally, the resulting framework will be run to allow the RLRA learning optimized reconfiguration policies and eventually improve first design specifications with learning feedback. The learning phase supports both offline and online learning and is based on a Q-Learning algorithm.

Paper Citation

in Harvard Style

Ben Hadj Ali A. and Ben Ahmed S. (2024). A Unified Conceptual Framework Integrating UML and RL for Efficient Reconfiguration Design. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 604-613. DOI: 10.5220/0012468600003636

in Bibtex Style

@conference{icaart24,
author={Amen Ben Hadj Ali and Samir Ben Ahmed},
title={A Unified Conceptual Framework Integrating UML and RL for Efficient Reconfiguration Design},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2024},
pages={604-613},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012468600003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - A Unified Conceptual Framework Integrating UML and RL for Efficient Reconfiguration Design
SN - 978-989-758-680-4
AU - Ben Hadj Ali A.
AU - Ben Ahmed S.
PY - 2024
SP - 604
EP - 613
DO - 10.5220/0012468600003636
PB - SciTePress