Authors:
Andreas Sedlmeier
;
Michael Kölle
;
Robert Müller
;
Leo Baudrexel
and
Claudia Linnhoff-Popien
Affiliation:
LMU Munich, Munich, Germany
Keyword(s):
Uncertainty, Multimodality, World Models, Model-based Deep Reinforcement Learning, Mixture-density Networks.
Abstract:
Model-based Deep Reinforcement Learning (RL) assumes the availability of a model of an environment’s underlying transition dynamics. This model can be used to predict future effects of an agent’s possible actions. When no such model is available, it is possible to learn an approximation of the real environment, e.g. by using generative neural networks, sometimes also called World Models. As most real-world environments are stochastic in nature and the transition dynamics are oftentimes multimodal, it is important to use a modelling technique that is able to reflect this multimodal uncertainty. In order to safely deploy such learning systems in the real world, especially in an industrial context, it is paramount to consider these uncertainties. In this work, we analyze existing and propose new metrics for the detection and quantification of multimodal uncertainty in RL based World Models. The correct modelling & detection of uncertain future states lays the foundation for handling cri
tical situations in a safe way, which is a prerequisite for deploying RL systems in real-world settings.
(More)