Authors:
Michael Castronovo
;
Vincent François-Lavet
;
Raphaël Fonteneau
;
Damien Ernst
and
Adrien Couëtoux
Affiliation:
Montefiore Institute and Universite de Liège, Belgium
Keyword(s):
Bayesian Reinforcement Learning, Artificial Neural Networks, Offline Policy Search.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Artificial Intelligence and Decision Support Systems
;
Bayesian Networks
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Enterprise Information Systems
;
Evolutionary Computing
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Symbolic Systems
;
Theory and Methods
Abstract:
Bayesian Reinforcement Learning (BRL) agents aim to maximise the expected collected rewards obtained when interacting with an unknown Markov Decision Process (MDP) while using some prior knowledge.
State-of-the-art BRL agents rely on frequent updates of the belief on the MDP, as new observations of the environment are made.
This offers theoretical guarantees to converge to an optimum, but is computationally intractable, even on small-scale problems.
In this paper, we present a method that circumvents this issue by training a parametric policy able to recommend an action directly from raw observations.
Artificial Neural Networks (ANNs) are used to represent this policy, and are trained on the trajectories sampled from the prior.
The trained model is then used online, and is able to act on the real MDP at a very low computational cost.
Our new algorithm shows strong empirical performance, on a wide range of test problems, and is robust to inaccuracies of the prior distribution.