Authors:
Phong Nguyen
1
;
Takayuki Akiyama
1
and
Hiroki Ohashi
2
Affiliations:
1
Hitachi Ltd, Japan
;
2
Hitachi Europe GmbH, Germany
Keyword(s):
Reinforcement Learning, Experience Replay, Similarity, Distance, Experience Filtering.
Related
Ontology
Subjects/Areas/Topics:
Agents
;
AI and Creativity
;
Artificial Intelligence
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Evolutionary Computing
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Industrial Applications of AI
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Methodologies and Methods
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Robot and Multi-Robot Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Symbolic Systems
;
Theory and Methods
Abstract:
We propose a stochastic method of storing a new experience into replay memory to increase the performance
of the Deep Q-learning (DQL) algorithm, especially under the condition of a small memory. The conventional
standard DQL method with the Prioritized Experience Replay method attempts to use experiences in the replay
memory for improving learning efficiency; however, it does not guarantee the diversity of experience in the
replay memory. Our method calculates the similarity of a new experience with other existing experiences in
the memory based on a distance function and determines whether to store this new experience stochastically.
This method leads to the improvement in experience diversity in the replay memory and better utilization of
rare experiences during the training process. In an experiment to train a moving robot, our proposed method
improved the performance of the standard DQL algorithm with a memory buffer of less than 10,000 stored
experiences.