Integration of Efficient Deep Q-Network Techniques Into QT-Opt Reinforcement Learning Structure

Shudao Wei, Chenxing Li, Chenxing Li, Jan Seyler, Shahram Eivazi, Shahram Eivazi

2023

Abstract

There has been a growing interest in the development of offline reinforcement learning (RL) algorithms for real-world applications. For example, offline algorithms like qt-opt has demonstrated an impressive performance in grasping task. The primary motivation is to avoid the challenges associated with online data collection. However, these algorithms require extremely large dataset as well as huge computational resources. In this paper we investigate the applicability of well known improvement techniques from Deep Q-learning (DQN) methods to the QT-Opt offline algorithm, for both on-policy and mixed-policy training. For the first time, we show that prioritized experience replay(PER), noisy network, and distributional DQN can be used within QT-Opt framework. As result,for example, in a reacher environment from Pybullet simulation, we observe an obvious improvements in the learning process for the integrated techniques.

Paper Citation

in Harvard Style

Wei S., Li C., Seyler J. and Eivazi S. (2023). Integration of Efficient Deep Q-Network Techniques Into QT-Opt Reinforcement Learning Structure. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 592-599. DOI: 10.5220/0011715000003393

in Bibtex Style

@conference{icaart23,
author={Shudao Wei and Chenxing Li and Jan Seyler and Shahram Eivazi},
title={Integration of Efficient Deep Q-Network Techniques Into QT-Opt Reinforcement Learning Structure},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={592-599},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011715000003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Integration of Efficient Deep Q-Network Techniques Into QT-Opt Reinforcement Learning Structure
SN - 978-989-758-623-1
AU - Wei S.
AU - Li C.
AU - Seyler J.
AU - Eivazi S.
PY - 2023
SP - 592
EP - 599
DO - 10.5220/0011715000003393