Quantum Advantage Actor-Critic for Reinforcement Learning

Michael Kölle, Mohamad Hgog, Fabian Ritz, Philipp Altmann, Maximilian Zorn, Jonas Stein, Claudia Linnhoff-Popien

2024

Abstract

Quantum computing offers efficient encapsulation of high-dimensional states. In this work, we propose a novel quantum reinforcement learning approach that combines the Advantage Actor-Critic algorithm with variational quantum circuits by substituting parts of the classical components. This approach addresses reinforcement learning’s scalability concerns while maintaining high performance. We empirically test multiple quantum Advantage Actor-Critic configurations with the well known Cart Pole environment to evaluate our approach in control tasks with continuous state spaces. Our results indicate that the hybrid strategy of using either a quantum actor or quantum critic with classical post-processing yields a substantial performance increase compared to pure classical and pure quantum variants with similar parameter counts. They further reveal the limits of current quantum approaches due to the hardware constraints of noisy intermediate-scale quantum computers, suggesting further research to scale hybrid approaches for larger and more complex control tasks.

Paper Citation

in Harvard Style

Kölle M., Hgog M., Ritz F., Altmann P., Zorn M., Stein J. and Linnhoff-Popien C. (2024). Quantum Advantage Actor-Critic for Reinforcement Learning. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART; ISBN 978-989-758-680-4, SciTePress, pages 297-304. DOI: 10.5220/0012383900003636

in Bibtex Style

@conference{icaart24,
author={Michael Kölle and Mohamad Hgog and Fabian Ritz and Philipp Altmann and Maximilian Zorn and Jonas Stein and Claudia Linnhoff-Popien},
title={Quantum Advantage Actor-Critic for Reinforcement Learning},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART},
year={2024},
pages={297-304},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012383900003636},
isbn={978-989-758-680-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART
TI - Quantum Advantage Actor-Critic for Reinforcement Learning
SN - 978-989-758-680-4
AU - Kölle M.
AU - Hgog M.
AU - Ritz F.
AU - Altmann P.
AU - Zorn M.
AU - Stein J.
AU - Linnhoff-Popien C.
PY - 2024
SP - 297
EP - 304
DO - 10.5220/0012383900003636
PB - SciTePress