loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: Fernando Fradique Duarte 1 ; Nuno Lau 2 ; Artur Pereira 2 and Luís Reis 3

Affiliations: 1 Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, Aveiro, Portugal ; 2 Department of Electronics, Telecommunications and Informatics, University of Aveiro, Aveiro, Portugal ; 3 Faculty of Engineering, Department of Informatics Engineering, University of Porto, Porto, Portugal

Keyword(s): Deep Reinforcement Learning, Multi-Head Attention, Advantage Actor-Critic.

Abstract: Deep Learning agents are known to be very sensitive to their parameterization values. Attention-based Deep Reinforcement Learning agents further complicate this issue due to the additional parameterization associated to the computation of their attention function. One example of this concerns the number of attention heads to use when dealing with multi-head attention-based agents. Usually, these hyperparameters are set manually, which may be neither optimal nor efficient. This work addresses the issue of choosing the appropriate number of attention heads dynamically, by endowing the agent with a policy πh trained with policy gradient. At each timestep of agent-environment interaction, πh is responsible for choosing the most suitable number of attention heads according to the contextual memory of the agent. This dynamic parameterization is compared to a static parameterization in terms of performance. The role of πh is further assessed by providing additional analysis concerning the d istribution of the number of attention heads throughout the training procedure and the course of the game. The Atari 2600 videogame benchmark was used to perform and validate all the experiments. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 34.239.158.223

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Fradique Duarte, F.; Lau, N.; Pereira, A. and Reis, L. (2024). Dynamically Choosing the Number of Heads in Multi-Head Attention. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-680-4; ISSN 2184-433X, SciTePress, pages 358-367. DOI: 10.5220/0012384500003636

@conference{icaart24,
author={Fernando {Fradique Duarte}. and Nuno Lau. and Artur Pereira. and Luís Reis.},
title={Dynamically Choosing the Number of Heads in Multi-Head Attention},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2024},
pages={358-367},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012384500003636},
isbn={978-989-758-680-4},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - Dynamically Choosing the Number of Heads in Multi-Head Attention
SN - 978-989-758-680-4
IS - 2184-433X
AU - Fradique Duarte, F.
AU - Lau, N.
AU - Pereira, A.
AU - Reis, L.
PY - 2024
SP - 358
EP - 367
DO - 10.5220/0012384500003636
PB - SciTePress