Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains

Divya Srivastava, Spencer Frazier, Mark Riedl, Karen Feigh

Abstract

Interactive machine learning involves humans teaching with agents during their learning process. As this field grows, it is pertinent that laymen teachers, i.e. those without programming or extensive ML experience, are able to easily and effectively teach the agents. Previous work has investigated which factors contribute to the teacher’s experience when training agents in a fully observable domain. In this paper, we investigate how four different interaction methods affect agent performance and teacher experience in partially observable domains. As the domain in which the agent is learning becomes more complex, it accumulates less reward overall and needs more advice from the teacher. It is found that the most salient features that affect teacher satisfaction are agent compliance to advice, response speed, instruction quantity required, and reliability in agent response. It is suggested that machine learning algorithms incorporate a short time delay in the agent’s response and maximize the agent’s adherence to advice to increase reliability of the agent’s behavior. The need to generalize advice over time to reduce the amount of instruction needed varies depending on the presence of penalties in the environment.

Download


Paper Citation


in Harvard Style

Srivastava D., Frazier S., Riedl M. and Feigh K. (2021). Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains.In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: HUCAPP, ISBN 978-989-758-488-6, pages 174-181. DOI: 10.5220/0010240101740181


in Bibtex Style

@conference{hucapp21,
author={Divya Srivastava and Spencer Frazier and Mark Riedl and Karen Feigh},
title={Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: HUCAPP,},
year={2021},
pages={174-181},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010240101740181},
isbn={978-989-758-488-6},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 1: HUCAPP,
TI - Effect of Interaction Design of Reinforcement Learning Agents on Human Satisfaction in Partially Observable Domains
SN - 978-989-758-488-6
AU - Srivastava D.
AU - Frazier S.
AU - Riedl M.
AU - Feigh K.
PY - 2021
SP - 174
EP - 181
DO - 10.5220/0010240101740181