loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Gavin Rens 1 ; Jean-François Raskin 2 ; Raphaël Reynouard 2 and Giuseppe Marra 1

Affiliations: 1 DTAI Group, KU Leuven, Belgium ; 2 Université Libre de Bruxelles, Belgium

Keyword(s): non-Markovian Rewards, Learning Mealy Machines, Angluin’s Algorithm.

Abstract: There are situations in which an agent should receive rewards only after having accomplished a series of previous tasks, that is, rewards are non-Markovian. One natural and quite general way to represent history- dependent rewards is via a Mealy machine. In our formal setting, we consider a Markov decision process (MDP) that models the dynamics of the environment in which the agent evolves and a Mealy machine synchronized with this MDP to formalize the non-Markovian reward function. While the MDP is known by the agent, the reward function is unknown to the agent and must be learned. Our approach to overcome this challenge is to use Angluin’s L∗ active learning algorithm to learn a Mealy machine representing the underlying non-Markovian reward machine (MRM). Formal methods are used to determine the optimal strategy for answering so-called membership queries posed by L∗. Moreover, we prove that the expected reward achieved will eventually be at least as much as a given, reasonable valu e provided by a domain expert. We evaluate our framework on two problems. The results show that using L∗ to learn an MRM in a non-Markovian reward decision process is effective. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.140.185.170

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Rens, G.; Raskin, J.; Reynouard, R. and Marra, G. (2021). Online Learning of non-Markovian Reward Models. In Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-758-484-8; ISSN 2184-433X, SciTePress, pages 74-86. DOI: 10.5220/0010212000740086

@conference{icaart21,
author={Gavin Rens. and Jean{-}Fran\c{C}ois Raskin. and Raphaël Reynouard. and Giuseppe Marra.},
title={Online Learning of non-Markovian Reward Models},
booktitle={Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2021},
pages={74-86},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010212000740086},
isbn={978-989-758-484-8},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 13th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - Online Learning of non-Markovian Reward Models
SN - 978-989-758-484-8
IS - 2184-433X
AU - Rens, G.
AU - Raskin, J.
AU - Reynouard, R.
AU - Marra, G.
PY - 2021
SP - 74
EP - 86
DO - 10.5220/0010212000740086
PB - SciTePress

- Science and Technology Publications, Lda.
RESOURCES

Proceedings

Papers

Authors

Ontology

CONTACTS

Science and Technology Publications, Lda
Avenida de S. Francisco Xavier, Lote 7 Cv. C,
2900-616 Setúbal, Portugal.

Phone: +351 265 520 185 (National fixed network call)
Fax: +351 265 520 186
Email: info@scitepress.org

EXTERNAL LINKS

PRIMORIS

INSTICC

SCITEVENTS

CROSSREF

PROCEEDINGS SUBMITTED FOR INDEXATION BY:

dblp

Ei Compendex

SCOPUS

Semantic Scholar

Google Scholar

Microsoft Academic