loading
Documents

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Raphael Fonteneau 1 ; Susan A. Murphy 2 ; Louis Wehenkel 1 and Damien Ernst 3

Affiliations: 1 University of Liège, Belgium ; 2 University of Michigan, United States ; 3 University of Liege, Belgium

ISBN: 978-989-674-021-4

Keyword(s): Reinforcement learning, Prior knowledge, Cautious generalization.

Related Ontology Subjects/Areas/Topics: Artificial Intelligence ; Computational Intelligence ; Evolutionary Computing ; Knowledge Discovery and Information Retrieval ; Knowledge-Based Systems ; Machine Learning ; Soft Computing ; Symbolic Systems ; Uncertainty in AI

Abstract: In the context of a deterministic Lipschitz continuous environment over continuous state spaces, finite action spaces, and a finite optimization horizon, we propose an algorithm of polynomial complexity which exploits weak prior knowledge about its environment for computing from a given sample of trajectories and for a given initial state a sequence of actions. The proposed Viterbi-like algorithm maximizes a recently proposed lower bound on the return depending on the initial state, and uses to this end prior knowledge about the environment provided in the form of upper bounds on its Lipschitz constants. It thereby avoids, in way depending on the initial state and on the the prior knowledge, those regions of the state space where the sample is too sparse to make safe generalizations. Our experiments show that it can lead to more cautious policies than algorithms combining dynamic programming with function approximators. We give also a condition on the sample sparsity ensuring that, for a given initial state, the proposed algorithm produces an optimal sequence of actions in open-loop. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.156.51.193

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Fonteneau R., Murphy S., Wehenkel L. and Ernst D. (2010). A CAUTIOUS APPROACH TO GENERALIZATION IN REINFORCEMENT LEARNING.In Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-674-021-4, pages 64-73. DOI: 10.5220/0002726900640073

@conference{icaart10,
author={Raphael Fonteneau and Susan A. Murphy and Louis Wehenkel and Damien Ernst},
title={A CAUTIOUS APPROACH TO GENERALIZATION IN REINFORCEMENT LEARNING},
booktitle={Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2010},
pages={64-73},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002726900640073},
isbn={978-989-674-021-4},
}

TY - CONF

JO - Proceedings of the 2nd International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - A CAUTIOUS APPROACH TO GENERALIZATION IN REINFORCEMENT LEARNING
SN - 978-989-674-021-4
AU - Fonteneau R.
AU - Murphy S.
AU - Wehenkel L.
AU - Ernst D.
PY - 2010
SP - 64
EP - 73
DO - 10.5220/0002726900640073

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.