loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Steven Loscalzo 1 and Robert Wright 2

Affiliations: 1 Binghamton University, United States ; 2 Air Force Research Laboratory, United States

Keyword(s): State space abstraction, Reinforcement learning.

Related Ontology Subjects/Areas/Topics: Artificial Intelligence ; Biomedical Engineering ; Biomedical Signal Processing ; Computational Intelligence ; Evolutionary Computing ; Health Engineering and Technology Applications ; Human-Computer Interaction ; Knowledge Discovery and Information Retrieval ; Knowledge-Based Systems ; Machine Learning ; Methodologies and Methods ; Neural Networks ; Neurocomputing ; Neurotechnology, Electronics and Informatics ; Pattern Recognition ; Physiological Computing Systems ; Sensor Networks ; Signal Processing ; Soft Computing ; State Space Search ; Symbolic Systems ; Theory and Methods

Abstract: Applying reinforcement learning techniques in continuous environments is challenging because there are infinitely many states to visit in order to learn an optimal policy. To make this situation tractable, abstractions are often used to reduce the infinite state space down to a small and finite one. Some of the more powerful and commonplace abstractions, tiling abstractions such as CMAC, work by aggregating many base states into a single abstract state. Unfortunately, significant manual effort is often necessary in order to apply them to nontrivial control problems. Here we develop an automatic state space aggregation algorithm, Maximum Density Separation, which can produce a meaningful abstraction with minimal manual effort. This method leverages the density of observations in the space to construct a partition and aggregate states in a dense region to the same abstract state. We show that the abstractions produced by this method on two benchmark reinforcement learning problems can outperform fixed tiling methods in terms of both the convergence rate of a learning algorithm and the number of abstract states needed. (More)

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 44.220.44.148

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Loscalzo, S. and Wright, R. (2011). AUTOMATIC STATE SPACE AGGREGATION USING A DENSITY BASED TECHNIQUE. In Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART; ISBN 978-989-8425-40-9; ISSN 2184-433X, SciTePress, pages 249-256. DOI: 10.5220/0003150202490256

@conference{icaart11,
author={Steven Loscalzo. and Robert Wright.},
title={AUTOMATIC STATE SPACE AGGREGATION USING A DENSITY BASED TECHNIQUE},
booktitle={Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART},
year={2011},
pages={249-256},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003150202490256},
isbn={978-989-8425-40-9},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 3rd International Conference on Agents and Artificial Intelligence - Volume 2: ICAART
TI - AUTOMATIC STATE SPACE AGGREGATION USING A DENSITY BASED TECHNIQUE
SN - 978-989-8425-40-9
IS - 2184-433X
AU - Loscalzo, S.
AU - Wright, R.
PY - 2011
SP - 249
EP - 256
DO - 10.5220/0003150202490256
PB - SciTePress