Authors:
Miroslav Kárný
and
František Hůla
Affiliation:
The Czech Academy of Sciences, Inst. of Inf. Theory and Automation, POB 18, 182 08 Prague 8 and Czech Republic
Keyword(s):
Exploitation, Exploration, Bayesian Estimation, Adaptive Systems, Fully Probabilistic Design, Markov Decision Process.
Related
Ontology
Subjects/Areas/Topics:
Agents
;
Artificial Intelligence
;
Cognitive Robotics
;
Computational Intelligence
;
Evolutionary Computing
;
Informatics in Control, Automation and Robotics
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Model-Based Reasoning
;
Robotics and Automation
;
Soft Computing
;
Symbolic Systems
;
Uncertainty in AI
Abstract:
Adaptive decision making learns an environment model serving a design of a decision policy. The policy-generated actions influence both the acquired reward and the future knowledge. The optimal policy properly balances exploitation with exploration. The inherent dimensionality curse of decision making under incomplete knowledge prevents the realisation of the optimal design. This has stimulated repetitive attempts to reach this balance at least approximately. Usually, either: (a) the exploitative reward is enriched by a part reflecting the exploration quality and a feasible approximate certainty-equivalent design is made; or (b) an explorative random noise is added to the purely exploitative actions. This paper avoids the inauspicious (a) and improves (b) by employing the non-standard fully probabilistic design (FPD) of decision policies, which naturally generates random actions. Monte-Carlo experiments confirm its achieved quality. The quality stems from methodological contributions
, which include: (i) an improvement of the relation between FPD and standard Markov decision processes; (ii) a design of an adaptive tuning of an FPD-parameter. The latter also suits for the tuning of the temperature in both simulated annealing and Boltzmann’s machine.
(More)