IMPROVED REVISION OF RANKING FUNCTIONS FOR THE GENERALIZATION OF BELIEF IN THE CONTEXT OF UNOBSERVED VARIABLES

Klaus Häming, Gabriele Peters

Abstract

To enable a reinforcement learning agent to acquire symbolical knowledge, we augment it with a high-level knowledge representation. This representation consists of ordinal conditional functions (OCF) which allow it to rank world models. By this means the agent is enabled to complement the self-organizing capabilities of the low-level reinforcement learning sub-system by reasoning capabilities of a high-level learning component. We briefly summarize the state-of-the-art method how new information is included into the OCF. To improve the emergence of plausible behavior, we then introduce a modification of this method. The viability of this modification is examined first, for the inclusion of conditional information with negated consequents and second, for the generalization of belief in the context of unobserved variables. Besides providing a theoretical justification for this modification, we also show the advantages of our approach in comparison to the state-of-the-art method of revision in a reinforcement learning application.

References

  1. Alchourron, C. E., Gardenfors, P., and Makinson, D. (1985). On the logic of theory change: Partial meet contraction and revision functions. J. Symbolic Logic, 50(2):510-530.
  2. Anderson, J. R. (1983). The architecture of cognition. Hardvard University Press, Cambridge, MA.
  3. Darwiche, A. and Pearl, J. (1996). On the logic of iterated belief revision. Artificial intelligence, 89:1-29.
  4. Dzeroski, S., Raedt, L. D., and Driessens, K. (2001). Relational reinforcement learning. In Machine Learning, volume 43, pages 7-52.
  5. Gombert, J.-E. (2003). Implicit and explicit learning to read: Implication as for subtypes of dyslexia. Current psychology letters, 1(10).
  6. Häming, K. and Peters, G. (2010). An alternative approach to the revision of ordinal conditional functions in the context of multi-valued logic. In Proceedings of the 20th International Conference on Artificial Neural Networks, volume LNCS 6353, pages 200-203.
  7. Kern-Isberner, G. (2001). Conditionals in nonmonotonic reasoning and belief revision: considering conditionals as agents. Springer-Verlag New York, Inc.
  8. Leopold, T., Kern Isberner, G., and Peters, G. (2008). Combining reinforcement learning and belief revision: A learning system for active vision. In Proceedings of the 19th British Machine Vision Conference, pages 473-482.
  9. Reber, A. S. (1989). Implicit learning and tacit knowledge. Journal of Experimental Psycology: General, 3(118):219-235.
  10. Spohn, W. (2009). A survey of ranking theory. In Degrees of Belief. Springer.
  11. Sun, R., Merrill, E., and Peterson, T. (2001). From implicit skills to explicit knowledge: A bottom-up model of skill learning. In Cognitive Science, volume 25(2), pages 203-244.
  12. Sun, R., Terry, C., and Slusarz, P. (2005). The interaction of the explicit and the implicit in skill learning: A dualprocess approach. Psychological Review, 112:159- 192.
  13. Sun, R., Zhang, X., Slusarz, P., and Mathews, R. (2006). The interaction of implicit learning, explicit hypothesis testing, and implicit-to-explicit knowledge extraction. Neural Networks, 1(20):34-47.
  14. Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press, Cambridge.
  15. Ye, C., Yung, N. H. C., and Wang, D. (2003). A fuzzy controller with supervised learning assisted reinforcement learning algorithm for obstacle avoidance. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 33(1):17-27.
Download


Paper Citation


in Harvard Style

Häming K. and Peters G. (2011). IMPROVED REVISION OF RANKING FUNCTIONS FOR THE GENERALIZATION OF BELIEF IN THE CONTEXT OF UNOBSERVED VARIABLES . In Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011) ISBN 978-989-8425-84-3, pages 118-123. DOI: 10.5220/0003669501180123


in Bibtex Style

@conference{ncta11,
author={Klaus Häming and Gabriele Peters},
title={IMPROVED REVISION OF RANKING FUNCTIONS FOR THE GENERALIZATION OF BELIEF IN THE CONTEXT OF UNOBSERVED VARIABLES},
booktitle={Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011)},
year={2011},
pages={118-123},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003669501180123},
isbn={978-989-8425-84-3},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Neural Computation Theory and Applications - Volume 1: NCTA, (IJCCI 2011)
TI - IMPROVED REVISION OF RANKING FUNCTIONS FOR THE GENERALIZATION OF BELIEF IN THE CONTEXT OF UNOBSERVED VARIABLES
SN - 978-989-8425-84-3
AU - Häming K.
AU - Peters G.
PY - 2011
SP - 118
EP - 123
DO - 10.5220/0003669501180123