Assisted Feature Engineering and Feature Learning to Build Knowledge-based Agents for Arcade Games

Bastian Andelefski, Stefan Schiffer

Abstract

Human knowledge can greatly increase the performance of autonomous agents. Leveraging this knowledge is sometimes neither straightforward nor easy. In this paper, we present an approach for assisted feature engineering and feature learning to build knowledge-based agents for three arcade games within the Arcade Learning Environment. While existing approaches mostly use model-free approaches we aim at creating a descriptive set of features for world modelling and building agents. To this end, we provide (visual) assistance in identifying and modelling features from RAM, we allow for learning features based on labeled game data, and we allow for creating basic agents using the above features. In our evaluation, we compare different methods to learn features from the RAM. We then compare several agents using different sets of manual and learned features with one another and with the state-of-the-art.

References

  1. Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. (2013). The arcade learning environment: An evaluation platform for general agents. Journal of Artificial Intelligence Research, 47:253-279.
  2. Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
  3. Boser, B. E., Guyon, I. M., and Vapnik, V. N. (1992). A training algorithm for optimal margin classifiers. In Proceedings of the fifth annual workshop on Computational learning theory, pages 144-152. ACM.
  4. Breiman, L. (2001). Random forests. Machine Learning, 45(1):5-32.
  5. Chang, C.-C. and Lin, C.-J. (2011). Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol., 2(3):27:1-27:27.
  6. Coates, A. and Ng, A. Y. (2012). Learning feature representations with k-means. In Neural Networks: Tricks of the Trade, pages 561-580. Springer.
  7. Fernández-Delgado, M., Cernadas, E., Barro, S., and Amorim, D. (2014). Do we need hundreds of classifiers to solve real world classification problems?The Journal of Machine Learning Research, 15(1):3133- 3181.
  8. Fix, E. and Hodges Jr, J. L. (1951). Discriminatory analysisnonparametric discrimination: consistency properties. Technical report, DTIC Document.
  9. Hausknecht, M., Khandelwal, P., Miikkulainen, R., and Stone, P. (2012). Hyperneat-ggp: A hyperneat-based atari general game player. In Proceedings of the fourteenth international conference on Genetic and evolutionary computation conference, pages 217-224.
  10. Ho, T. K. (1998). The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(8):832-844.
  11. Lipovetzky, N., Ramirez, M., and Geffner, H. (2015). Classical planning with simulators: Results on the atari video games. Proc. IJCAI 2015.
  12. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. NIPS Deep Learning Workshop 2013.
  13. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., and Hassabis, D. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540):529-533.
  14. Naddaf, Y. (2010). Game-independent ai agents for playing atari 2600 console games. University of Alberta.
  15. Nair, A., Srinivasan, P., and Blackwell, S. (2015). Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296.
  16. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. The Journal of Machine Learning Research, 12:2825-2830.
  17. Sculley, D. et al. (2011). Results from a semi-supervised feature learning competition. NIPS 2011 Workshop on Deep Learning and Unsupervised Feature Learning.
  18. Vapnik, V. and Chapelle, O. (2000). Bounds on error expectation for support vector machines. Neural computation, 12(9):2013-2036.
Download


Paper Citation


in Harvard Style

Andelefski B. and Schiffer S. (2017). Assisted Feature Engineering and Feature Learning to Build Knowledge-based Agents for Arcade Games . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-220-2, pages 228-238. DOI: 10.5220/0006202602280238


in Bibtex Style

@conference{icaart17,
author={Bastian Andelefski and Stefan Schiffer},
title={Assisted Feature Engineering and Feature Learning to Build Knowledge-based Agents for Arcade Games},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2017},
pages={228-238},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006202602280238},
isbn={978-989-758-220-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Assisted Feature Engineering and Feature Learning to Build Knowledge-based Agents for Arcade Games
SN - 978-989-758-220-2
AU - Andelefski B.
AU - Schiffer S.
PY - 2017
SP - 228
EP - 238
DO - 10.5220/0006202602280238