Training Agents with Neural Networks in Systems with Imperfect Information

Yulia Korukhova, Sergey Kuryshev

2017

Abstract

The paper deals with multi-agent system that represents trading agents acting in the environment with imperfect information. Fictitious play algorithm, first proposed by Brown in 1951, is a popular theoretical model of training agents. However, it is not applicable to larger systems with imperfect information due to its computational complexity. In this paper we propose a modification of the algorithm. We use neural networks for fast approximate calculation of the best responses. An important feature of the algorithm is the absence of agent’s a priori knowledge about the system. Agents’ learning goes through trial and error with winning actions being reinforced and entered into the training set and losing actions being cut from the strategy. The proposed algorithm has been used in a small game with imperfect information. And the ability of the algorithm to remove iteratively dominated strategies of agents' behavior has been demonstrated.

References

  1. Nash, J. 1951. Non-cooperative games. The Annals of Mathematics, Second Series, Volume 54, Issue 2, pp. 286-295.
  2. Kuhn, H.W., 1950. Simplified Two-Person Poker. In Kuhn, H.W.; Tucker, A.W. Contributions to the Theory of Games 1. Princeton University Press. pp. 97-103.
  3. Brown, G.W., 1951. Iterative Solutions of Games by Fictitious Play. In Activity Analysis of Production and Allocation, TC Koopmans (Ed.), New York: Wiley.
  4. Gibson, R., 2014. Regret Minimization in Games and the Development of Champion Multiplayer Computer Poker-Playing Agents. Ph.D. Dissertation, University of Alberta, Dept. of Computing Science.
  5. Johanson, M., 2013. Measuring the Size of Large NoLimit Poker Games. Technical Report TR13-01, Department of Computing Science, University of Alberta.
  6. Zinkevich, M., Johanson, M., Bowling, M., Piccione,C., 2007. Regret Minimization in Games with Incomplete Information. Advances in Neural Information Processing Systems 20 (NIPS).
  7. Osborne M.J., Rubinstein A., 1994. A course in game theory. MIT Press.
  8. Koller, D., Megiddo, N. and von Stengel, B., 1994. Fast algorithms for finding randomized strategies in game trees. Proceedings of the 26th CAN Symposium on the Theory of Computing, pp. 750-759.
Download


Paper Citation


in Harvard Style

Korukhova Y. and Kuryshev S. (2017). Training Agents with Neural Networks in Systems with Imperfect Information . In Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART, ISBN 978-989-758-219-6, pages 296-301. DOI: 10.5220/0006242102960301


in Bibtex Style

@conference{icaart17,
author={Yulia Korukhova and Sergey Kuryshev},
title={Training Agents with Neural Networks in Systems with Imperfect Information},
booktitle={Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,},
year={2017},
pages={296-301},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006242102960301},
isbn={978-989-758-219-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 9th International Conference on Agents and Artificial Intelligence - Volume 1: ICAART,
TI - Training Agents with Neural Networks in Systems with Imperfect Information
SN - 978-989-758-219-6
AU - Korukhova Y.
AU - Kuryshev S.
PY - 2017
SP - 296
EP - 301
DO - 10.5220/0006242102960301