Ensemble UCT Needs High Exploitation

S. Ali Mirsoleimani, Aske Plaat, Jaap van den Herik

Abstract

Recent results have shown that the MCTS algorithm (a new, adaptive, randomized optimization algorithm) is effective in a remarkably diverse set of applications in Artificial Intelligence, Operations Research, and High Energy Physics. MCTS can find good solutions without domain dependent heuristics, using the UCT formula to balance exploitation and exploration. It has been suggested that the optimum in the exploitation-exploration balance differs for different search tree sizes: small search trees needs more exploitation; large search trees need more exploration. Small search trees occur in variations of MCTS, such as parallel and ensemble approaches. This paper investigates the possibility of improving the performance of Ensemble UCT by increasing the level of exploitation. As the search trees become smaller we achieve an improved performance. The results are important for improving the performance of large scale parallelism of MCTS.

References

  1. Arneson, B., Hayward, R. B., and Henderson, P. (2010). Monte Carlo Tree Search in Hex. IEEE Transactions on Computational Intelligence and AI in Games, 2(4):251-258.
  2. Herik, J. (2008a). Parallel Monte-Carlo Tree Search. In the 6th Internatioal Conference on Computers and Games, volume 5131, pages 60-71. Springer Berlin Heidelberg.
  3. Chaslot, G. M. J. B., Winands, M. H. M., van den Herik, J., Uiterwijk, J. W. H. M., and Bouzy, B. (2008b). Progressive strategies for Monte-Carlo tree search. New Mathematics and Natural Computation, 4(03):343- 357.
  4. Coulom, R. (2006). Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search. In Proceedings of the 5th International Conference on Computers and Games, volume 4630 of CG'06, pages 72-83. Springer-Verlag.
  5. Fern, A. and Lewis, P. (2011). Ensemble Monte-Carlo Planning: An Empirical Study. In ICAPS, pages 58-65.
  6. Galil, Z. and Italiano, G. F. (1991). Data Structures and Algorithms for Disjoint Set Union Problems. ACM Comput. Surv., 23(3):319-344.
  7. Gelly, S. and Silver, D. (2007). Combining online and offline knowledge in UCT. In the 24th International Conference on Machine Learning, pages 273-280, New York, USA. ACM Press.
  8. Kocsis, L. and Szepesvári, C. (2006). Machine Learning: ECML 2006, volume 4212 of Lecture Notes in Computer Science. Springer Berlin Heidelberg.
  9. Kuipers, J., Plaat, A., Vermaseren, J., and van den Herik, J. (2013). Improving Multivariate Horner Schemes with Monte Carlo Tree Search. Computer Physics Communications, 184(11):2391-2395.
  10. Mirsoleimani, S. A., Plaat, A., van den Herik, J., and Vermaseren, J. (2015). Parallel Monte Carlo Tree Search from Multi-core to Many-core Processors. In ISPA 2015 : The 13th IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA), pages 77-83, Helsinki.
  11. Mirsoleimani, S. A., Plaat, A., Vermaseren, J., and van den Herik, J. (2014). Performance analysis of a 240 thread tournament level MCTS Go program on the Intel Xeon Phi. In The 2014 European Simulation and Modeling Conference (ESM'2014), pages 88-94, Porto, Portugal. Eurosis.
  12. Romein, J. W. (2001). Multigame - An Environment for Distributed Game-Tree Search. PhD thesis, Vrije Universiteit.
  13. Ruijl, B., Vermaseren, J., Plaat, A., and van den Herik, J. (2014). Combining Simulated Annealing and Monte Carlo Tree Search for Expression Simplification. Proceedings of ICAART Conference 2014, 1(1):724-731.
  14. Soejima, Y., Kishimoto, A., and Watanabe, O. (2010). Evaluating Root Parallelization in Go. IEEE Transactions on Computational Intelligence and AI in Games, 2(4):278-287.
  15. Teytaud, F. and Dehos, J. (2015). One the Tactical and Strategic Behaviour of MCTS When Biasing Random Simulations. ICCA Journal, 38(2):67-80.
Download


Paper Citation


in Harvard Style

Mirsoleimani S., Plaat A. and van den Herik J. (2016). Ensemble UCT Needs High Exploitation . In Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART, ISBN 978-989-758-172-4, pages 370-376. DOI: 10.5220/0005711603700376


in Bibtex Style

@conference{icaart16,
author={S. Ali Mirsoleimani and Aske Plaat and Jaap van den Herik},
title={Ensemble UCT Needs High Exploitation},
booktitle={Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,},
year={2016},
pages={370-376},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005711603700376},
isbn={978-989-758-172-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 8th International Conference on Agents and Artificial Intelligence - Volume 2: ICAART,
TI - Ensemble UCT Needs High Exploitation
SN - 978-989-758-172-4
AU - Mirsoleimani S.
AU - Plaat A.
AU - van den Herik J.
PY - 2016
SP - 370
EP - 376
DO - 10.5220/0005711603700376