Analysis of Hybrid Deep Learning Agent for Gomoku

In recent years, machine learning methods and artificial intelligence have developed rapidly in many areas, and in particular, games. For example, DeepMind’s AlphaGo Zero was able to beat the world champion of Go using a combination of Monte Carlo Tree Search and self-training neural networks to teach itself how to play. Gomoku (historically played on a Go board with the same stones), has the objective of connecting 5 stones in a row where black (always first) and white alternate placing stones on the board. Many rulesets have been made to try and minimize the advantage that black has by going first which has been proved already. This project compared different machine learning methods on the game of Gomoku starting with a Minimax agent and building up to a Deep learning agent on a pre-trained data set. The main algorithm behind the deep learning agent was based on MCTS. At first, the pure MCTS player was found to be much less successful at the game than the Minimax agent which almost immediately showed promising results as it won nearly every game it played at a depth of only 2. The MCTS player was then trained on an existing data set and was able to train for two days. Following this, significantly better performance was recorded both as player 1 and player 2 against other baseline agents. DeepMind’s Alpha Go algorithm clearly shows that it works and with more advanced computational power, creating an agent to play from scratch like DeepMind’s Alpha Zero is an improvement to explore in the future.