Using Deep Q-Learning to Play Two-Player Yahtzee

Yahtzee, a dice game based on poker, can be played either by yourself or with multiple players. Today’s computers can handily solve the single-player version of Yahtzee, but the multiplayer version of the game is too complicated to be solved—optimal play must be approximated through other means. This thesis explores the development and performance of a Deep Q-Learning Network (DQN) agent designed to learn and play the game of two-player Yahtzee. The objective is to create an AI agent that not only understands the basic rules of Yahtzee but also learns strategy and improves as much as possible. The DQN agent is trained through reinforcement learning, where it learns optimal strategies by playing numerous games against various opponents, including two deterministic rule-based agents and a randomized agent. The deterministic rule-based agents are designed to play the optimal solitaire strategy and a greedy solitaire strategy, which are derived from the findings of previous literature and rebuilt in Python in this thesis. Nine different variants of the DQN agent are trained and evaluated against the three aforementioned opponents. The variants differ in their reward functions, training opponents, and training length. Ultimately, the best of the nine variants was not able to match the performance of the optimal solitaire agent, scoring an average of 159 points against the optimal agent with a winrate of 6%, but it did quickly learn the rules of Yahtzee and make significant improvements over the random agent. With easily customizable DQN and Yahtzee environment implementations, this thesis provides a solid foundation for future research into improving the DQN agent’s performance to match or exceed the optimal solitaire agent.