This option involves either applying reinforcement learning to a problem whose state space is very large and/or applying artificial neural networks to the learning of a static evaluation function. Starter code is available. It gives a state class definition, and a simple game master. In either case, you'll develop an agent that can play our version of Backgammon. Unlike Tic-Tac-Toe, Toro-Tile Straight, or Baroque Chess, the game of Backgammon involves the use of dice, and so the expectiminimax algorithm naturally plays an important role.

We are simplifying the rules of Backgammon for this project option, keeping most of its character, but making it easier to work with in a 2.5-week project: no "doubling" allowed; White always goes first; a player can pass on either using both dice or just one; a player does not have to use the higher-valued die when only one or the other can be played; having doubles on the dice (like a pair of sixes) does NOT accrue any special benefit such as getting to move 4 checkers; Bearing off a checker requires either an exact roll of a die, or if the checker is a farthest one from the goal, a die roll one greater.

To get started with this option, read the rules of Backgammon, and the following pages in our reading material on Reinforcement Learning (see the Readings page of our web site): Sutton and Barto, pp.347-351, covering the TD-Gammon development.

Another reference of possible interest is the blog post of Oct. 14, 2018 by Nikos Bozinis on his work related to analyzing TD-Gammon.

gameMaster.py in starter code updated on Nov. 27 with fixes for (a) disallowing moves from points when a player has a checker on the bar, and (b) handling hits from the bar.