Red neuronal enfocada en el aprendizaje supervisado del ajedrez

Enfocado en el reciente trabajo teórico de Google DeepMind (AlphaGo), he desarrollado e implementado una versión en TensorFlow de dicha arquitectura, pero realizando modificaciones para enfocar el diseño en el juego del ajedrez en lugar del Go (el paper original es: https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf).

El modelo ideado originalmente por DeepMind fue divulgado con detalle en este artículo: https://deepmind.com/research/alphago/, y mi implementation del mismo (con las modificaciones necesarias para enfocarlo todo en el juego del ajedrez), se encuentra en el siguiente repositorio de mi cuenta personal en GitHub: https://github.com/Zeta36/Policy-chess. Se trata como podéis ver, de un desarrollo Python usando el framework de Google, TensorFlow.

Os dejo a continuación toda la información técnica sobre mi trabajo, aunque siento no tener tiempo para traducirlo y simplemente copiaré a continuación la introducción que hice del mismo en inglés (prometo más adelante escribir una entrada dedicada a divulgar de manera más sencilla el potencial de esta red neuronal, y el indicio que puede contener sobre el modo en que nuestro propio cerebro biológico realiza ciertas tareas relacionadas con la generalización de conceptos, en este caso; aprender a clasificar con cierta habilidad y de manera "intuitiva" cual es una buena jugada de ajedrez de una mala):

A Policy Network in Tensorflow to classify chess moves

This work is inspired in the SL policy network used by Google DeepMind in the program AlphaGo (https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf). The network models the probability for every legal chess move given a chess board based only in the raw state of the game. In this sense, the input s to the policy network is a simple representation of the board state using a tensor (batc_sizex8x8x8) with information of the chess board piece state, the number of the movement in the game, the current player, etc. The SL policy network Pσ(a|s) alternates between convolutional layers with weights σ, and rectifier nonlinearities. A final softmax layer outputs a probability distribution over all legal moves a (labels). The policy network is trained on randomly sampled state-action pairs (s, a), using stochastic gradient ascent to maximize the likelihood of the human move a selected in state.

Preparing the Data sets

We train the 3-layer policy network using any set of chess games stored as PGN files. To prepare the training and the validation data set, we just need to download many PGN file (more data means more accuracy after the training) and put them in the datasets folder (there is in the repository some pgn examples to use).

After that, we run in the console:

python pgn-to-txt.py

In this way, the PGN files will be reformated in the proper way, and chuncked in a tuple of (board state, human move). We the pgn-to-txt.py script finish, go into the datasets folder and copy almost all the "*.txt" files generated into a new folders called "data_train", and some text files into another folder called "data_validation".

Finally, you have to run

python pgn-to-label.py

And we will get the labels for the SL. This labels will be generated and saved in a labels.txt file inside the "labels" folder.

Training

Training is a easy step. Just run:

python train.py

You can adjust before if you wish some hyperparameters inside this python script.

Playing

Once the model is trained (and the loss has converged), you can play a chess game against the SL policy network. Just type:

python play.py

The machine moves will be generate by the policy network, and the human moves in the game will be asked to you to be type in the keyboard. In order to move, you have to know the san Algebraic notation(https://en.wikipedia.org/wiki/Algebraic_notation_(chess)).

The game board is printed in ASCII, but you can use any online chess board configuration (like this http://www.apronus.com/chess/wbeditor.php) to mimic the movements so you can see clearly the game.

Requirements

TensorFlow needs to be installed before running the training script. TensorFlow 0.10 and the current master version are supported.

In addition, python-chess must be installed for reading and writing PGN files, and for the play.py script to work.

Results

After some thousands of training steps, the model is able to generalize and play a reasonable chess game based only in the prediction of the human movements in the training process.