Chess Transformer — Neural Network That Learns To Play Chess

Chess Transformer is a language model trained to play chess. The neural network predicts the next move based on the history of the moves in the game. The Transformer model was trained on 2.8 million chess games in Portable Game Notation. The developers have published a Colab laptop in which you can play a game of chess with a model.

Image for post
Image for post

Training details

The GPT-2 with 774 million parameters was used as the transformer architecture. GPT-2 is a state-of-the-art generative model for natural language processing tasks. The model was developed by researchers from OpenAI.

The model was retrained in 30 thousand steps. The trained model correctly filters out invalid moves and demonstrates defense strategies such as Slav Exchange.

Common Move Frequency Statistics of diverse example
 moves from generative model.
Common Move Frequency Statistics of diverse example
 moves from generative model.
Image for post
Image for post

Chess Transformer is an example of how transformer models, initially adapted to work with NLP problems, can solve more general strategic modeling problems.

Interested in Deep Learning?

If you found this article helpful, click the💚 or 👏 button below or share the article on Facebook so your friends can benefit from it too.

Written by

Bioinformatician at Oncobox Inc. (@oncobox). Research Associate at Moscow Institute of Physics and Technology (@mipt_eng).

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store