Conclusion

-

AlphaZero is a pivotal use of neural networks and Monte Carlo tree search to create computers that become subject-matter experts without input from human experts.  The amount of time or computing power to train these networks to the point of being an expert is enormous, but the results are beyond promising.  The following video provides a surface-level recap of AlphaZero and how it is used to intelligently play complex deterministic games.


Again, to reiterate, our results shows that the AI trained to play Pente improved drastically in the first few iterations of training, and then the performance begins to slowly degrade over time.



Figure 1: Elo Rating vs Training Iteration

But why should anyone care about the results of this project or the results of AlphaZero?  As the video summarizes, we are obligated to ask "why is this worth doing?" when the final version of a model like this requires a tremendous amount of computing power to create.  

This question was answered two months ago when DeepMind published an article describing how they used essentially the same combination of algorithms to discover a new, much more efficient method of matrix multiplication than the previous best method known.  In other words, methods like AlphaZero are having a big impact beyond just the rather eccentric use case of playing board games.  Matrix multiplication isn't going anywhere, so the one-time use of a very large number of computing resources has created the opportunity to reduce energy usage and computing power needs for all future operations involving matrix operations.

Beyond the technical world, AlphaZero has potential use cases in all areas of graph exploration, whether to make better internet search engines, music recommendations, self-driving cars, directions to the grocery store, or any number of other uses.  All of these potential use cases make it a valuable algorithm to understand.

Comments

Post a Comment

Popular posts from this blog

Simulating Pente in Python (Data Generation and Labeling)

-
Image
  The goal here is not to try to solve a traditional supervised training neural network task, like image classification or house price regression, so the data collection methods must be different.  Instead of "collecting" data, a game simulation must be used to simulate the game logic behind Pente, and to enable running thousands of games of Pente in only a few minutes.  This has several implications for data cleaning and preparation... mainly, these steps are integrated into the data generation step so they don't need to be done separately later. For those interested, the entire code for the simulator can be interacted with here on Google Colab . Simulating the Game Pente is a fairly easy game to simulate because the win conditions and rules are both simple.  Legal moves are simply determined by whether a board position is occupied or not, and finding unoccupied positions on the board is not a computationally or conceptually difficult process.  Then, out o...
Read more

Introduction to Pente AI

-
Image
Approximating Optimal Solutions to Massively Complex, Deterministic Games What do Google Maps, many board games, and social media websites have in common?  These are a small subset of things that have parts that can be modeled as "graphs".  For anyone who is not familiar with computer science or discrete math lingo, the word " graph " here does not refer to a visualization or plot, but instead it refers to a collection of connected data points, where each data point is called a "node" and each connection between nodes is called an "edge". For example, with Google Maps, nodes may represent road intersections and edges may represent the streets or highways connecting those intersections.  With a board game like Chess, though, the nodes may represent game states (i.e. a snapshot of where all pieces are at or before a player takes a turn) and the edges may be the moves taken to get from one game state to another.  With social media websites, the node...
Read more

Training a Neural Network to Estimate Win Probability

-
Image
 Data Prep Visualization Visualizing the entire dataset used for a training iteration is particularly difficult because it is a very large graph.  So instead of visualizing what an actual train/test split looks like, here's an example on a small example graph in figure 1. Figure 1: Example Illustration of Train/Test Split The goal of this figure is to demonstrate that the test dataset is built by randomly selecting about 10% of the nodes from the graph created using MCTS, and then the remaining data is the train dataset. Again, as a reminder, the data contained in each node is the following: Figure 2: Reminder of how our data is stored in nodes An example graph with 10,000+ nodes for reference can be found  here , or you can generate your own dataset using the code here . The large number of features in the dataset (upwards of 700) and the number of nodes in even just a small graph (10,000+) make it particularly difficult to visualize the dataset well.  Figure 1 and ...
Read more