Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self-play
Por um escritor misterioso
Last updated 23 maio 2024
Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.
A general reinforcement learning algorithm that masters chess
Train on Small, Play the Large: Scaling Up Board Games with
The Data Problem III: Machine Learning Without Data - Synthesis AI
Electronics, Free Full-Text
Policy or Value ? Loss Function and Playing Strength in AlphaZero
Adaptive Warm-Start MCTS in AlphaZero-Like Deep Reinforcement
Policy and value heads are from AlphaGo Zero, not Alpha Zero
Value targets in off-policy AlphaZero: a new greedy backup
Simple Alpha Zero
Recomendado para você
-
Deepmind's AlphaZero Plays Chess23 maio 2024
-
AlphaGo Zero Explained In One Diagram, by David Foster, Applied Data Science23 maio 2024
-
R] Understanding AlphaZero Neural Network's SuperHuman Chess Ability (Summary of the Paper 'Acquisition of Chess Knowledge in AlphaZero') : r/MachineLearning23 maio 2024
-
The Data Problem III: Machine Learning Without Data - Synthesis AI23 maio 2024
-
Simple Alpha Zero23 maio 2024
-
Leela Zero( A Neural Network engine similar to Alpha Zero) - Chess Forums - Page 1523 maio 2024
-
How DeepMind's AlphaGo Became the World's Top Go Player, by Andre Ye23 maio 2024
-
AlphaZero: DeepMind's New Chess AI23 maio 2024
-
Free Course: DeepMind's AlphaGo Zero and AlphaZero, RL paper explained from Aleksa Gordić - The AI Epiphany23 maio 2024
-
Oren Neumann on X: Do #RL models have scaling laws like LLMs? #AlphaZero does, and the laws imply SotA models were too small for their compute budgets. Check out our new paper23 maio 2024
você pode gostar
-
Bota Caterpillar Feminina - Roupas - Porto DAntas, Aracaju 125654667823 maio 2024
-
Quebra Cabeça Infantil de Madeira MDF Animais da Floresta23 maio 2024
-
FNAF Plush, Nightmare Bonnie, Puppet, FNAF Plush, Sly Plush - Plush Toys - FNAF, Nightmare Plush, All Character Plush Gifts (Twisted Freddy) : Toys & Games23 maio 2024
-
PLS DONATE 💸 [UPDATE] - Roblox23 maio 2024
-
Slideshow: New menu items from IHOP, Cracker Barrel, Cicis23 maio 2024
-
Nathan SpeedDraw Plus Insulated … curated on LTK23 maio 2024
-
Hajduk Split fined nearly £50,000 over trouble and racist chants23 maio 2024
-
it girl nail color 💅🏻❤️23 maio 2024
-
Vampire Survivors XBOX ONE / Series X23 maio 2024
-
PT-BR] Zoro - Figuarts Zero Battle Ver. (Rengoku Onigiri) - One Piece - Unboxing (Aliexpress)23 maio 2024