After watching Leela chess zero evaluate boards, I'm realizing that #AI has a lot to teach us, if only we'd be humble enough to listen.

So I'm watching this game play out between Leela Chess Zero (think of an open source Alpha Zero) and Xiphos 0.65... a very strong, deep, conventional chess algorithm.

All of a sudden, Leela goes from slightly advantaged to this huge positional advantage.

It runs for a few minutes, then the huge advantage evaporates. Okay, I'm running on my work PC, a Microsoft Surface, and it's GPU is... okay, but not amazing. 778 billion multiplications/second, that sort of thing.

I'm also playing the chess in a weird configuration, where I give each side 6 seconds to think every turn. Leela only uses around 4 seconds of the time, so maybe it wasn't strong enough to convert the advantage that it had into a win.

Anyway, so I'm thinking, hey, what happened here?
I stop the tournament after the current game so I can see what is happening, investigate further.

Something you should know about Leela Chess Zero is that like its cousin Alpha Zero, it learns by self-play. It sits there, grinding the heck out of chess games until it can destroy itself, and trains itself how to play chess.

Watching it play, it's like this alien species that discovered chess on its own, doing a trillion moves until it's figured out the best play... but not in human way, the cultural way, learning from generations of grandmasters. In a deeply other way.

So I unpack the game (modern chess engines let you back up moves, play out configurations, evaluate principal variations, and annotate and comment games easily), and find out that it started to hugely diverge at this position (I call it three pawns).

(Sorry about the auto-alignment of images, it looks like Twitter is doing its "focus on white" thing again.)

Here Leela is white, and Xiphos is black.
Leela evaluates this position as a slight advantage, the equivalent of +2.3 pawns ahead for white.
Xiphos passes up the pawn trade, playing c6-c5, and rates its position worse, at -3.9 pawns behind for black.

Now Leela basically starts to jump up and down and wave its hands in the air. Leela plays d5-d6, and evaluates its position at the equivalent of +7.0 pawns ahead for white.
(Leela doesn't use pawns internally, just probability of victory, but there's an conversion for humans.)