What is a chess engine’s raison d’être?
I ask the question, only partially tongue in cheek, because of the early reaction to Part I of this review, which hit the web on Sunday. For those who have not read Part I, here’s a brief recap:
Fritz 17 includes the new Fat Fritz engine, a neural network with roots in the open-source Leela Chess Zero project. After describing the difference between traditional engines and neural nets, I then discussed the results of my early testing of Fat Fritz, and in particular, its performance in six engine-engine matches. I found that Fat Fritz was marginally weaker than the strongest Leela nets and development versions of Stockfish on my machine, something that appears to be borne out by further independent testing.
Almost immediately some Twitter followers began to ask (I assume in good faith!) why they should be “excited” about a commercial neural net engine when Leela and Stockfish, both free to download and use, are “better.” It’s a good question, but to answer it, we have to do a bit of back tracking and ask another one: what’s the point of using a chess engine at all?
Let’s begin by admitting the obvious. Almost every mature, well-written engine is more than strong enough to defeat any human alive. It’s gotten to the point that the authors of Komodo are, in conjunction with Chess.com, sponsoring handicap matches between Komodo and masters, even spotting FIDE Masters a full knight. The war is over, folks. Our silicon friends have won.
No one plays the engines anymore. They’re just too strong. But that immense, unfathomable strength is what makes them so important for human players to use and understand. Anyone with even a modern cellphone effectively has a Super-GM analyst and coach at their beck and call, willing and able to answer questions 24 hours a day.
In 2008 I referred to Fritz, then the strongest engine around, as something akin to the spell-check function in your word processor, a utility to catch blunders during analysis. In hindsight, this was a mistake. By that point Fritz, Hiarcs, Rybka, and Shredder were, from a practical perspective, the strongest chess playing entities on the planet. What they lacked in “understanding” they more than made up for in tactical ingenuity, and already they were critical parts of top-level preparation. Kasparov’s 2005 victory over Rustam Kasimdzhanov was due, in no small part, to a key opening idea that Deep Junior had found in 2003.
If the world’s elite were already “cyborgs” in 2008, their chess an amalgam of carbon- and silicon-bases intelligence, how indisputably truer this is today. The seeming omnipotence of the engines revolutionized opening preparation almost overnight, and players grew confident playing “ugly” but pragmatically valuable ideas.
One need only read Michiel Abeln’s new (and brilliant!) The Anand Files to see this paradigm shift unfold in real-time, from the mainline theoretical battles of the 2008 Kramnik match to the reactionary, “safety first” contests in the 2012 match with Gelfand. By 2010 the limits of Anand’s memory were already being tested, in no small part due to the “analysis paralysis” wrought by the computers:
The team thought it was not Vishy’s memory that had deteriorated since the Bonn match. Instead it was clear that the amount of knowledge he needed to remember was growing exponentially. More and more lines were being analysed with strong engines that didn’t exist just a few years earlier. … Anand also felt the stress of needing to know all his lines perfectly because of Topalov’s feared new supercomputer. (§5.1, “On the Rebound”)
Today Stockfish is ubiquitous at all levels of chess. It is embedded into popular web interfaces, its evaluation taken as gospel. Players have internalized silicon insights such that the machines set the standards for truth and falsity. If Stockfish says it’s a good move, it’s a good move, and nothing more need be said.
But after AlphaZero, that is beginning to change.
What makes neural net based engines like AlphaZero, Leela, and Fat Fritz so valuable is that they bring a set of new ideas into chess, expanding our notion of the possible once more. Jokes about “AlphaZero moves” abound in post-game interviews and game annotations, and implicit in them is the recognition that a new source of information and inspiration has arrived.
Where Stockfish returns 0.00 evaluations, neural nets tend to consider who has initiative, who has more ways to lose, and they factor that into their output. The world’s top players are already scrambing to mine NNs for their insights, and this is trickling down to less augustly rated players like me.
Let’s be clear. If all you want is blunder-checking, there’s no need to go beyond Stockfish. But if you want to build understanding, if you want to generate your own ideas in conjunction with the engines, you need to use all the tools at your disposal, and you need to understand their strengths and weaknesses.
Alpha-beta engines like Stockfish are fantastic at calculation, defense, and (with tablebase input) endgames, but they lack elite positional sense and they can underestimate initiative and “compensation.” Neural networks like Leela and Fat Fritz have brilliant understanding, but sometimes miss concrete continuations, and they can meander in winning endgames. Used together, taking the best from each, there is the possibility to generate fascinating ideas and insight into positions of all types.
Like Stockfish, Leela is an open-source product, free to download and use. So why should someone pay $89.95 for Fritz 17 and Fat Fritz, especially if (as we saw in Part I) it is marginally weaker than both Stockfish and Leela? Bracketing everything else that comes in the Fritz 17 package – the new Ginkgo / Fritz 17 alpha-beta engine, the automated install of Leela, the updated GUI and GUI features, and the six months of Playchess.com membership – it seems to me that the question comes down to style and the generation of ideas.
An engine’s objective strength is important, but the subjective element – what kind of ideas it generates – is equally, if not more, essential to me. Does Fat Fritz have a different style than does Leela? Does it “see” different possibilities in a position, and can it help us, the end-users, generate insights that we might not have otherwise?
Subjective Style and Tests
After a few weeks using two iterations of Fat Fritz, I would argue that (a) it does have a somewhat different style than Leela, and (b) that serious analysts will find it worth the purchase price.
Fat Fritz is aggressive in a less alien way than an alpha-beta engine like Stockfish, valuing initiative and complexity. Its weaknesses are similar to those of Leela: it sometimes misses tactics, it can drift in endings, but its ‘understanding’ is very, very high. Compared to Leela, Fat Fritz feels stronger tactically (even if tests don’t entirely bear this out), more comfortable in messier positions, and perhaps less positionally oriented, but this is impressionistic. Both Fat Fritz and Leela are ridiculously strong, and while I can watch how the evals move in the engine box, it’s sometimes hard for me to judge what those evaluations mean.
Knowing my limitations, I asked a titled friend who has looked at a lot of the publicly released games for his opinion. He confirmed that he too saw a stylistic difference between Fat Fritz and Leela, mentioning a few specific points.
- Fat Fritz loves outposts.
- Fat Fritz is better at pawn races / play on both wings.
- Leela likes domination and space.
- Leela has a better sense of material.
Ultimately readers will want to take a look at the 1046 downloadable Fat Fritz games that accompany this review to make their own assessments. (Thanks to ChessBase and CEGT for allowing me to include their test games in the downloadable file.)
Some specific games stood out. Chess engines are not immune to blundering, especially at quick time controls, and both Stockfish and Fat Fritz played howlers that led to quick losses. These are games #160 and 550 in the downloadable pgn.
King’s Indian structures are notoriously hard for traditional alpha-beta engines to understand. Fat Fritz, perhaps because it was fed games with closed positions during training, appears to do well in them. Here are two games (#304, 834) I came across where Fat Fritz uses the standard …g4-g3 pawn break / sac to attack the White king.
Stockfish remains a vastly strong competitor, and in more than a few cases it simply outcalculated Fat Fritz. Check out this (#607) demolition job.
Readers are encouraged to point out and discuss other interesting games and positions in the comments to this post.
What about specific analytical examples? I mention one in the December issue of Chess Life, where Fat Fritz comes up with a new idea in Fischer-Geller (Monaco, 1967). Here are a few more more esoteric positions I tested using:
- Fat Fritz, running on a RTX 2060.
- Leela, running the TCEC network T40B.4-160 remotely on a RTX 2080.
- the then-current development version of Stockfish (SF dev), running on four cores of a Ryzen 7 1700 at 3.7Ghz.
All six positions (with accompanying analysis) are provided in a pgn file at the end of this article, along with a few bonus positions / engine analyses that were cut for space.
Here Gusev famously sacrifices his queen with 24. Qxe5!!, ending up in a position where White has rook and two bishops for two rooks and the queen, but is absolutely dominating.
SF dev does not see the correct move in any reasonable timeframe – the last I saw, it may get there after about 70 ply! – but Fat Fritz wanted to sac the queen in all of 11 seconds. Leela (running the TCEC network T40B.4-160 on a RTX 2080) took a bit longer, but after 44 seconds, it came around.
One of the most heavily analyzed positions of recent years comes from the 2013 game between Svidler and Grischuk at the London Candidates. Grischuk famously sacrificed a piece after 12. f4 Nxc4 and the board exploded in complications.
All three engines wanted to avoid Grischuk’s idea with 12. Bg5, echoing Kotronias’ analysis in his book for Quality Chess. After 12. f4, they unanimously agreed with Grischuk and wanted to pitch the piece.
3. Minski and Afek, Benko 90 JT, 2019
One of the relative weaknesses of NN engines is that they are not as precise in deep calculations, especially in the endgame. Studies are an interesting place to test this, and here, in a study by Martin Minski and Yochanan Afek, they miss the mark. SF dev sees the correct idea (1. a7!) almost immediately, but both Fat Fritz and Leela get stuck on 1. Bd4+ Kd5 2. Rd1 Ra2 3. Bg1+ Ke4 4. a7 Nf5, when there is no win. The solution is provided in the pgn file of examples at the end of this article.
4. Van Breukelen, Schakend Nederland, 1990
Just for fun, here’s a study that no engine (that I know of) can answer from the initial position. Leela is the first to see the correct answer after being fed 1. Nf6+ Bg7, while SF dev and Fat Fritz only understand what’s going on after seeing 3… Kxh5.
What’s interesting is that after analyzing Black’s third move and going back to the initial position, Stockfish ‘gets’ it, while the neural net engines ‘forget’ what they have just analyzed. It appears that Fat Fritz and Leela cannot access their stored hash tables when moving backwards in a game, which may hamper them in analysis mode.
An aside: another difference, this time in favor of the NN engines, is that running Fat Fritz or Leela in multi-PV mode (showing two more lines in the engine window) does not have any cost, whereas Stockfish is weakened slightly in multi-PV mode.
Check out the pgn file at the end of this article for the complete solution.
Engines are forcing us to revisit and revise much of what we know about chess. Sometimes they refute previous analysis, but sometimes they simply remind us that what is possible on a chessboard is often much broader a category than we have imagined.
Mark Dvoretsky writes in School of Chess Excellence 3: Strategic Play of this position, taken from a game he played in a 1964 simultaneous exhibition against Mikhail Botvinnik. He writes:
“After playing the opening excellently, White then acted uncertainly and ended up in an inferior position. The move in the game (23. … f6) seemed to me to be completely natural – it attacks the enemy centre, and includes the inactive dark-squared bishop in the play.” (84)
Vladimir Simagin was shown the game and asked the young Dvoretsky, “Why weaken the position of your king and give White counterplay? Surely you can manage without this move? … Look at the ‘holes’ in the opponent’s position on the light squares. Your knight is dreaming of reaching there.” (ibid.) And only then did Dvoretsky consider 23. … a5!! (his exclams), which will drop the knight on b4 and reroute it to c2, d3, or d5.
I was interested to see how different types of silicon intelligence would handle this position. All three engines think that there are multiple decent moves here for Black, including 1. … a5 and 1. … Qd7, with scores that are nearly identical. 1. … f6 is also seen as entirely viable, with all engines scoring it around 0.00. 1. … a5 may be the best ‘human’ move, but one of the great lessons of our ‘metal friends’ is the expansion of moves that humans have to consider.
In the final analysis, I think Fat Fritz is an important new tool for players and analysts, a terribly strong network / engine with a novel and interesting playing style.
Chess players have grown spoiled by the availability of Leela and Stockfish, both open-source and free to use, but Fat Fritz represents good value for its price, especially when considered as part of the Fritz 17 package. As mentioned in Part I of this review, Fritz 17 includes Fat Fritz, the traditional Fritz 17 alpha-beta engine (Ginkgo, formerly private), the Fritz 17 GUI or graphical interface, and a six month Premium membership at Playchess.com and Chessbase.com. It also automatically installs Leela, which may be difficult for non tech-saavy users.
While there are useful improvements to the GUI – the addition of an opening repertoire training function feels like a direct response to Chessable’s newfound popularity – it’s really Fat Fritz that “sells” Fritz 17. And it’s on that basis that purchasing decisions will undoubtedly be made.
Could you get by with Leela and Stockfish, or just Stockfish, or an old version of Fritz 13 for that matter? Of course. But neural networks are the next big thing in computer chess, and with its impressive strength and interesting style, Fat Fritz is at the forefront of where chess is heading. Analysts, enthusiasts, and improvers will want to have it as part of their toolbox.
Leela UCI options (applicable to Fat Fritz as well)