Loading...

Follow Chessdom | Chess, chess news, live chess games on Feedspot

Continue with Google
Continue with Facebook
or

Valid

Season 16 of the Top Chess Engine Championship starts this Sunday July 14th at 15:00 CEST. A total of 18 engines will battle for six promotion spots in the newly formed Qualification Division. Among them are three Neural Networks – ScorpioNN, ChessFighterNN, and Stoofvlees. Their main competitors are the seasoned engines Rodent, Wasp, chess22k, and Winter. However, as most engines come with solid updates any surprise can be expected.

Format of the Qualification Division

TCEC Qualification Division is entirely invitational field. Initially set to be 16 engines, it was expanded to 18 due to the entrance of the brand new Asymptote engine (suggested by Graham Banks from CCRL) and a promo engine.

The engines will play a 1x DRR with time control 30+5. The top six will qualify for League 2 of TCEC, you can see the format of Season 16 here

Promo engine in the field

Many engines are now hopping on the Neural Network bandwagon, making full use of GPUs. Yet, the champion Lc0 has released a strong version playing on CPU only! It is not a unique engine by the TCEC criteria, but it certainly is exciting to see it perform against real competitors of 3000 ELO+ strength. Thus, Lc0 CPU enters TCEC as a promo engine in the field. It will play all games in the division, but at the end of the division all results of the engine will nullified.

Participants list

1. Bagatur 1.7b
2. The Baron 3.44.1
3. Cheese 2.1
4. chess22k 1.12
5. ChessFighterNN v2.1-n8x128c_7411
6. Igel 1.8.0
7. Jumbo 0.6.117
8. Marvin 3.4.0
9. Minic 0.76
10. ScorpioNN v2.9.0-n_maddex_INT8
11. Stoofvlees II a10
12. Topple 0.7.2_dev
13. Tucano 7.07
14. Wasp 3.68
15. Winter 0.6
16. Asymptote 0.6.2
17. Rodent III
18. Lc0 CPU

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

TCEC Season 16 is going to start mid July with the participation of the strongest computer chess software. For the first time ever a Neural Network – Leela Chess Zero aka Lc0 – will defend both a league title and a Cup title.

TCEC Season 16 will feature a new format. It will keep the acclaimed Premier Division and Superfinal as in previous seasons, but will give more chances to starting engines to climb up the rankings. For this reason three leagues of 16 engines will be created. The event will start with a Qualification League where all newcomers will be featured, including new AB engines and Neural Networks. The top 6 of the Qualification League will promote to League 2 (also 6 will relegate), and the top 4 of League 2 will promote to League 1 (also 4 will relegate). Finally, the best placed 2 engines from League 1 will qualify for the Premier Division (also 2 will relegate).

The change of structure for TCEC is necessary due to the present rapid development of computer chess. With the new format more engines will have a chance to climb up the ranks and have a shot to enter in the elite.

Here is the list of participants for the Premier Division and the newly formed leagues. The seeding of the engines is based entirely on Season 15.
Superfinal (100 games with time control 120′+10″ – using a new book by Jeroen Noomen)
1. Premier Division winner
2. Premier Division runner up
Premier Division (3x DRR with time control 90′+5″ – using a new 8 move book by Cato aka Nelson Hernandez)
1. Lc0 – current TCEC champion
2. Stockfish
3. Komodo
4. AllieStein
5. Houdini
6. Komodo MCTS
7. League 1 winner
8. League 1 runner up
League 1 (1x DRR with time control 45+5 - using a 6 move book, randomized)
1. Ethereal
2. Fire
3. Xiphos
4. Laser
5. Andscacs
6. Fizbo
7. Jonny
8. Chiron
9. Ginkgo
10. ChessBrainVB
11. Booot
12. Rofchade
13. League 2 qualification
14. League 2 qualification
15. League 2 qualification
16. League 2 qualification
League 2 (1x DRR with time control 30+5 – using a 4 move book, randomized)
1. Fritz
2. Nirvana
3. Arasan
4. Texel
5. Vajolet
6. Gull
7. Pedone
8. Nemorino
9. Rubichess
10. Pirarucu
11. Qualification
12. Qualification
13. Qualification
14. Qualification
15. Qualification
16. Qualification
Qualification (1x DRR with time control 30+5 – using a 2 move book, randomized)
Up to 16 engines playing for 6 qualification positions. This is a mix of engines from Div 4 and new entries. The complete list of participants will be announced soon.
Stay tuned for updates
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 


Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA

Corresponding author: g.haworth@reading.ac.uk

This is the latest in our series of analytical articles on past TCEC events. The main text can be read below on this webpage, and at the bottom you will find a link to the full layouted article in pdf format, including the important tables, graphs and images.

TCEC is very grateful to the authors for their kind permission to publish these substantial and scholarly analyses of its events!

Introduction

TCEC Season 15 started on March 5th 2019 with a more liberal Division 4 featuring several engines in their first TCEC season. At the top end, interest would centre on whether the recent entries, ETHEREAL, KOMODO MCTS and LEELA CHESS ZERO would again improve their already impressive performances. Fig. 1 and Table 1 provide the logos and details on the enlarged field of 44 engines.

Fig. 1. Logos for the TCEC 15 engines (CPW, 2019) as in their original divisions.

There were a few nudges to the rules. In the event of network breaks, if both engines were in the 7-man and/or TCEC win (or draw) zone, the game was adjudicated as a win (or draw). Otherwise, TCEC resumed games with extra initialisation time rather than restart them.

The common platform for TCEC15, as for TCEC14, consisted of two computers. One was the established, formidable 44-core server of TCEC11-14 (Intel, 2017) with 64GiB of DDR4 ECC RAM and a Crucial CT250M500 240 GB SSD for the EGTs. The ‘GPU server’, upgraded to a Quad Core i5 3570k with 32GiB DDR3 RAM, sported Nvidia (2018) GeForce RTX 2080 Ti and 2080 GPUs.

Table 1. The TCEC15 engines (CPW, 2019), details, authors and progress.

Initial interest centred on a third non-Shannon engine, this time the hybrid ALLIESTEIN, a cross between Adam Treat’s ALLIE and Mark Jordan’s LEELENSTEIN (Chessdom, 2019) and a pupil of supervised rather than ‘tabula rasa’ learning. Would this combine the best of ‘Shannon AB’ and neural-network approaches in a case of ‘1+1=3’ or would it be a fall between two stools? There were several reasons for believing that ALLIESTEIN would not tarry in Division 4 for too long.
HANNIBAL was recuperating after its crashes in TCEC14 but new to TCEC were some other ‘AB conventional’ engines – CHEESE, BAGATUR, IGEL, JUMBO, MARVIN, MINIC, MONOLITH, RUBICHESS and TOPPLE – a generous addition to the line-up. Given the wide range of estimated ELOs, there was likely to be a higher degree of carnage in Division 4 and so it proved. The division was in fact split into two halves with a play-off for the promotion places between the top two in each half.

Division 4a/b: each half of 1 DRR, 2 RRs, 18 rounds, 90 games @ 30′+5″/m

As for TCEC12-14, each engine played both White and Black from four-ply openings defined by the second author here. The results are as in Tables 2-3: ‘P%’ is the %-score and ‘ELO±’ the change to the engine’s nominal ELO based on its performance. Generic stats are in Tables 11 and 12. In part 4a, CHEESE’s win ‘against the otherwise unchallenged NEMORINO in game 11.2/52 was a clear outlier. RUBICHESS took second by virtue of its 2-0 result against WINTER.
In Act 2 game 12.5/60, SCORPIONN had a 7-man win (dtm = 22m) on move 80 but took 108 moves to get a 6-man EGT result. WASP disconnected against ALLIESTEIN in game 15.4/74, effectively a 1.5-point swing for second place as ‘crashes’ are the first tie-breaker. Unusually, games 29 and 87 ended in mate, and SCORPIO exhibited its ‘resigns’ move in games 5, 28 and 78.

The network crashed in g32, ALLIESTEIN–CHESS22K, on move 49w with CHESS22K’s evaluation at +5.3, a clear case for a continuation rather than a restart, surely the default response. ALLIESTEIN worked through to a 6-man RB-BP ‘mate in 18’ win with some difficulty. It would have been interesting to see it actually achieve this as neither engine was using the 6-man endgame tables (de Man, 2018).

Table 2. The TCEC15 Division 4a cross-table: one DRR phase, 18 rounds, 90 games.

Table 3. The TCEC15 Division 4b cross-table: one DRR phase, 18 rounds, 90 games.

The Division 4 play-off was marred and skewed by two PIRARUCU technical concessions. Throttled back by fiat from 43 threads to 16, this engine then underperformed and missed a likely second-place promotion. This should not obscure the fact that ALLIESTEIN showed new form to win, remarkably beating NEMORINO 4-0 and RUBICHESS 2½ 1½. The only blot on ALLIESTEIN’s escutcheon was its loss as Black to RUBICHESS in game 9.1/17: the power of two queens is not to be underestimated.

Table 4. The TCEC15 Division 4 play-off cross-table: two DRR phases, 12 rounds, 24 games.

Division 3: 2 DRR phases, 14 rounds, 112 games @ 30′+5″/m, 4-ply openings

With LEELA and KOMODOMCTS now in the higher divisions, competition for promotion was bound to be more open and keenly contested. Indeed, each engine lost at least two and won at least three games. There were plenty of wins below the diagonal of the eventual x-table, notably by PEDONE and VAJOLET at the expense of ROFCHADE, games 8/2.2 and 30/8.2.

Game 45, ALLIESTEIN-ARASAN, broke the TCEC shortest-win record in g45/12.1 with a mate in 20 moves (The shortest TCEC-draw was TEXEL-GULL in Season 10, Stage 1. After the mandated 1. b4 d5 2. Bb2 Qd6, play went 3. a3 a5 4. Nf3 axb4 5. Be5 Qb6 6. Bd4 Qd6 7. Be5 Qb6 8. Bd4 Qd6 9. Be5 {3x} ½-½.) (g45/12.1, AS-Ar: 1. e4 e5 2. Nf3 Nc6 {as mandated} 3. Nc3 Nf6 4. d4 exd4 5. Nxd4 Bb4 6. Nxc6 bxc6 7. Qd4? (7. Bd3) Qe7 8. f3 Bc5 9. Qd3 O-O 10. Bg5 Rb8 11. O-O-O d5 12. exd5 h6 13. Bh4 Ba6 14. Bxf6 Be3+ 15. Kb1?? (15. Rd2) Rxb2+ 16. Ka1 gxf6 17. Qxa6 Rfb8 18. Bd3?? (18. Qa5) Qb4 19. Ne2 Rxa2+ 20. Kxa2 Qb2# 0-1.), too sudden and savage for the TCEC referee to intervene. This was not the first evidence that there are still some bugs to be fixed. ALLIESTEIN was weak on moves 4 and 7 and blundered on 15 and 18 – the m4/m7 mistakes apparently connected to it failing to recognise castling options.

Despite this, ALLIESTEIN stayed in the promotion fight and its chances were conclusively enhanced with the 0-1 result of g100/50.2, ARASAN–ROFCHADE in which ARASAN got tangled up. The final results contained some surprises. No tiebreak rules were needed and ROFCHADE deservedly took first place after being unbeaten by the top half of the table including a unique 4-0 result against TEXEL. In con¬trast, ALLIESTEIN in second lost its matches against ARASAN and TEXEL but was unbeaten by the bottom four engines. The hope was that its known weaknesses had been sorted out. As the under-performing GULL was the reference engine at 3300 in the TCEC ELO scale, all other engines increased their TCEC ELO (GULL’s role is similar to that of the ‘IPK’ International Prototype Kilogram which defined the mass of one kilogram. If it hypothetically lost a gram, everything else would increase in mass by 0.1% even though unchanged. This parallel ceased to be valid on 20th May 2019 when the new definition of the Kilogram became operative (BIPM, 2019).), even NEMORINO which returned to Division 4. Wool (2019) picked out the round 19 game ARASAN–ALLIESTEIN which ARASAN would have ended with position-repetition but which in fact ended in a 6-man EGT-adjudicated draw.

Table 5. The TCEC15 Division 3 cross-table: two DRR phases, 28 rounds, 112 games.

Division 2: 2 DRR phases, 14 rounds, 112 games @ 30′+5″/m, 8-ply openings

A burnished ALLIESTEIN checked in for Division 2 with endgame table support and without the castling-related bug that manifested itself in Division 3: ROFCHADE was also refreshed. A point of interest was whether the newly promoted engines would hang on to their new status.
After RR1, ALLIESTEIN headed the standings with XIPHOS on +3 despite losing the drawn g21/5.1 to ROFCHADE after 109. Ke5?? instead of Ke7. ROFCHADE itself was on -2 and in the drop zone. At the halfway point, the leaders were XIPHOS (+7), ALLIESTEIN (+5) and CHESSBRAINVB (+2), despite the fact that ALLIESTEIN lost to ROFCHADE again. Can an engine play so badly that it plays well?

In the third round robin, there were few upsets. CHESSBRAINVB lost to FRITZ and ceded third place to BOOOT (+2) which had previously had difficulty winning. ALLIESTEIN (+8) and XIPHOS (+11) took their promotion prospects past the 97.6% and 99.9% confidence-levels respectively. NIRVANA was similarly headed for Division 3. In RR4, ROFCHADE escaped the drop in the final cliffhanger at the expense of FRITZ. XIPHOS and ALLIESTEIN gained promotion easily as BOOOT fell away badly. Interestingly, ALLIESTEIN performed better than XIPHOS against the better engines. Would this trend continue in Division 1?

Table 6. The TCEC15 Division 2 cross-table: two DRR phases, 28 rounds, 112 games.

Division 1: 2 DRR phases, 28 rounds, 112 games, tempo 60′+5″/m

Second author ‘Cato’ provided 12-ply openings for this division. The news was that the KOMODOMCTS crash problem was solved and so expectations were that it would promote easily. After seven rounds, KOMODOMCTS led ALLIESTEIN and FIZBO, ALLIESTEIN second courtesy of a default by CHIRON in game 25 when the latter failed to play a single move and crashed after 2.5 minutes: hash-table initiali¬sation may have been the problem. CHIRON has been on the cusp of Divisions 1 and P since it crashed three times in Season 12. KOMODOMCTS and FIZBO did not know that they were less likely to receive a similar gift but CHIRON was then cut back to 32 cores.

ALLIESTEIN (+4) reached half time strongly with 2.5/3 and a key win against FIZBO. KOMODOMCTS (+3) was second after being gifted a win by CHIRON in g46/12.2. With 12 moves to go to a 50-move draw, CHIRON lowered the drawbridge of its own fortress with 138. … f6?? and welcomed its enemy in with red carpet and heralds. As Karsten Müller confirmed, Ke8 was always available to guarantee the draw. LASER (+1) was gapped in third, sound, unbeaten but draw-centric. JONNY and CHIRON looked earmarked for Division 2 with just one win, literally between them.

Round Robin 3 ended with fireworks, nine of twelve games being decisive. ALLIESTEIN’s first win of RR3, a key one in g77/20.1 against XIPHOS, was followed by a network crash against CHIRON at move 70 in g81/21.1. This was posted as a ‘no fault’ loss but continued later and was drawn as expected. Meanwhile, KOMODOMCTS had moved confidently back to first place with three straight wins, the last to previously undefeated LASER. XIPHOS and LASER contested third place but were effectively two points behind ALLIESTEIN because of tiebreaks.
The last quarter started badly for ALLIESTEIN: its first genuine loss in g85/22.1 against ANDSCACS (Wool, 2019), a sharp, tactical Q-RR fight, not its forte. However, its win against KOMODOMCTS in g93/24.1 more than revived its promotion chances, giving it tie-breaking advantage against the other contenders. ALLIESTEIN and XIPHOS met in g105/27.1 for almost all the marbles and drew. As the dust settled and the GPUs cooled, positions 1-5 were decided clearly on points despite the closeness of this division. FIZBO escaped relegation only by virtue of the third ‘number of wins’ tiebreaker at the expense of JONNY which had the better SB score: TCEC follow FIDE’s tie-break priorities here. CHIRON, tailed off last at -10 and with one more albeit ultimately irrelevant crash to its name, now finds itself two divisions below its personal zenith. The two promotion spots went to KOMODOMCTS (+7) and newcomer ALLIESTEIN (+5) with fellow promotee XIPHOS (+3) a creditable third.

Table 7. The TCEC15 Division 1 cross-table: two DRR phases, 28 rounds, 112 games.

Division P, three DRR phases, 42 rounds, 168 games, tempo 90′+5″/m

The heavyweight Division P promised three weeks of the best computer chess to be found anywhere. It featured three non-Shannon (1950) MCTS engines: TCEC14 runner-up ‘LC0’ LEELA CHESS ZERO, returning KOMODOMCTS and serial promotee ALLIESTEIN. The majority of games, those between the tac¬tical minimaxers and the strategic Monegasques, were bound to be a clash of two styles and partic¬ularly interesting. After three rounds, STOCKFISH, LC0 and HOUDINI shared the three wins: the stable¬mates KOMODO and KOMODOMCTS were also unbeaten. In g24/6.4, ALLIESTEIN posted the first win for Black, an apparent draw at m74 but a promising R-BPPP endgame at m80 closed out only after another 90 moves of suspenseful exploration. Round Robin 1 featured just five precious wins and left four engines on +1 with only ETHEREAL ( 1) and FIRE ( 2) in deficit. The three ‘all-MCTS’ games were each drawn in over 112 moves.

After RR2 which sported seven wins, STOCKFISH (+4) and LC0 (+3) opened up on KOMODOMCTS and HOUDINI (+1). KOMODO, a three-time TCEC Champion redefined ‘solid’ on 14 draws from 14, only good enough for 6th: ETHEREAL ( 3) and FIRE (-5) were the principal donors and looked like joint tenants of the basement. ‘MCTS v Shannon’ with 9 of the 12 decisive games stood at +6=21-3. LC0 beat ALLIESTEIN, g39/10.3. All the red ink was below the x-table diagonal.

In round robin 3, STOCKFISH scored wins over the previously unbeaten KOMODO and HOUDINI, and over ETHEREAL and FIRE – a complete take-out of the last four. All others were 1 except ETHEREAL which lost touch with -3. ALLIESTEIN dropped a win against KOMODOMCTS in g19.2 with 36. Re1? instead of 36. Kh4! Both engines were against the clock in the drawn FIRE-ALLIESTEIN g80/20.4 until 189. … Kf5?? was preferred to the essential Kh6. This was the longest TCEC15 win to date and the first win by the underdog.

The fourth round robin resulted in a clear 2-4-2 formation: STOCKFISH and LEELA well out front, FIRE and ETHEREAL detached, and the remaining four on -1. We saw the shortest sequence of played moves in TCEC15: after the provided opening of g107/27.3, ETHEREAL demolished KOMODOMCTS in an ama¬zing 24 moves. The latter never seemed to be on terms with the situation. The MCTS-Shannon match moved on to +9=45-6 with three wins each in RR3-4.

With the Superfinal and demotions essentially if not formally decided, interest in the last third of Division P perhaps focused on the midfield contest. KOMODO rose while KOMODOMCTS fell, crashing against ALLIESTEIN and losing quickly again, this time to STOCKFISH. LEELA beat STOCKFISH to win their head-to-head: what did this say about the Superfinal to come?!

This is an appropriate moment to recommend Assaf Wool’s (2019) coverage of the TCEC games. For this Premier Division, Wool touched on all decisive games and some draws. He particularly focused on game 6 (HOUDINI–ALLIESTEIN, 1-0), g37 (STOCKFISH–KOMODOMCTS, 1-0), another clash of styles, g57 (KOMODO–STOCKFISH, 0-1), g86 (LEELA-KOMODOMCTS, 1-0), g127 (KOMODO-ALLIESTEIN, 1 0) and g161 (LEELA–STOCKFISH, 1-0).

And so the stage is set for a repeat of the TCEC14 Superfinal. LEELA comes through unbeaten with a pos¬itive score against all except HOUDINI. This is a remarkable achievement in a heavyweight division: the average game length of some 80 moves and median of 70 moves indicates that games were on the whole closely contested.

Table 8. The TCEC15 Premier Division cross-table: three DRR phases, 42 rounds, 168 games.

Table 9. The TCEC15 Premier Division figures: head-to-head and round-by-round scores.

The TCEC15 Superfinal match: 100 games, tempo 120″+10′/m

Again, after an intermission for the knockout TCEC Cup 3 (Haworth and Hernandez, 2019a), both STOCKFISH and LEELA CHESS ZERO came to the Superfinal in new versions. LEELA had won the last two TCEC Cup events, and a ‘bonus, no opening book’ match against STOCKFISH at the same Rapid tempo. Probably because of the very close TCEC14 result, a win by the smallest possible margin of one game, and the fact that LEELA was expected to have improved more than STOCKFISH, the six-times TCEC champion was no longer the favourite in the initial straw-poll.

Some evidence that LEELA was stronger in the endgame came to hand before the Superfinal started. The pause after TCEC Cup 3 included a replaying of the TCEC14 superfinal game 65 from the 7-man KNPPKBP position after 73. Kxf3. In the actual Superfinal, LEELA did not latch on to the key winning ideas quickly enough even with the help of the 6-man EGTs: the 50-move rule intervened. In ‘bonus mode’, the endgame was revisited with a later version of LEELA and this time, LEELA secured the win with less than half its previous inaccuracies. Fig. 2 shows the tracks of the two contests, depth in plies plotted against plies played. Optimal play is also shown for comparison.

Fig. 2. TCEC14 game 65 from KNPPKBP position 73b: (a) as played in the Sufi, (b) as replayed, and (c) optimal play.

This Superfinal was even for the first 15 games with one win to each side. STOCKFISH ominously opened its account with a win as Black. The expected close contest was in prospect but games 16-26 saw four wins by LEELA without reply. With hindsight, this was where most of the damage was done. Games 35-45 saw a flurry of decisive games with STOCKFISH pulling one back overall to improve the mood in its fanbase. However, LEELA struck with back-to-back wins in games 61-62: perhaps we will hear why the Trompowsky Attack, also associated with Bill Ruth and Karel Opočenský, seems not to suit STOCKFISH.

Thoughts of a comeback were rather dulled by eighteen draws but then STOCKFISH won again and in spectacular fashion with game 81. After 27. … Rae8, STOCKFISH saw a clear win with 26. h6 which LEELA had equally clearly missed. Sure enough, LEELA had to lose queen for knight in order to create the merest distraction. This was not the first time the new-style ANN engines had failed to find a sharp, tactical needle during a Monte-Carlo Tree-Search. The match was clearly not over. If LEELA could score four in short order, so could STOCKFISH. In fact, this did not happen. LEELA hit back immediately in game 82 and added insult to injury with two more wins in games 88 and 94.

‘The king is dead: long live the Queen’. Table 10 and Fig. 3 provide the core data. The final score of 53½ 46½ was more decisive than expected and it is easy to think that STOCKFISH did not play well.

This of course is not the case: LEELA just played better, some 27 ELO ahead in Implied Performance terms. Table 11 shows that games were a superhuman 89 moves long on average, not the 99 moves of their TCEC14 contest but the incremental time was less and the games were 20 shorter. Energy drinks please, not for the players but for the audience. Clearly, TCEC Superfinals are increasingly attracting top players to TCEC and we will hear more of these games. ‘Kingscrusher’ (2019b-u), known to his parents and now to others as CM Tryfon Gavriel, continues to provide richly informative video-commentaries. ‘Kingscrusher’’s (2019a) interview with Game Changer’s Matthew Sadler and Natasha Regan (2019) is also relevant to LEELA’s play. Wool (2019) and ‘GM TheChessPuzzler’ (2019) are also making extended and valuable contributions. Long may these continue.

Demis Hassabis’ (2019) tweet “Great to see learning systems come out on top. Huge congrats to the @LeelaChessZero team and community!” was typically enthusiastic and encouraging.

Table 10. The TCEC 15 Superfinal match of 100 games: the decisive games, Black wins underlined.

Fig. 3. The TCEC15 Superfinal: the incidence of decisive results and LEELA CHESS ZERO’s lead.

Table 11. Generic statistics for each phase of TCEC15: results, terminations and average game-length.

Summary

After fifteen seasons of TCEC, it is worth reflecting that league tables tend to emphasise the ranking of chess engines rather than their relative differences, and certainly rather than their absolute prowess. Also, given the nearly non-stop nature of TCEC events, it is easy to forget that hours of top-level chess are passing before us in all the divisions – and inevitably, without getting the attention and analysis that they deserve, despite the hints on the TCEC and Chessbomb (2019) sites and Sadler’s (2019) perspective. Any of the TCEC15 engines would give a Grandmaster a serious game and most would perhaps have to be handicapped by a Blitz or even Armageddon tempo. Even so, there is evidence here that if there is some ‘ceiling’ asymptote to quality of play, it is still some way off. The admirable Emil Vlasák (2019) has, for example, clearly demonstrated that LEELA would not be at all competitive in a Computer-solv¬ing Champion¬ship as it struggles to find study-like wins.

Congratulations to TCEC’s new champion, LEELA CHESS ZERO, and to all who have assisted in her conception and evolution. STOCKFISH, champion for TCEC seasons 6, 9 and 11-14, continues to domi¬nate the rest of the field and we can expect to see hostilities renewed. Perhaps LEELA’s vulnerability to tactical shots will be exposed further. ‘Kudos’ to all other participants and to the core TCEC team who make all this happen.

Will further engines of the ‘new genre’ join the fray, and will hybrid engines appear, incorporating the best of the ‘minimax’ and MCTS perspectives? Will TCEC be able to combine the strengths of their two servers in one platform in order to facilitate this? Will computer-based tools emerge to help us understand..

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

The Russian chess player Aleksandra Goryachkina won the Women Candidates Chess 2019 with two rounds to spare. The youngest participant of the tournament finished with 9.5 points out of 14 and secured the right to play the Women’s World Championship Match against the Women`s World Champion Ju Wenjun (China). The prize fund of the coming championship match is 500,000 Euro, which is 150% higher than in the previous match.

Anna Muzychuk (Ukraine) is second with 8 points. Katerina Lagno (Russia) and Tan Zhongyi (China) shared 3-4 places with 7 points.

Final standings:

1. Aleksandra Goryachkina (Russia) – 9.5, 2. Anna Muzychuk (Ukraine) – 8, 3-4. Kateryna Lagno (Russia), Тan Zhongyi (China) – 7, 5-6. Nana Dzagnidze (Georgia), Mariya Muzychuk (Ukraine) – 6.5, 7. Alexandra Kosteniuk (Russia) – 6, 8. Valentina Gunina (Russia) – 5.5.

The closing ceremony took place in the Nogai Hotel, the tournament venue, on June 18.

In the beginning of the ceremony, a special prize for the most beautiful game of the tournament, provided by the Russian Chess Federation and AB InBev Efes company, was awarded. The jury consisted of chairman Maxim Notkin, editor-in-chief of the 64-Chess Review, the tournament commentators GMs Sergey Shipov and Evgeny Miroshnichenko, and the Head of the Appeals Committee Jeroen van den Berg. A shortlist of four games was formed.

The beauty prize was awarded to Mariya Muzychuk for the game Muzychuk-Goryachkina, played in the last round, by Natalia Rostova, the Deputy Director of the local branch of the AB InBev Efes, and Maxim Notkin.

FIDE President Arkady Dvorkovich, RCF Executive Director Mark Glukhovsky, and Minister of sport of the Republic of Tatarstan Vladimir Leonov delivered speeches. The Chief Arbiter of the tournament, IA Hal Bond (Canada) announced the competition results. After his announcement, the players received their prizes from the honored guests.

Total prize fund of the FIDE Women`s Candidates is €200,000.

The organizers are FIDE, Russian Chess Federation, Government of the Republic of Tatarstan, Ministry of Sports of the Republic of Tatarstan, and city administration of Kazan.

The tournament is sponsored by PJSC PhosAgro and Russian Railways.

Official website: https://fwct2019.com/en/

Final table: https://fwct2019.com/crosstable

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA

[Corresponding author: g.haworth@reading.ac.uk]

This is the next article in our series of analytical articles on past TCEC events. The main text can be read below on this webpage; to view the full layouted article in pdf format, including the important tables, graphs and images CLICK HERE.

TCEC is very grateful to the authors for the authors’ kind permission to publish these substantial and scholarly analyses of its events!

Introduction

The second TCEC Cup (Haworth and Hernandez, 2019a) was won by ‘LC0’ LEELA CHESS ZERO beat-ing HOUDINI after the latter surprisingly took out STOCKFISH in their semi-final. The event, with its Rapid tempo of 30′+5″/move continued to be the favoured curtain-raiser before the current TCEC season’s Superfinal (Haworth and Hernandez, 2019b). TCEC Cup 3 began on April 29th 2019.

The following engines sent in updates for the cup: ALLIESTEIN, ARASAN, FIRE, GINKGO, KOMODO, KOMODOMCTS, LCZERO, MARVIN, NEMORINO, RODENT III, ROFCHADE, RUBICHESS, STOCKFISH, VAJOLET2, WASP, WINTER and XIPHOS. The settings of CHIRON and PIRARUCU were changed. So clearly, the international computer chess programme continues on its dynamic way (CPW, 2019). The engine logos are listed in Fig. 1.

The ‘standard pairing’ was again used, with seed s playing seed 26-r-s+1 in round r if the wins all go to the higher seed. Thus, seed s1 plays s32, s16, …, s2 if all survive long enough. The higher seed is listed first in Table 1. This time, the matches – eight games plus any necessary game-pair tiebreaks – were played out only until the result was decided.

The usual ‘TCEC opening’ team, the second author here and Jeroen Noomen, randomly chose from three books with some regard for frequency over the board. Greater variety of play ensued from round 1’s 8-ply openings and 12-ply openings thereafter up to and including the semi-finals. The final took openings of various lengths from JN’s TCEC Superfinal books for seasons 9-14.

As in previous TCEC Cup events, interest focused on actual performance ‘%P’ compared with expected performance ‘E%P’ implied by TCEC ELO difference ‘ELO Δ’. The accuracy of the TCEC ELOs, the upgrades to over half the field and the character of the random openings were to be the main influences.

1 Round 1

As expected, ‘LC0’ LEELA CHESS ZERO opened its campaign with a 5-0 clean sweep. ALLIESTEIN, HOUDINI and KOMODOMCTS repeated this feat. In the top half of the draw, the eventual winner did not concede a single game but this was to change. Marginal favourite GINKGO lost its first game, won its second and was taken to fourteen games before persevering against ROFCHADE. This was the only comeback and tiebreak of the first round.

The other match-winners to lose a game were CHIRON (to FRITZ) and ANDSCACS (to TEXEL) so ‘kudos’ to those engines. Best performers relative to expectations were ETHEREAL, ALLIESTEIN, FIRE and especially BOOOT which comprehensively eliminated CHESSBRAINVB, the seed above it. The draw between STOCKFISH and RODENT is also well worth a visit. The field was now exclusively TCEC15 Divisions P, 1 and 2 – the top 15 plus seed 17, BOOOT, almost as expected.

2 Round 2

BOOOT put up a valiant fight against LC0 in some long and memorable games, particularly game 1 (Kingscrusher, 2019a) but still only scored a half-point. FIRE edged a match win in game seven against immediate rival XIPHOS. STOCKFISH beat GINKGO, g25, the final KRPPKRP just beyond adjudication echoing Carlsen-Caruana WCC 2018, Rapid game 1. LASER scored an early win against ETHEREAL but lost with its last Black of eight. Fifteen draws followed before ETHEREAL nosed ahead to win the longest TCEC Cup match to date. LASER’s fans are entitled to be disappointed, having come so close.

3 The quarter-finals, semi-finals, third-place play-off and final

LEELA, by virtue of being top seed, gets the lowest seed left as long as results go with seeding. However, at this stage, no match is easy. LEELA duly overcame FIRE but it was only after a great fight, a credit to both sides. HOUDINI, like BOOOT in Round 1, overturned the seed immediately above it, in this case ALLIESTEIN.

In the lower half of the draw, ETHEREAL surprised the growing audience and STOCKFISH with a straight eight draws but then STOCKFISH reeled off two wins – the second of which it could arguably have lost after 28. … Rbd8. Enter the dragons, KOMODO and KOMODOMCTS, over-hungry after the delay and more than ready for a fight, see Fig. 2. After much effort, the upstart newcomer overturned the seeding even more than HOUDINI, its win in the last of the eight scheduled games allowing no response.

In the semi-final, the favourites were not challenged and came through, both being unbeaten so far. This left HOUDINI to face KOMODOMCTS in the play-off for third. This was won comfortably by HOUDINI which continues to impress at this level despite not being updated.

In the final, LEELA’s game 3 win was not decisive as STOCKFISH won convincingly in the last scheduled game. The tie-break then ensued and LEELA ironically took advantage of two rare but not unknown 7-man STOCKFISH errors (Aloril, 2019), here in positions 155b and 167w. LEELA progressed to the win without its usual hesitancy as it now had the use of the 6-man EGTs. Fig. 3 shows the defence, the value and depth-shedding errors and the progress to the win. In the return game, LEELA eroded STOCKFISH’s initial advantage and then attacked in a drawn position: only perpetual checks across 100 moves prevented STOCKFISH from being mated.

4 A summary

The early rounds went very much as predicted by the form book though several losers put up stronger resistance than expected. At the top level, few mistakes were made by the closely-matched engines so in the short matches anything could have happened. Nevertheless, LEELA confirmed that its win in TCEC Cup 2 was no fluke and it retained the title. Neural networks do finally seem to be coming through with genuine advances, at Deep Mind (Hassabis, 2019) and elsewhere, but troublingly it is not obvious why they work and when they go wrong. The engines created in the Shannon (1950) genre are at least valuable as benchmarks and have their reputations to defend. Congratulations to LEELA (Chessdom, 2019; Linscott, 2018) and to all participants for some top quality chess. We will see its equal but will we be equal to appreciating it? Helpfully, Kingscrusher (2019a/b) continues to reveal the context, themes and dynamics of the games with his rich commentaries. The e-version of this report (Haworth and Hernandez, 2019c) provides statistics beyond Table 6 and all games with some analysis including play-outs of all decisive games, some of which end more obviously than others. Semi-final LEELA–HOUDINI game 1, for example, is relatively complex in the field of TCEC-adjudicated wins.

REFERENCES
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

TCEC Cup is coming, with the top 32 engines in a knockout competition

The TCEC Cup is now an integral part of the seasons of the Top Chess Engine Championship. It brings a special twist to the regular pace of the season, gives a chance for lower rated engines to meet the best, and also provides opportunity for engines with new versions to show their best and climb up the ladder.

The first TCEC Cup was convincingly won by Stockfish. It proved superior in Season 13 and did not give a chance to its opponents at any moment during the cup competition as well. TCEC Cup 2, however, was a different story. Stockfish stumbled at the semi-final against the S10 champion version of Houdini. This pitted Houdini against the bearer of the NN revolution Lc0, and the latter won its first major title in computer chess.

Replay TCEC Cup 2 final

TCEC Cup 3 starts this Monday April 29 at 19:00 UTC. The cup holder Lc0 is going to defend its title in a race with the top 32 engines of the ongoing season. Its first match will be against a newcomer to the TCEC race – Marvin by Martin Danielsson (see Marvin’s rating at CCRL).

The main challenger of of Lc0 will be Stockfish. It starts at the other end of the brackets and the two can meet only at the final. The first match of Stockfish is against the 31st seeded – Rodent III by Pawel Koziol (see Rodent’s rating at CCRL). Rodent comes with last minute improvements and although having little chance against the many times TCEC champion, it will be a nice test altogether.

Despite an easy pairing in round 1, both Lc0 and Stockfish will have a difficult way to the final. Last year the surprise was Houdini. This year the pitfalls along the way are many. Komodo is playing with a new stronger version and is keeping up with the pace of the favorites. The new NN engine AllieStein is bringing hard time to the top engines, often putting decisive pressure on them and even defeating Houdini. Komodo MCTS is also among the dark horses – it just does not lose against its brother Komodo or against Houdini, while it drew 6/7 games against Lc0 in the Premier division. Xiphos, a stable Ginkgo, the updated Fire, or a new Laser can also bring havoc to the field.

TCEC Cup 3 pairings

The TCEC Cup 3 seedings are traditionally based on the ongoing season. As TCEC_Spectator explains, “TCEC Cup 1 pairings followed entirely the standings of the regular season. TCEC Cup 2, and subsequent cups, will have the winner of the previous Cup seeded as #1, while all the other participants will be seeded according to the regular season. The Premier Division will give the rest of the top 8 seeds of the competition. Seedings based on Divisions 1, 2 and 3 are straight forward. The 6 engines that did not promote are seeded in order of finish – Division 1 gives seeds 9 through 14, Division 2 gives seeds 15 through 20, Division 3 gives seeds 21 through 26. That leaves Division 4a and 4b. The teams finishing #3 and #4 in the 4 engine playoff for promotion to Division 3 make up seeds 27 and 28 respectively. The final 4 seeds are the 3rd and 4th place finishers (that did not make the playoff) in each of the two divisions 4a and 4b. The 3rd place teams are seeds 29 and 30 with the engine with higher points given the 29th seed. Likewise the 4th place teams are seeds 31 and 32 (finishing with equal points but Rodent III had more wins so was given seed 31).”

The full pairings will be completed once the Premier Division is completed – follow it live here. For engine ratings check out the computer chess engine rating list CCRL

1. Lc0 (title holder)

2. Stockfish

3. Div P #3

4. Div P #4

5. Div P #5

6. Div P #6

7. Ethereal

8. Fire

9. Xiphos

10. Laser

11. Andscacs

12. Fizbo

13. Jonny

14. Chiron

15. Ginkgo

16. ChessBrainVB

17. Booot

18. rofChade

19. Fritz

20. Nirvana

21. Arasan

22. Texel

23. Vajolet2

24. Gull

25. Pedone

26. Nemorino

27. pirarucu

28. RubiChess

29. Wasp

30. Winter

31. Rodent III

32. Marvin

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Written by Guy Haworth and Nelson Hernandez
Reading, UK and Maryland, USA

Corresponding author: g.haworth@reading.ac.uk

This is the latest in our series of analytical articles on past TCEC events. The main text can be read below on this webpage, and at the bottom you will find a link to the full layouted article in pdf format, including the important tables, graphs and images.

TCEC is very grateful to the authors for their kind permission to publish these substantial and scholarly analyses of its events!

Introduction

TCEC Season 14 started on November 12th 2018 and introduced a number of changes from TCEC 13 (Haworth and Hernandez, 2019b). An enlarged Division 4 featured twelve engines and seven newcomers to accommodate the increasing interest in computer chess and this competition in particular. The other divisions remained eight strong. The five divisions played two or more double round-robins (‘DRR’) each with promotions and relegations following. Tempi gradually lengthened from ‘Rapid’ to ‘Classical’, and the Premier division’s top two engines played a 100-game match to determine the Grand Champion.

The trio of STOCKFISH, KOMODO and HOUDINI have dominated the TCEC medals for several seasons and a key point of interest was whether others would reach the podium. LEELA CHESS ZERO and ETHEREAL were certainly expected to perform well in Division P, having shown remarkable improvement in the previous few months. KOMODO MCTS was a dark horse.

There were a few nudges to TCEC’s adjudication rules. Draw adjudication could be invoked after move 35 (rather than move 40) and the two engines had to both evaluate within ±0.08 (rather than ±0.05) for eight consecutive plies and with plycount≠0. While draw-adjudication requirements were relaxed, win-adjudication requirements were tightened. Engine evaluations had to be outside ±10 (rather than ±6.5) for ten consecutive plies (rather than eight); plycount was not a factor. This change was welcomed by those of us who wanted to see a clearer demonstration of superiority on the board: it will be interesting to see how long it prolongs the decisive games and what mysteries remain.

The common platform for TCEC14 consisted of two computers. One was the established, formidable 44-core server of TCEC11-13 (Intel, 2017) with 64GB of DDR4 ECC RAM and a Crucial CT250M500 240 GB SSD for the EGTs. The ‘GPU server’, a Quad Core i5 2600k, was sporting Nvidia (2019) GeForce RTX 2080 Ti and 2080 GPUs for those engines which could exploit them.

The engines

Season 13 competitors BOBCAT, DEUS X, HANNIBAL and SENPAI rested for this TCEC season. TCEC welcomed first appearances for engines DEMOLITO, KOMODO MCTS, PIRARUCU, ROFCHADE, SCHOONER, SCORPIONN and WINTER, see Fig. 1 and Table 1.

Division 4: 2 DRRs, 4 round robins, 264 games, 30′+10″/m

As for TCEC12/13, each engine played both White and Black from four-ply openings defined by the second author here. The results are as in Table 2: ‘P%’ is the %-score and ‘ELO±’ is the change to the engine’s nominal ELO based on its performance. Generic stats are in Tables 9 and 10.

Online interest naturally focused on the newcomers, especially KOMODO MCTS (Chessdom, 2019), a further innovation from the Lefler/Kaufman camp. The engines had a wide range of ability leading to only 34.1% of games being drawn: those given a default ‘TCEC-entry ELO’ of 2900 ranged across the field. WINTER was always headed for a demotion spot. SCORPIONN clearly was not ready for the contest and even though it disconnected eight times, it did not impact the ranking elsewhere. The bottom three missed TCEC Cup 2 (Haworth and Hernandez, 2019c). The three engines promoted were clearly ahead: KOMODO MCTS, ROFCHADE and NEMORINO.

Division 3: two DRR phases, 14 rounds, 112 games, tempo 30′+10″/m

Again, the eight engines involved played both sides of 14 prescribed four-ply openings. With GPU operating temperatures more stable, LCZERO was expected to do well after its performance in TCEC Cup 1 (Haworth and Hernandez, 2019a) and it did not disappoint, see Table 3. KOMODO MCTS also distanced the rest of the field and continued on up the divisions. Crashes remained a problem: this time, HANNIBAL incurred five. In game 26/7.2, ROFCHADE disconnected in a 7-man tablebase drawn position. In g93, NIRVANA retained the KBNPKRN draw for 101 moves but claimed the 50-move draw with 165. Bd6 – which loses to 165. … Nf7+ 166. K~ Nxd6. Do chess programs do irony?

Division 2: two DRR phases, 14 rounds, 112 games, tempo 30′+10″/m

Game 64/16.4, KOMODO MCTS – LEELA, ended in a rare stalemate on m172. Game 93/24.1, NIRVANA–LEELA, was drawn at position 115b but a mate for Black in 29 moves when the 50-move draw rule intervened. Demoted GULL beat BOOOT and BOOOT beat LEELA which otherwise moved smoothly away to win the division again, see Table 4. The silver medal went to KOMODO MCTS, courtesy of one less loss to LEELA than XIPHOS and one more win to the rest of the field.

Division 1: two DRR phases, 28 rounds, 112 games, tempo 60′+10″/m

The penultimate game 28.3/111 was the longest ever for TCEC Division 1 at 308 moves: ‘new wave’ LEELA versus ‘old guard, oldest brand’ FRITZ 16. The win is routine enough with rook and passed pawn against a half-sighted bishop but endgame solver FINALGEN (Romero, 2012) sees 20 moves before a clear win, a line that results in mate on move 337 at best (Haworth and Hernandez, 2019d).

GINKGO surprisingly crashed four times and was disqualified so the formal results are slightly different from those of Table 5 even if promotions/relegations are otherwise unaffected. FRITZ never saw a win in this company and also was demoted to division 2.

Division P, three DRR phases, 42 rounds, 168 games, tempo 90′+10″/m

The line-up for Division P had only a semi-familiar look. After the TCEC13 podium trio of STOCKFISH, KOMODO and HOUDINI, we had the other survivors FIRE, ETHEREAL and ANDSCACS. Interest however centered on the newcomers LEELA CHESS ZERO and KOMODO MCTS, both bringing MCTS search to the game. The contest was three DRRs rather than the four of TCEC13.

After the first round-robin, STOCKFISH had jumped out into the lead with four wins. After the first DRR, with colour-bias eliminated, STOCKFISH maintained a healthy lead and remained unbeaten, a feat shared with KOMODO and LEELA. Was the TCEC podium about to change? KOMODO MCTS had disconnected and lost twice against KOMODO in drawn positions. A third disconnection would be bad for both engines: disqualification for MCTS and elimination of Komodo’s crash-wins from the table.

Game 64 saw STOCKFISH beat KOMODO, opening the door for LEELA. In game 68 at the foot of the table, ANDSCACS beat ETHEREAL with Black. At the half-way point, LEELA was edging the contest for second place and remained unbeaten. The fourth round-robin saw LEELA consolidate its second place with four straight wins against the tail including one as Black against ETHEREAL. The competition for second place remained open as STOCKFISH finally ended LEELA’s unbeaten run in the last RR4 game, g28.4/112.

The fifth round-robin saw plenty of drama. LEELA lost as Black to both KOMODO and FIRE, the first having serious tie-break significance and the second being seriously unexpected. GPU fan-settings were thought to be a contributory factor but not enough to trigger replays. In game 33.1/129 v HOUDINI, KOMODO MCTS disconnected for a third time, was disqualified and relegated with its games discounted. Hopefully, Mark Lefler will sort out the technical problems for TCEC15. This restored LEELA to second place. With one round-robin to go, adjusted scores at the top were STOCKFISH well clear on 21, LEELA 16.5, KOMODO and HOUDINI 16. The second relegation spot was between ETHEREAL on 11.5 and ANDSCACS on 11.

Every win was now going to be a major event, especially as the last round of 28 games started with seven draws. KOMODO as White lost to STOCKFISH in g37.4/148. Both LEELA and KOMODO beat FIRE. In the penultimate game, KOMODO beat ANDSCACS: ETHEREAL breathed again, having narrowly survived without a single win in this division. In the last game, a cliffhanger, STOCKFISH searched the endgame tables a thousand times more than LEELA and thought it had a feasible advantage, but LEELA held out in KRPPKRP to draw on move 93.

The raw figures of Tables 6 and 7 need adjustment because KOMODO MCTS’ disqualification flipped the ranking at both ends of the table. In fact, STOCKFISH ultimately had 25 points, LEELA 20, KOMODO 19.5, ETHEREAL 14 and ANDSCACS 13.5. The ‘big three’ became the ‘big four’ but the Shannon-AB engine mould was cracked again: the still-improving LEELA had remarkably progressed from Division 3 all the way to the TCEC Superfinal.

As in TCEC13, a knockout event was interposed between this tournament and the Superfinal. Would the LEELA team roll out an improved network in preparation for the big finish? A hint came in a ‘bonus match’ between a more recent ‘LEELA 32585’ and ‘STOCKFISH 8’, the latter having only 12 threads and a 4M hash-table. This was an echo and ‘simulation’ of the ALPHAZERO–STOCKFISH match: LEELA won +24=71-5. We reported on TCEC Cup 2 separately (Haworth and Hernandez, 2019c).

The TCEC14 Superfinal match: 100 games, tempo 120′+15″/m

TCEC’s ELOs suggested a STOCKFISH win by eleven. However, both engines came to the board in new versions: the match was now STOCKFISH v190203 versus LEELA v20.2-32930. There was bound to be a clash of styles occasioned by the different modes of evaluation and use of hardware. This dynamic was eagerly anticipated with viewer numbers often topping 2000. Jeroen Noomen (2019) again created a suitable opening book, aiming as before for at least 20% decisive results. Assaf Wool returned from his ‘TCEC Cup break’ to comment on all the games. GMThechesspuzzler and Kingscrusher were active on Youtube (Wool (2019) picked out positions from games 7, 8, 10-11, 13, 16-17, 20-22, 25, 27, 29, 35, 41, 49, 53, 55, 58, 63, 65-66, 71, 75, 80, 85 and 87. Kingscrusher (2019) commentated on games 7, 10-11, 13, 16, 17, 53, 66 and 85. Games 2, 7-8, 13, 17 20, 29, 49, 65-66, 80, 85 and 100 were covered by GM Thechesspuzzler, 2019). Soren Riis provided the authors with detailed analysis of games 7-8, 20-22, 65-66 and 71 which we provide via our pgn file for reader convenience rather than here. GM Matthew Sadler (2019), having analysed the STOCKFISH–ALPHAZERO games (Sadler and Regan, 2019) has also contributed his own view of this Superfinal.

The play and the results did not disappoint. STOCKFISH opened its account with wins from games 7 and 10 but LEELA replied with wins from games 11 and 13. There were twelve wins in the first thirty games, a hit rate of exactly 40%, see Table 8 and Fig. 2. At this point, the score was 15-15, suggesting that this would be the closest TCEC Superfinal since Season 5 in 2013 even though LEELA had never led. The same situation appertained at 24-24 after a run of 19 draws (not a record: the TCEC8 KOMODO 9.3x – STOCKFISH 021115 Superfinal games 14-37 and 47-71 were all draws). At this point, LEELA dramatically jumped out front with wins in games g49 and g53. This lead held until game 80 which STOCKFISH won. Ultimately, it was the single 0-1 win in another sea of 19 draws that allowed STOCKFISH to retain the title. Each game was closely contested with average length being one ply short of 100 moves – and not just because LEELA was reluctant to visit the draw zone.

Of course, suitably equipped grandmasters could write a book about this entirely gripping match and this would be most welcome. Here, we can only pick out a few chessic highlights which perhaps complement the analyses of the commentators above.

The hints from the evaluations of STOCKFISH suggest that it welcomed LEELA’s 15. Bb2 (g07), 51. … Be3 (g08, a missed win), 34. Kf1?? (g21) and 31. … Qd6 (g22). In game 35, 29. Ke1 rather than h7 seemed to lose LEELA’s winning advantage. Game 58 was adjudicated with a rare ‘mate in one’ on the board: the camera cut away just before the blow was struck. Game 63: LEELA was happy to trade pawns for position as early as eleven moves into the play. STOCKFISH did not see a serious problem until six moves later. LEELA create a passed pawn despite being three pawns down and this led to a crushing 41-move win, the shortest of the match.

If there was a pivotal juncture in this Superfinal, it was games 65-66 – a crucial one or two-point swing to STOCKFISH. In game 65, LEELA missed a KNP(c4)P(d5)KBP(c5) win with the winning capture admittedly 26 moves down the line (de Man, 2018). STOCKFISH clearly saw it was lost and LEELA would have been awarded the win under the TCEC13 ‘6.5+’ win-adjudication rule. LEELA was within 11 ply of winning with 9 ply to go and it is worth speculating as to how soon it would have found the winning idea, K on b5/c6 before Nxc5, had the plycount not intervened. Game 66 had to be restarted after two server crashes before LEELA – lost. Had it been possible to return to the game-state after the last completed move, the temperature of the partisanship in the chat room would have been lower. A minor cost, but transaction-checkpoint/restart might be applicable here.

Game 85 was the final win: the 12-move King’s Indian opening had already defined the major asymmetry of Queen versus BBPP. LEELA went from apparent equality to negative territory by move 25. Ultimately, LEELA’s QR were unable to prevent mate by a BBNNPP team, only five moves away when the referee stepped in. Game 86 was the longest ever TCEC game at 362 moves.

The Bonus 4-way and 2-way Rapid events

TCEC treated us to two bonus events at the Rapid tempo of 12′+3″/move. The first featured the top four - HOUDINI, KOMODO, LEELA CHESS ZERO and STOCKFISH: 20 DRRs, 40 round robins, 120 rounds and 240 games. STOCKFISH had a good first half and was never headed even if pursued closely by LEELA. HOUDINI and KOMODO tailed off, eventually in that order as KOMODO fared poorly in the second half. ELO-predicted net scores were +9/+1/-2/-8 but ‘actuals’ were +12/+6/-7/-11. The longest wins were g116.1 (1-0, 139 moves) and g37.2 (0-1, 125m): the longest draw, g12.1 (318m). Game 18.1 between Leela and Komodo was something of an anti-climax as a 3x-repetition draw after ten played moves. Full details are included with the repository e-version of this note (Haworth and Hernandez, 2019d).

The second event was a 100-game STOCKFISH–LEELA match from the initial position: no prescribed openings. LEELA won 16-4, perhaps by being single-minded about its openings (Wool, 2019).

Summary

The Google DeepMind company in St. Pancras, London have been remarkably open in sharing the core ideas of their intelligence initiative. In the year it has taken for DeepMind’s papers on ALPHAZERO (Silver et al, 2017/18) to mature and satisfy the referees, we have seen TCEC invest in Nvidia GPUs and foster several innovations going beyond the classic Shannon (1950) minimaxing AB model of a chess engine. We have seen a leading chess-engine author, Mark Lefler, move his focus successfully from top engine KOMODO to KOMODO MCTS (Chessdom, 2018). With one less technical break, this engine would have come all the way through the divisions to fully justify its place in Division P at the first attempt.

We have also seen a community come together to support and train the open-source LEELA CHESS ZERO echo of ALPHAZERO. Again, this has been rewarded by success, and how. LEELA edged out KOMODO and HOUDINI to take the challenger’s place in the Superfinal here. It was not expected to beat STOCKFISH but came within one game of drawing the classic phase.

Chess24 and Chessbomb, with its useful colour-coding of moves, covered the TCEC14 Superfinal so we were treated to kibitzing by three different, objective but hardly neutral versions of STOCKFISH. The Twitch TCEC channel claims that viewers’ computers have to date had a window open to TCEC Seasons 10-14 for a total of over half a million hours.

REFERENCES Full article

To read the full article in pdf, click HERE

published March 11, 2019

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Season 15 of the Top Chess Engine Championship will begin this March 5th at 20:00 CET. It will feature a record number of engines to accommodate the booming sector and a record ELO of the participants. A total of 44 engines will face off in a division based system to determine the current status quo of strength and the grand champion of TCEC.

Stockfish has dominated the Top Chess Engine Championship for four consecutive seasons or a total of 10 months. It has a chance to expand this dominance in Season 15 for over 1 year (see the analysis of GM Matthew Saddler). Stockfish will have to show its best as significant challenge is coming from DeepMind’s Alpha Zero open source sister Leela Chess Zero. LCZero applied significant pressure in the S14 Superfinal, losing by a narrow margin 49,5-50,5. It also convincingly won its first major title the TCEC Cup and the inter season rapid bonus, thus showing high inspiration and determination for the title. These two engines, however, cannot relax, as Komodo with its innovative MCTS version and all other engines from the higher divisions will also bring in their best. Additionally, a new unique supervised learning neural network Allie+Stein will start from the bottom division and is aspiring to attack the top from its very first season.

Where to follow the Top Chess Engine Championship?

The Top Chess Engine Championship has a dedicated website with games going on 24/7 at http://tcec.chessdom.com . You can enjoy the games with live commentary by BlueFish (Stockfish running on 172 cores, represented by a blue line on the graph), Redmodo (Komodo on 128 cores, represented by a red line on the graph) and chat with engine authors, computer chess experts, and friends in the ever lively TCEC chat.

The previous season was record breaking in audience, including in the video broadcast of the event. A total of 500 000+ viewer hours were registered in the past 30 days, which is #1 position on Twitch among the chess channel, and higher than 50% of the total chess audience viewership altogether. You can follow the alternative video broadcast at TCEC Twitch TV

TCEC is the most watched chess channel and competition by far, according to the official Twitch Metrics

Growth of TCEC session viewers season by season from 2015 to 2019 visualized by Google analytics. S14 was a !boom (click on the image to expand)

TCEC Season 15 participants

Premier Division

1. Stockfish
2. LCZero
3. Komodo
4. Houdini
5. Fire
6. Ethereal
7. Promotion from Div 1
8. Promotion from Div 1

Division 1

1. Andscacs
2. Komodo MCTS
3. Fizbo
4. Chiron
5. Laser
6. Jonny
7. Promotion from Div 2
8. Promotion from Div 2

Division 2

1. Fritz
2. Ginkgo
3. Xiphos
4. Booot
5. Nirvana
6. ChessBrainVB
7. Promotion from Div 3
8. Promotion from Div 3

Division 3

1. Gull
2. Texel
3. Arasan
4. Vajolet
5. Pedone
6. rofChade
7. Promotion from Div 4
8. Promotion from Div 4

Division 4

Division 4 of the Top Chess Engine Championship will evolve this season 15. It will expand to accommodate all engines that are active and have ELO above 3000. It still remains testing ground for new engines and as support for the computer chess field, however, one cannot ignore that from this division have started engines like LCZero, Komodo MCTS, Ethereal, ChessBrainVB, etc.

Division 4 will be with two groups, !division4a and !division4b. Each will contain ten engines that will play a 1xDRR tournament with time control 30 min + 5 sec. The top two engines of the groups will promote to a playoff – !div4playoff. That will be a 2xDRR event which will determine the two engines advancing to Div 3.

Here is the full list of participants

Division 4 A

1. Winter
2. Tucano
3. Topple
4. Rodent III
5. Nemorino
6. The Baron
7. Cheese
8. Igel
9. Minic
10. RubiChess

Division 4 B

1. Pirarucu
2. Allie+Stein
3. Wasp
4. Chess22k
5. Marvin
6. Monolith
7. ScorpioNN
8. Jumbo
9. Bagatur
10. Gaviota

Rules and regulation changes for Season 15

The rules set of TCEC advances one more step this season 15, to resolve specific cases that arose in previous seasons. Besides the expansion of Division 4, here is what will change in all divisions:

* Increment in Div P, Div 1, Div 2, Div 3, and Div 4 will be 5 sec per move. The Superfinal increment is reduced to 10 sec per move

* Black wins will be removed from the tiebreak criteria, due to the usage of advanced books. The order of tiebreak criteria is as follows 1: # of crashes. 2: Direct encounter. 3: # of wins. 4: SB. 5: TD decision

* In case of a server disconnect or other interruption not caused by the engines:
**If the web server crashes, the game continues unaffected, and broadcasting will be resumed as soon as possible;
**If the evaluation of both engines is >=ABS(10) at the moment of game server interruption, the game is scored as a win ;
**If the evaluation of both engines is <=ABS(0.10) at the moment of game server interruption, and both engines have completed 35 moves, the game is scored as a draw;
**If a game interrupts with 7 pieces on the board, position on the board at the instant of game server interruption will be adjudicated according to 7-man EGTB.
** In all other cases the game is restarted from the position that the two engines reached before the disconnect, with time compensation to fill up the cache

* The top 4 engines of !div4a and the top 4 of !div4b, so 8 in total, qualify for the TCEC Cup. The engines in the playoff are seeded by their final standings, the other ones are placed on a random draw seeds 29-32

There will be no change of the 3 strike out rule this season, due to the multiple cases that have to be covered. An engine update during a division will count as a strike.

The engine uniqueness requirement stays intact , as worded in the S14 rules and regulations.

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

The new neural network entering TCEC Season 15

Season 12 of the Top Chess Engine Championship saw the arrival of NN engines, modeled after DeepMind findings Alpha Zero, into public computer chess events. At that season LCZero played with CPU only. In Season 13 it was handed a GPU, and only a season later that neural network won its first major title.

Besides LCZero, one more neural network entered TCEC. It was Scorpio NN by Daniel Shauwl. And even though it did not make it past Div 4, it showed a trend in computer chess: people understood the potential value and advantages of neural networks in chess.

Season 15 is about to start and one more neural network is going to enter TCEC. This is a combined project by Adam Treat and his Allie and Mark Jordan and his Leelenstein. The new engine is called Allie+Stein, a unique engine by the TCEC rules that will start its quest for top positions and climb of the ladder from Division 4.

Here is an extensive interview with the authors Adam Treat and Mark Jordan.

Season 15 of TCEC will start this March 4th, live on the official website of TCEC and the TCEC Chess TV.

Your engine Allie+Stein will be the new neural network in TCEC, making its debut this Season 15. Welcome to the Top Chess Engine Championship!

Adam Treat: Allie is a very new chess engine, but represents a lot of hard work over the last several months, so I’m hoping TCEC can provide an opportunity to see how she stacks up against a host of more established engines. Combining Allie with the Leelenstein network will also be interesting given that both introduce new avenues of research in NN chess engines. I an new to the chess programming community and culture and so I am excited to participate and learn through TCEC.

Mark Jordan: I am excited to see how far an engine that uses supervised learning (SL) exclusively can go and I hope there will be more strong SL networks to compare against for benchmarking average performance and maximum performance of the method.

So far there have been two unique NNs in TCEC – LCZero and Scorpio. What makes your engine the third unique NN engine by the TCEC rules

AT: Like Leela, Allie is based on the same concepts and algorithms that were introduced by Deepmind in the AlphaZero paper(s), but her code is original and contains an alternative implementation of those ideas. You can think of Allie as a young cousin of Leela that can utilize the same networks produced by the Lc0 project or other compatible networks. The Leelenstein network is also a novelty in that it introduces supervised learning into the TCEC competition. Finally, Allie+Stein will be using MCTS for the beginning portions of the tournament, but I’m hoping to switch to AlphaBeta search during later rounds… if she makes it that far :)

MJ: Allie+Stein is a completely new engine and neural network produced, thus easily satisfying 2 out if the 3 conditions for uniqueness. It is possible eventually that I will rewrite the training scripts completely with some more new ideas in the future. Currently, training has some changes to the Leela training scripts.

Can you share more on how is Allie+Stein engine being trained?

MJ: Training started from a random initialized network, and consisted of mostly CCRL computer games (I used all of them available), and some weak Leela t30 games and some games from other experiments I was able to gather. These were were all about 100 elo weaker than t10. I tried several new learning techniques using these same games. My goals have mostly been focused on NN learning experimentation. I compared the learning schedule and optimizer that Leela used to different ones from some academic papers, and it seemed to improve performance, but it was still about 50 elo weaker than the best Leela nets. So I replaced all the weak t30 games with many more recent ones, using all of January’s games, while still keeping the CCRL games in the training window, and continuing on several more of my cyclical learning rate cycles.

And how about search?

AT: To begin the tournament, Allie will perform MCTS based search with absolute fpu where new nodes start off with win pct of -1. The search is modeled after Deepminds paper’s. As I said above, I’m hoping to switch to AlphaBeta for the long term direction of the project. I’ve experimented with many, many ways of doing this with the networks generated by the Lc0 project and I think I’ve hit upon a way to achieve the depths required to maintain ELO level with MCTS based search, but it is not ready yet. In the future, I imagine we’ll see a lot of experimentation with different variations of search (mcts, ab) + eval (handwritten, NN) in computer chess engines. Hoping to be a part of that and to contribute to the shared pool of knowledge.

What is the strength of Allie+Stein? What division do you expect to reach? Do you think stronger versions will come out as the season progresses and thus have a better shot at the Premier Division?

MJ: I think the neural network seems to scale pretty well with more nodes. In bullet testing it loses to Stockfish quite a bit, but seems to hold its own in tournaments with more time. I believe it has potential to get to division 1, but will require some more work to better utilize multiple GPUs and support tablebases. I have some ideas to use tablebases in learning as well that could end up making the neural network Premier Division material. But for this season I will be happy even with just getting to Division 2.

AT: This is a question I get quite often. Most people familiar with chess engines know that it is extremely hard to compare strengths other than through heads up competition with a set of match rules/controls in place. That is exactly what TCEC provides. A levelish playing field and a set of machines/rules agreed upon beforehand to determine the relative strength of different algorithms and their implementations. So with a huge grain of salt I’ll say that on my own rather meager hardware I have experienced heads up matches between Allie and Lc0 at short time controls with the same network with a relative difference of 50-100 elo in Lc0′s favor. Obviously, that is with only one GPU. Throw in the advanced multi-gpu hardware here at TCEC and that is a very big new variable. Add the other engines and we have another large unknown variable. Add in that we’ll be using a Supervised Learning network and yet another big variable. Then we have the fact that Allie will start off the tournament without TB support. Just lots and lots of variables. So, ok I expect she will be able to advance out of Div 4, but anything is possible. I’ll be happy if she is able to get winning positions and mate consistently in them :)

Comparing to the top NN engine now Lc0 , do you expect with your approach to have better future?

MJ: The beauty of my approach using only existing games is I can train whole new networks from scratch in a week or two to try a new idea, something that would take the Lc0 project several months, even with 100x or 1000x more compute than I have. I hope to eventually show that some of my ideas must be good by how strong the network is and Lc0 can try them to become strong as well. And I will also continue to try good ideas from there as well. And the project is a great resource for people who need many millions of strong games, as it already has many more games available than all of CCRL even though it is much younger. And sadly fishtest doesn’t store and host all of its games. So I expect to see some great symbiosis.

AT: I expect that whatever advances are made by one engine will (with time and effort) be incorporated into other engines. As they should be! We are all standing on the shoulders of the pioneers in computer science and chess engine programming that came before. I do think that AlphaBeta is a superior search method and that there is no reason that an AB+(eval method) engine can’t compete favorably with an MCTS+(eval method) regardless the eval method. But this is just my theory and worth very little until proven. Only the future will tell.

What are your thoughts on the current hardware balance at TCEC? Are you happy with the hardware your engine is going to play on?

MJ: There are always debates about hardware balance, and I think a wide range power’s are somewhat fair, and as long as the specifications and NPS numbers are published and maintained throughout the tournament it is a reasonable tournament data point. The balance seems to be within a 2x order of magnitude of fair based on any of purchase price, watts, and total cost of ownership metrics which I think is as close as it can possibly get to please everyone. But it does leave room for future debate and improvement. 2x the power or cost doesn’t translate to 2x the nodes so the real elo difference is probably not enough to change what division any engine will end up in most cases, even if it changes a result or two. The format is not guaranteed to find 20-30 elo differences anyway, so I don’t see it as a big issue except during boring games :)

AT: Considering that I’ve never personally tested Allie on this level hardware, sure I’m happy. I just hope she scales well enough.

Read: Stockfish dominance continues and Analysis by GM Matthew Sadler

Stockfish has been dominant at TCEC for almost 1 year.

AT: Stockfish is the strongest engine until proven (convincingly) otherwise. I have great respect for all the developers who work on Stockfish (and other engines for that matter) and think the community of chess programmers is pretty collegial.

MJ: I very much support non-private engines, so I am glad to see that there has never been a private engine to win the Superfinal. And even more glad that it seems that free and even open-source projects can win. I am glad such excellent chess is truly accessible to all.

Read: interview with Alexander Lyashuk

Alexander Lyashuk from Lc0 shared in an interview that he expects at least 5 NNs to appear and they to dominate computer chess. Do you share this vision? Do you expect one of those NN engines to be Allie+Stein?

AT: I do think it is only a matter of time before NN eval is shown to beat handwritten eval regardless the search method. Right now, if you limit the number of nodes – hands down – any of the NN’s will beat the traditional handwritten eval engines. This is just a fact. Still, I have great respect for the ingenuity of those writing the traditional engines. I hope Allie+Stein will be a meaningful engine in terms of helping to advance the state of art in computer chess programming in the near future.

MJ: As I mentioned above, I am excited to see more projects, and I hope there will be enough of them that we have to pick the most exciting and unique ones. We can use them to develop more tests to determine how unique they are in their ideas of openings and in general play, and which ones are truly beyond average. Of course I hope to have one of these beyond, so the top 5 sounds nice!

  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

The TCEC14 Computer Chess Superfinal: a perspective
by Matthew Sadler
London, UK

Season 14 has been amazing! The clash of styles between STOCKFISH and LEELA produces consistently interesting chess – just like the match between STOCKFISH and ALPHAZERO did. You really hope that this sort of clash of styles can continue for the next few seasons.

Just a few impressions about the games. I was really impressed by LEELA’s handling of the White side of G17, the Budapest. I’ve done a fair amount of analysis on this line and a few good players have got stuck against it. Aronian defeated Ivanchuk in 2013 at the London Candidates but more due to Ivanchuk’s terrible time trouble than due to getting anything out of the opening. Giving the c-pawn back very quickly is a great way of putting the pressure right back on Black. I also liked G45 a lot: a Hedgehog where LEELA generated a huge amount of play with some risky but incisive pawn play on both sides of the board. The LEELA win in G53 was extremely nice too: again, it’s impressive to see Black being stretched so greatly with rook’s pawn thrusts on both sides (18. a4 and 23. h5) then a thrust of the g-pawn (24. g6) followed by returning to the queenside with 28. c4. This sort of whole board vision is what grabbed me so much when I first played through ALPHAZERO’s games – the game ‘exactly how to attack’ (Sadler and Regan, 2019) is the best example of this – and it makes for wonderful chess when you have a player who can execute it so well.

I found G62 incredible: the move 10. … 0-0-0 would never have occurred to me! In some ways, I felt that LEELA was playing like STOCKFISH – grabbing a hot pawn and betting on surviving with a rather dodgy king – and STOCKFISH was playing like LEELA – sacrificing a pawn for open lines against the king. G63 was magnificent: perhaps the most ALPHAZERO-like game of the whole match. Material balance is not important, accurate assessment of an opposite-coloured bishop position, restriction of the opponent’s king, entombing of the Black rook on h8 (a little like the game ‘Python squeeze’ featured in Game Changer). It’s a classic game: it was wonderful to watch and I’ve played through it a few times since with great pleasure. G66 was a wonderful effort from STOCKFISH. It followed a STOCKFISH–ALPHAZERO game until move 16 in which ALPHAZERO was also under great pressure and the power of STOCKFISH’s attack was huge! G81 and G82 (the opening part) was a great pair of games in a sharp Slav gambit. G81 was particularly nice with rook’s pawn thrusts on both sides of the board from LEELA to tie Black up on the queenside – even at the cost of another pawn – while LEELA targets the kingside (it’s another theme you see quite often in ALPHAZERO’s games). G85 and G86 (the opening part) were also fantastic. STOCKFISH handles these positions with so much energy: quite amazing!

REFERENCES
Sadler, M. and Regan, N. (2019). Game Changer, esp. pp 38-43 and 99-100. New in Chess.

Communicating author: https://matthewsadler.me.uk/

Read for later

Articles marked as Favorite are saved for later viewing.
close
  • Show original
  • .
  • Share
  • .
  • Favorite
  • .
  • Email
  • .
  • Add Tags 

Separate tags by commas
To access this feature, please upgrade your account.
Start your free month
Free Preview