Chessnut have now updated the software in the EVO that allows the Maia engine to perform lookahead. There is a slider control for setting a value from 0 to 6 ply. Zero value of course means no lookahead and Maia just relies on the neuronal weights it was trained with to find its moves. This in keeping with the concept of the Maia project, but as many of us are aware this results in rather poor play with the engine blundering pieces. I was shocked to see Maia1900 (apparently trained from human games rated at 1900) not really matching player's of that standard, in fact playing more in the lower to middle 1400's.

With the slider set to 6 ply lookahead, which is more in keeping with the depths reached by the early dedicated machines coupled with the neuronal training, I have seen a marked improvement in play. No longer does it blunder pieces but also plays quite interesting chess. So I decided to perform some test games against Shredder Chess for Android using the playing strength control and auto strength adjustment options. Initially setting Shredder to 1899 ELO (I could not quite get 1900 on the small screen) and just playing several games without recording them. This is just to get an idea of the Maia1900's actual ELO (assuming Shredder engine's ELO calculations are correct and I'm pretty sure they will give a good idea). As I play the games, Shredder adjusts it's strength for the next game and updates it's ELO calculation of the EVO. I've played around 30 games so far as they are very rapid with both engines moving almost immediately. The EVO's rating started at over 2200 but has slowly been decreasing. Today it started at 2169 and has been fluctuating +/- 30 ELO, but this fluctuation is beginning to lessen and it's current rating after 30 games is 2140.

I'll carry on until the calculated rating settles (if it does!) and report the result. Once this is known I plan to pit it against one of my dedicated machines and post the games and results.

I have suggested to Chessnut that they allow the user to type in a max depth of search rather than being limited to 6 ply because it would be interesting to see just how much stronger the Maia engine could be. They responded saying they like the idea and so we may see this option added.

One thing I have noticed about the games with Maia, when it plays with the white pieces it will always start with P-K4. I'm not aware of any opening books on the Evo, so at the start of a game Maia is playing from the weights file and any allowed depth of search. In this respect, the Maia will be less attractive to some players who will want to play against other openings. It is of course possible to easily set up a start position and play from there, so any book opening could be followed.

I have also noticed that Maia1900 is lacking in end game knowledge. I witnessed one game where it had King and Rook against King and only achieved a 3-fold draw by moving it's king back and forth, never utilising its rook! I think a magnitude higher number of games are required in order for it to recognise certain end game scenarios.

So, a big improvement on the Maia side and Chessnut seem to be very proactive on this project.

Regards
Ray

After more testing Evo's Maia1900 ELO rating has still been fluctuating a fair amount - in the region of 100 points - and now after 36 games has held a rating of 2208 for two consecutive games. It needs more test games before I can possibly achieve a more definitive estimate using Shredder's ELO estimation method. Assuming Shredder's calculations are correct then all I would say at the moment is that Maia1900 with 6ply search has a rating probably at least 2100.

I have noticed that Maia does lack certain end game knowledge such as King and Rook against King as mentioned before. However, I came across the following position with Maia playing Black and Black to move and it appears this confuses the Maia engine as it seems to get stuck!



The game is a draw as Black must take the Rook with his Bishop, but I left Maia running for 10 minutes and no move was given. This is way longer than is required to search 6 ply on the Evo. I then tried the built-in Lc0 engine with it's default weight file and this was the same, so there is a bug in the Evo's Lc0 engine. I suspected this might be the case because Maia uses the Lc0 engine and Maia is really just a set of weight files used by Lc0.

Ray

Well, after playing 50 or so games between Maia1900 with 6ply search against Shredder Android, the Maia personality achieved an average rating of 2200. With this in mind I decided to pit it against the original Millennium ChessGenius (MCG) that has an Active chess rating of 2220 on the Wiki ELO list. I thought this would be a good test of the Maia's actual rating. However, after three games I stopped because the MCG was being crushed, one game in particular was in spectacular style! If anyone would like to see the games let me know and I'll post them. So I have been wondering at the 2200 rating that Shredder Android gave it as there is something not quite adding up. What I don't know of course is how Shredder Android calculates the rating, it could be based on the FIDE method in which case it's giving Maia around 100 points less than the Wiki ELO system may give. So I decided to pit Maia 1900 with 6 ply lookahead against Millennium ChessGenius Pro 2024 at Active chess for 10 games. Currently about to start game 7 of 10 with the MCGP2024 using the classic opening book. I'm thinking of playing another match using the HIARCS book for comparison. Obviously the MCGP2024 is proving to be a better opponent for the Maia personality. I'll post the games as soon as the match is over. With the Maia moving very rapidly, the games don't take long to complete. Of course, the processing speed of the Evo is not so relevant here because it's limited to 6 ply and does not calculate in opponents time. The only disadvantage the MCGP2024 has is less pondering time and not sure how much that will impact on the final result.

There is some interesting play and the new lookahead feature on the Chessnut Evo's Maia bot has certainly transformed it's playing ability dramatically. I'll post more comments with the games. The only thing I would say is that the Maia bot always opens with P-K4 when playing white, and there is a good chance of seeing repeated games.

Ray

Not looking at the bug, only at the position:
The game is a draw. Certainly !

The game is a draw as Black must take the rook.
No, not really.

Imagine a team match, last game, trailing a whole point - you must win otherwise the match is lost.
Here I would avoid the direct stalemate draw with BxR, instead I would play on with 1. ...Kd4 hoping for 2. Ra2 c4, perhaps a swindle against a weaker
player might still be possible.
Of course the position even after 1. ...Kd4 is still a forced draw, a stronger player would aim for 2. Re6 Bb7 3. Rb6 Bc8 4. Rc6 Bb7 and he has
the choice to go for a threefold repetition or even 5. Rxc5 draws as the white king goes to the a1-corner and black can't win as he has a bishop of the wrong colour.

At least a try...maybe Maia/Lc0 thinks in a similar way...I'm not going full throttle...whom do I play...a patzer ??

Regards
Hans-Jürgen

@Ray: If you don't mind I would like to have a look at that game with - as you called it in #3 - spectacular style.

Regards!

I will post the three games out of interest. I'm just writing up results from the current match between MCGPro2024 and Maia.

Now the first match has finished - with a good result all round, that being a win for the MCGP2024 but a respectable rating for the Maia 1900 bot - I've just started a second round of 10 games this time with the MCGP2024 using the Modern (HIARCS) book. I'll post the results as usual, but just as a taster the first game was a draw (I accepted MCGP's offer as it was sensible) but in the two games that followed the MCGP2024 was totally crushed. It faced two King-side attacks, did not see it coming and was unable to defend. Can the change of opening book make this much difference or will the Millennium machine come back fighting?

Ray

Yes I agree for the situation you describe, play to the bitter end in the hopes the opponent will blunder. However in this game between computers unlikely to blunder it's better for me the game end sooner than be dragged out to the same conclusion :-)