Chessnut EVO Maia engine testing
Chessnut have now updated the software in the EVO that allows the Maia engine to perform lookahead. There is a slider control for setting a value from 0 to 6 ply. Zero value of course means no lookahead and Maia just relies on the neuronal weights it was trained with to find its moves. This in keeping with the concept of the Maia project, but as many of us are aware this results in rather poor play with the engine blundering pieces. I was shocked to see Maia1900 (apparently trained from human games rated at 1900) not really matching player's of that standard, in fact playing more in the lower to middle 1400's.
With the slider set to 6 ply lookahead, which is more in keeping with the depths reached by the early dedicated machines coupled with the neuronal training, I have seen a marked improvement in play. No longer does it blunder pieces but also plays quite interesting chess. So I decided to perform some test games against Shredder Chess for Android using the playing strength control and auto strength adjustment options. Initially setting Shredder to 1899 ELO (I could not quite get 1900 on the small screen) and just playing several games without recording them. This is just to get an idea of the Maia1900's actual ELO (assuming Shredder engine's ELO calculations are correct and I'm pretty sure they will give a good idea). As I play the games, Shredder adjusts it's strength for the next game and updates it's ELO calculation of the EVO. I've played around 30 games so far as they are very rapid with both engines moving almost immediately. The EVO's rating started at over 2200 but has slowly been decreasing. Today it started at 2169 and has been fluctuating +/- 30 ELO, but this fluctuation is beginning to lessen and it's current rating after 30 games is 2140.
I'll carry on until the calculated rating settles (if it does!) and report the result. Once this is known I plan to pit it against one of my dedicated machines and post the games and results.
I have suggested to Chessnut that they allow the user to type in a max depth of search rather than being limited to 6 ply because it would be interesting to see just how much stronger the Maia engine could be. They responded saying they like the idea and so we may see this option added.
One thing I have noticed about the games with Maia, when it plays with the white pieces it will always start with P-K4. I'm not aware of any opening books on the Evo, so at the start of a game Maia is playing from the weights file and any allowed depth of search. In this respect, the Maia will be less attractive to some players who will want to play against other openings. It is of course possible to easily set up a start position and play from there, so any book opening could be followed.
I have also noticed that Maia1900 is lacking in end game knowledge. I witnessed one game where it had King and Rook against King and only achieved a 3-fold draw by moving it's king back and forth, never utilising its rook! I think a magnitude higher number of games are required in order for it to recognise certain end game scenarios.
So, a big improvement on the Maia side and Chessnut seem to be very proactive on this project.
Regards
Ray
|