Developing Fire is rather straightforward… but many would find the process tedious, and it is very time consuming.
​
I use 3 modern Windows 10 systems that I use as a test bed. These are fast i7s (3930K and 4930K with hyper threading ‘on’) with 16 GB RAM. I run LittleBlitzer application (http://www.kimiensoftware.com/software/chess/littleblitzer) on each system.
I run 1 game per thread, which = 12 simultaneous games per system, 36 simultaneous game in total. The games are ultra-fast (1000 ms + 100 ms inc., about 14 secs. per game), so within a few hours you can have thousands of games played, establishing a high level on confidence in the result. I have remote access to each box, and I use a series of processing scripts to collect and concatenate the PGN files, run utilities like Ordo, etc., to calculate Elo and LOS. Each single change (no matter how small) to the source code is tested in this fashion.
In this manner, with discipline and over a period of months, you can make significant progress.
​
Most of the testing is ‘self-play’ (the most recent change tested against the version directly previous… for ex: Fire 590 vs Fire 589), but I also run the most recent dev version occasionally against 5 different strong opponents (in gauntlet mode) as well, to verify results.
​
I save and track the test results and Ordo calculations with a spreadsheet and text document.
With this information I determine if the code change is acceptable or not.
​
Games Completed = 16384 of 16384 (Avg game length = 13.869 sec)
Settings = Gauntlet/32MB/1000ms+100ms/M 500cp for 6 moves, D 120 moves/EPD:book.epd(31526)
Time = 38500 sec elapsed, 0 sec remaining
1. Fire 590 8215.5/16384 3892-3845-8647 (L: m=3 t=156 i=0 a=3686) (D: r=6217 i=1505 f=236 s=70 a=619) (tpm=110.3 d=14.74 nps=2359536)
2. Fire 589 8168.5/16384 3845-3892-8647 (L: m=9 t=162 i=0 a=3721) (D: r=6217 i=1505 f=236 s=70 a=619) ( tpm=110.3 d=14.74 nps=2354312)
​
Fire 590 3401 : 16384 (+3892,=8647,-3845), 50.1 %
vs. : games ( +, =, -), (%) : Diff, SD, CFS (%)
Fire 589 : 16384 ( 3892, 8647, 3845), 50.1 : +1, 2, 67.6
1 Fire 590 : 3400.5 2.4 8215.5 16384 50.1%
2 Fire 589 : 3399.5 2.4 8168.5 16384 49.9%