The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

PR

overall

Problem 20sec 1200sec 3600sec
1a7w 74.3 (-1.5) 99.8 (-1.0) 99.8 (-1.0)
1aie 0.0 (-inf) 100.0 (2.8) 100.0 (2.8)
1b2v 100.0 (-53.4) 100.0 (-53.4) 100.0 (-53.4)
1bxv 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
1cpq 0.0 (nan) 0.0 (nan) 0.0 (nan)
1ctj 0.0 (nan) 100.0 (-13.3) 100.0 (-13.3)
1cxy 100.0 (-38.0) 100.0 (-38.0) 100.0 (-38.0)
1f94 0.0 (-inf) 100.0 (-61.4) 100.0 (-61.4)
1fas 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
1g6x 99.8 (-27.2) 100.0 (-27.2) 100.0 (-27.2)
1jer 0.0 (-inf) 100.0 (-31.5) 100.0 (-31.5)
1kp6 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
1nkd 0.0 (nan) 0.0 (nan) 0.0 (nan)
1npl 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
1qt9 0.0 (-inf) 0.0 (nan) 0.0 (nan)
1rfs 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
1svy 0.0 (nan) 0.0 (nan) 0.0 (nan)
1vfy 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
2rta 100.0 (-55.2) 0.0 (-227.6) 0.0 (-227.6)
BN_12 99.5 (-3.6) 99.9 (-3.6) 99.9 (-3.6)
BN_13 98.3 (-2.4) 98.2 (-2.4) 98.2 (-2.4)
BN_15 94.1 (-6.0) 97.4 (-5.8) 97.4 (-5.8)
BN_31 0.0 (nan) 0.0 (nan) 0.0 (nan)
BN_74 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_10_01-0006 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_10_01-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_10_02-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_10_03-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_10_03-0010 0.0 (nan) 0.0 (nan) 0.0 (nan)
blockmap_15_01-0008 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
blockmap_15_02-0008 0.0 (nan) 100.0 (-2.4) 100.0 (-2.4)
blockmap_15_03-0008 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
blockmap_15_03-0010 0.0 (nan) 100.0 (-3.1) 100.0 (-3.1)
blockmap_20_01-0009 0.0 (nan) 100.0 (-1.3) 100.0 (-1.3)
blockmap_20_02-0009 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
blockmap_20_03-0010 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
blockmap_22_01-0006 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
blockmap_22_03-0009 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
bwt3ac.wcsp 100.0 (0.0) 100.0 (0.0) 100.0 (0.0)
fs-07 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0001 0.0 (nan) 100.0 (-12.7) 100.0 (-12.7)
mastermind_03_08_04-0002 0.0 (nan) 100.0 (-14.5) 100.0 (-14.5)
mastermind_03_08_04-0003 0.0 (nan) 100.0 (-11.5) 100.0 (-11.5)
mastermind_03_08_04-0007 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0008 0.0 (nan) 100.0 (-16.3) 100.0 (-16.3)
mastermind_03_08_04-0010 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0013 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_04-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0001 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0003 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0004 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0008 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0013 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0013 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_03-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0001 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0002 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0003 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0004 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0005 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0013 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0001 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0002 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0003 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0004 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0013 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0000 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0001 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0002 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0003 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0005 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0011 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0012 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0014 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0015 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0001 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0003 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0004 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0008 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0009 0.0 (nan) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0010 0.0 (nan) 0.0 (nan) 0.0 (nan)
myciel5g_3.wcsp 94.5 (-57.1) 100.0 (-61.2) 100.0 (-61.2)
myciel5g_4.wcsp 63.6 (-19.2) 92.5 (-5.7) 95.7 (-6.7)
or_chain_1.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_102.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_106.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_107.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_12.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_128.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_132.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_138.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_140.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_149.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_15.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_150.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_153.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_155.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_161.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_186.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_188.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_198.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_209.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_242.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_4.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_53.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_61.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_64.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_90.fg 0.0 (nan) 0.0 (nan) 0.0 (nan)
pedigree51 0.0 (nan) 0.0 (nan) 0.0 (nan)
queen5_5_3.wcsp 99.1 (-112.0) 100.0 (-113.0) 100.0 (-113.0)
queen5_5_4.wcsp 91.4 (-50.7) 97.7 (-44.4) 97.8 (-44.4)
rus2_20_40_0_1 97.8 (108.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_0_2 100.0 (120.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_0_3 100.0 (88.2) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_1 99.5 (107.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_2 99.7 (104.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_3 96.2 (104.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_1 100.0 (119.3) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_2 100.0 (118.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_3 97.4 (114.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_1 94.2 (113.8) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_2 96.3 (143.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_3 95.1 (126.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_1 97.6 (100.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_2 99.7 (107.2) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_3 94.9 (98.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_1 98.0 (96.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_2 92.5 (93.8) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_3 100.0 (169.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_1 97.4 (120.8) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_2 95.0 (71.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_3 99.7 (134.3) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_1 95.4 (91.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_2 99.7 (111.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_3 98.3 (82.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_1 94.6 (110.8) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_2 94.5 (103.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_3 94.5 (104.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_1 95.6 (97.7) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_2 93.9 (99.3) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_3 92.7 (69.1) 0.0 (nan) 0.0 (nan)
rus_20_40_0_1 100.0 (617.3) 0.0 (nan) 0.0 (nan)
rus_20_40_0_2 100.0 (791.6) 0.0 (nan) 0.0 (nan)
rus_20_40_0_3 100.0 (903.0) 0.0 (nan) 0.0 (nan)
rus_20_40_1_1 100.0 (1004.7) 0.0 (nan) 0.0 (nan)
rus_20_40_1_2 100.0 (910.2) 0.0 (nan) 0.0 (nan)
rus_20_40_1_3 100.0 (900.6) 0.0 (nan) 0.0 (nan)
rus_20_40_2_1 100.0 (797.8) 0.0 (nan) 0.0 (nan)
rus_20_40_2_2 100.0 (689.3) 0.0 (nan) 0.0 (nan)
rus_20_40_2_3 100.0 (749.3) 0.0 (nan) 0.0 (nan)
rus_20_40_3_1 100.0 (825.9) 0.0 (nan) 0.0 (nan)
rus_20_40_3_2 100.0 (839.0) 0.0 (nan) 0.0 (nan)
rus_20_40_3_3 100.0 (841.9) 0.0 (nan) 0.0 (nan)
rus_20_40_4_1 100.0 (931.5) 0.0 (nan) 0.0 (nan)
rus_20_40_4_2 100.0 (935.7) 0.0 (nan) 0.0 (nan)
rus_20_40_4_3 100.0 (853.3) 0.0 (nan) 0.0 (nan)
rus_20_40_5_1 100.0 (875.9) 0.0 (nan) 0.0 (nan)
rus_20_40_5_2 100.0 (839.7) 0.0 (nan) 0.0 (nan)
rus_20_40_5_3 100.0 (882.0) 0.0 (nan) 0.0 (nan)
rus_20_40_6_1 100.0 (723.9) 0.0 (nan) 0.0 (nan)
rus_20_40_6_2 100.0 (982.4) 0.0 (nan) 0.0 (nan)
rus_20_40_6_3 100.0 (966.7) 0.0 (nan) 0.0 (nan)
rus_20_40_7_1 100.0 (752.1) 0.0 (nan) 0.0 (nan)
rus_20_40_7_2 100.0 (756.2) 0.0 (nan) 0.0 (nan)
rus_20_40_7_3 100.0 (869.7) 0.0 (nan) 0.0 (nan)
rus_20_40_8_1 100.0 (726.9) 0.0 (nan) 0.0 (nan)
rus_20_40_8_2 100.0 (898.7) 0.0 (nan) 0.0 (nan)
rus_20_40_8_3 100.0 (703.8) 0.0 (nan) 0.0 (nan)
rus_20_40_9_1 100.0 (866.6) 0.0 (nan) 0.0 (nan)
rus_20_40_9_2 100.0 (834.5) 0.0 (nan) 0.0 (nan)
rus_20_40_9_3 100.0 (864.0) 0.0 (nan) 0.0 (nan)

Updated: