The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

PR

overall

Problem 20sec 1200sec 3600sec
1a7w 100.0 (-1.0) 0.0 (nan) 0.0 (nan)
1aie 99.4 (2.8) 0.0 (nan) 0.0 (nan)
1b2v 100.0 (-53.4) 100.0 (-53.4) 100.0 (-53.4)
1bxv 100.0 (-9.7) 0.0 (nan) 0.0 (nan)
1cpq 99.9 (-21.7) 0.0 (nan) 0.0 (nan)
1ctj 99.8 (-13.3) 100.0 (-13.3) 100.0 (-13.3)
1cxy 100.0 (-38.0) 0.0 (nan) 100.0 (-38.0)
1f94 99.9 (-61.4) 0.0 (nan) 0.0 (nan)
1fas 0.0 (nan) 0.0 (nan) 0.0 (nan)
1g6x 0.0 (nan) 0.0 (nan) 100.0 (-27.2)
1jer 100.0 (-31.5) 0.0 (nan) 100.0 (-31.5)
1kp6 99.9 (-45.1) 0.0 (nan) 100.0 (-45.1)
1nkd 100.0 (-8.8) 0.0 (nan) 0.0 (nan)
1npl 100.0 (-42.9) 0.0 (nan) 100.0 (-42.9)
1qt9 0.0 (nan) 0.0 (nan) 0.0 (nan)
1rfs 100.0 (-50.6) 0.0 (nan) 0.0 (nan)
1svy 100.0 (-38.6) 0.0 (nan) 0.0 (nan)
1vfy 0.0 (nan) 0.0 (nan) 0.0 (nan)
2rta 100.0 (-55.2) 0.0 (nan) 100.0 (-55.2)
BN_12 100.0 (-3.6) 100.0 (-3.6) 100.0 (-3.6)
BN_13 100.0 (-2.3) 0.0 (nan) 100.0 (-2.3)
BN_15 100.0 (-5.7) 0.0 (nan) 0.0 (nan)
BN_31 99.7 (-11.4) 0.0 (nan) 0.0 (nan)
BN_74 99.7 (-0.5) 100.0 (0.0) 0.0 (nan)
blockmap_10_01-0006 0.0 (-inf) 100.0 (-1.0) 0.0 (nan)
blockmap_10_01-0009 100.0 (-1.0) 0.0 (nan) 0.0 (nan)
blockmap_10_02-0009 99.8 (-2.2) 0.0 (nan) 0.0 (nan)
blockmap_10_03-0009 100.0 (-3.2) 100.0 (-3.2) 100.0 (-3.2)
blockmap_10_03-0010 100.0 (-2.9) 0.0 (nan) 0.0 (nan)
blockmap_15_01-0008 100.0 (-1.2) 100.0 (-1.2) 100.0 (-1.2)
blockmap_15_02-0008 100.0 (-2.4) 100.0 (-2.4) 100.0 (-2.4)
blockmap_15_03-0008 0.0 (-inf) 0.0 (nan) 0.0 (nan)
blockmap_15_03-0010 100.0 (-3.1) 100.0 (-3.1) 100.0 (-3.1)
blockmap_20_01-0009 100.0 (-1.3) 100.0 (-1.3) 100.0 (-1.3)
blockmap_20_02-0009 0.0 (nan) 100.0 (-2.6) 0.0 (nan)
blockmap_20_03-0010 0.0 (nan) 100.0 (-4.0) 100.0 (-4.0)
blockmap_22_01-0006 0.0 (nan) 100.0 (-1.3) 100.0 (-1.3)
blockmap_22_03-0009 0.0 (nan) 0.0 (nan) 100.0 (-4.1)
bwt3ac.wcsp 98.7 (-0.1) 100.0 (0.0) 100.0 (0.0)
fs-07 100.0 (-14.8) 100.0 (-14.8) 100.0 (-14.8)
mastermind_03_08_04-0000 0.0 (-inf) 0.0 (nan) 100.0 (-0.0)
mastermind_03_08_04-0001 99.7 (-13.0) 100.0 (-12.7) 100.0 (-12.7)
mastermind_03_08_04-0002 99.9 (-14.4) 100.0 (-14.5) 100.0 (-14.5)
mastermind_03_08_04-0003 99.5 (-12.2) 100.0 (-11.5) 100.0 (-11.5)
mastermind_03_08_04-0007 100.0 (-16.6) 100.0 (-16.6) 100.0 (-16.6)
mastermind_03_08_04-0008 100.0 (-16.3) 100.0 (-16.3) 100.0 (-16.3)
mastermind_03_08_04-0010 0.0 (-inf) 100.0 (-14.5) 100.0 (-14.5)
mastermind_03_08_04-0011 0.0 (-inf) 0.0 (nan) 100.0 (-9.8)
mastermind_03_08_04-0012 0.0 (-inf) 100.0 (-8.1) 0.0 (nan)
mastermind_03_08_04-0013 0.0 (-inf) 0.0 (nan) 100.0 (-8.6)
mastermind_03_08_04-0014 0.0 (-inf) 100.0 (-8.8) 100.0 (-8.9)
mastermind_03_08_04-0015 0.0 (-inf) 0.0 (nan) 100.0 (-8.1)
mastermind_03_08_05-0000 0.0 (-inf) 99.5 (-2.1) 98.6 (-5.8)
mastermind_03_08_05-0001 0.0 (-inf) 100.0 (-15.5) 100.0 (-15.4)
mastermind_03_08_05-0003 0.0 (-inf) 100.0 (-15.2) 100.0 (-15.2)
mastermind_03_08_05-0004 0.0 (-inf) 100.0 (-14.2) 100.0 (-14.2)
mastermind_03_08_05-0008 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_03_08_05-0009 100.0 (-21.5) 100.0 (-21.5) 100.0 (-21.5)
mastermind_03_08_05-0011 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
mastermind_03_08_05-0012 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
mastermind_03_08_05-0013 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
mastermind_03_08_05-0014 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
mastermind_03_08_05-0015 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
mastermind_04_08_03-0000 100.0 (-0.0) 100.0 (0.0) 100.0 (0.0)
mastermind_04_08_03-0011 100.0 (-4.7) 100.0 (-4.7) 100.0 (-4.7)
mastermind_04_08_03-0012 99.9 (-6.2) 100.0 (-6.0) 100.0 (-6.0)
mastermind_04_08_03-0013 99.9 (-7.4) 100.0 (-7.2) 100.0 (-7.2)
mastermind_04_08_03-0014 99.5 (-7.1) 100.0 (-6.5) 100.0 (-6.5)
mastermind_04_08_03-0015 99.7 (-6.7) 100.0 (-7.1) 100.0 (-7.1)
mastermind_04_08_04-0000 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0001 0.0 (-inf) 0.0 (nan) 100.0 (-11.7)
mastermind_04_08_04-0002 0.0 (-inf) 100.0 (-12.1) 0.0 (nan)
mastermind_04_08_04-0003 0.0 (-inf) 0.0 (nan) 100.0 (-13.6)
mastermind_04_08_04-0004 98.2 (-14.0) 100.0 (-11.2) 100.0 (-11.2)
mastermind_04_08_04-0005 0.0 (-inf) 100.0 (-15.9) 100.0 (-15.9)
mastermind_04_08_04-0011 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0012 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0013 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0014 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_04_08_04-0015 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0000 99.8 (-0.3) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0001 100.0 (-9.8) 100.0 (-9.8) 100.0 (-9.8)
mastermind_05_08_03-0002 100.0 (-12.3) 100.0 (-12.2) 100.0 (-12.2)
mastermind_05_08_03-0003 100.0 (-8.6) 100.0 (-8.6) 100.0 (-8.6)
mastermind_05_08_03-0004 100.0 (-11.7) 100.0 (-11.6) 100.0 (-11.6)
mastermind_05_08_03-0009 100.0 (-8.0) 100.0 (-8.0) 100.0 (-8.0)
mastermind_05_08_03-0011 99.1 (-7.5) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0012 99.7 (-8.0) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0013 99.8 (-7.0) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0014 99.9 (-5.1) 0.0 (nan) 0.0 (nan)
mastermind_05_08_03-0015 98.5 (-8.5) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0000 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0001 100.0 (-12.6) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0002 100.0 (-10.5) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0003 100.0 (-11.2) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0005 100.0 (-11.9) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0009 99.9 (-8.2) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0011 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0012 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0014 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_06_08_03-0015 0.0 (-inf) 0.0 (nan) 0.0 (nan)
mastermind_10_08_03-0001 0.0 (-inf) 100.0 (-13.6) 0.0 (nan)
mastermind_10_08_03-0003 0.0 (-inf) 100.0 (-15.6) 100.0 (-15.5)
mastermind_10_08_03-0004 0.0 (-inf) 0.0 (nan) 100.0 (-16.5)
mastermind_10_08_03-0008 100.0 (-23.2) 100.0 (-23.2) 100.0 (-23.2)
mastermind_10_08_03-0009 100.0 (-17.8) 100.0 (-17.8) 100.0 (-17.8)
mastermind_10_08_03-0010 0.0 (-inf) 100.0 (-10.2) 100.0 (-10.1)
myciel5g_3.wcsp 90.9 (-68.1) 98.3 (-62.5) 99.8 (-61.4)
myciel5g_4.wcsp 96.4 (-7.0) 98.7 (-8.5) 99.1 (-8.3)
or_chain_1.fg 87.0 (-11.6) 0.0 (nan) 0.0 (nan)
or_chain_102.fg 89.7 (-10.6) 0.0 (nan) 0.0 (nan)
or_chain_106.fg 85.0 (-11.1) 0.0 (nan) 0.0 (nan)
or_chain_107.fg 92.4 (-12.1) 0.0 (nan) 0.0 (nan)
or_chain_12.fg 76.5 (-13.9) 95.8 (-12.5) 95.8 (-12.5)
or_chain_128.fg 84.1 (-12.2) 96.3 (-11.2) 0.0 (nan)
or_chain_132.fg 98.1 (-8.5) 100.0 (-8.4) 100.0 (-8.4)
or_chain_138.fg 94.9 (-10.5) 0.0 (nan) 0.0 (nan)
or_chain_140.fg 35.1 (-9.4) 84.8 (-4.6) 89.4 (-6.2)
or_chain_149.fg 98.0 (-8.7) 100.0 (-8.6) 100.0 (-8.6)
or_chain_15.fg 97.6 (-12.0) 0.0 (nan) 99.9 (-11.9)
or_chain_150.fg 75.9 (-8.1) 99.8 (-7.3) 0.0 (nan)
or_chain_153.fg 91.7 (-9.3) 100.0 (-8.7) 100.0 (-8.7)
or_chain_155.fg 95.0 (-9.6) 0.0 (nan) 0.0 (nan)
or_chain_161.fg 91.2 (-8.7) 0.0 (nan) 0.0 (nan)
or_chain_186.fg 91.6 (-9.1) 100.0 (-9.5) 100.0 (-9.5)
or_chain_188.fg 95.6 (-12.6) 100.0 (-12.3) 100.0 (-12.3)
or_chain_198.fg 88.7 (-4.9) 0.0 (nan) 0.0 (nan)
or_chain_209.fg 82.4 (-12.7) 100.0 (-11.5) 0.0 (nan)
or_chain_242.fg 95.5 (-5.7) 0.0 (nan) 0.0 (nan)
or_chain_4.fg 85.7 (-12.7) 0.0 (nan) 0.0 (nan)
or_chain_53.fg 97.7 (-9.4) 99.9 (-9.2) 100.0 (-9.2)
or_chain_61.fg 94.0 (-8.3) 99.7 (-8.0) 99.8 (-8.0)
or_chain_64.fg 94.8 (-9.2) 100.0 (-8.8) 100.0 (-8.8)
or_chain_90.fg 91.0 (-9.1) 95.3 (-8.8) 95.3 (-8.8)
pedigree51 95.3 (-74.0) 95.0 (-73.9) 95.0 (-73.9)
queen5_5_3.wcsp 99.0 (-114.3) 100.0 (-113.0) 100.0 (-113.0)
queen5_5_4.wcsp 94.6 (-48.9) 98.6 (-46.5) 99.6 (-45.9)
rus2_20_40_0_1 99.9 (106.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_0_2 100.0 (120.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_0_3 100.0 (88.2) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_1 100.0 (107.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_2 100.0 (104.5) 0.0 (nan) 0.0 (nan)
rus2_20_40_1_3 99.9 (101.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_1 99.7 (119.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_2 97.0 (116.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_2_3 100.0 (112.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_1 97.5 (108.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_2 99.8 (141.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_3_3 99.8 (123.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_1 100.0 (99.2) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_2 99.9 (107.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_4_3 99.9 (95.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_1 99.9 (95.5) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_2 99.8 (88.2) 0.0 (nan) 0.0 (nan)
rus2_20_40_5_3 99.9 (169.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_1 100.0 (119.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_2 99.8 (68.5) 0.0 (nan) 0.0 (nan)
rus2_20_40_6_3 99.9 (134.1) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_1 99.9 (88.5) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_2 100.0 (111.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_7_3 99.9 (81.4) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_1 100.0 (106.9) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_2 99.9 (99.8) 0.0 (nan) 0.0 (nan)
rus2_20_40_8_3 99.9 (100.5) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_1 100.0 (94.6) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_2 99.1 (95.3) 0.0 (nan) 0.0 (nan)
rus2_20_40_9_3 100.0 (63.6) 0.0 (nan) 0.0 (nan)
rus_20_40_0_1 100.0 (617.3) 0.0 (nan) 0.0 (nan)
rus_20_40_0_2 100.0 (791.6) 0.0 (nan) 0.0 (nan)
rus_20_40_0_3 100.0 (903.0) 0.0 (nan) 0.0 (nan)
rus_20_40_1_1 100.0 (1004.6) 0.0 (nan) 0.0 (nan)
rus_20_40_1_2 100.0 (910.1) 0.0 (nan) 0.0 (nan)
rus_20_40_1_3 95.3 (891.4) 0.0 (nan) 0.0 (nan)
rus_20_40_2_1 100.0 (797.8) 0.0 (nan) 0.0 (nan)
rus_20_40_2_2 97.2 (683.7) 0.0 (nan) 0.0 (nan)
rus_20_40_2_3 95.4 (739.8) 0.0 (nan) 0.0 (nan)
rus_20_40_3_1 99.8 (825.5) 0.0 (nan) 0.0 (nan)
rus_20_40_3_2 100.0 (839.0) 0.0 (nan) 0.0 (nan)
rus_20_40_3_3 100.0 (841.9) 0.0 (nan) 0.0 (nan)
rus_20_40_4_1 95.2 (922.7) 0.0 (nan) 0.0 (nan)
rus_20_40_4_2 99.3 (934.4) 0.0 (nan) 0.0 (nan)
rus_20_40_4_3 100.0 (853.3) 0.0 (nan) 0.0 (nan)
rus_20_40_5_1 100.0 (875.9) 0.0 (nan) 0.0 (nan)
rus_20_40_5_2 98.7 (837.1) 0.0 (nan) 0.0 (nan)
rus_20_40_5_3 98.4 (878.7) 0.0 (nan) 0.0 (nan)
rus_20_40_6_1 95.1 (713.5) 0.0 (nan) 0.0 (nan)
rus_20_40_6_2 100.0 (982.4) 0.0 (nan) 0.0 (nan)
rus_20_40_6_3 96.3 (959.9) 0.0 (nan) 0.0 (nan)
rus_20_40_7_1 100.0 (752.1) 0.0 (nan) 0.0 (nan)
rus_20_40_7_2 100.0 (756.2) 0.0 (nan) 0.0 (nan)
rus_20_40_7_3 100.0 (869.7) 0.0 (nan) 0.0 (nan)
rus_20_40_8_1 99.5 (725.8) 0.0 (nan) 0.0 (nan)
rus_20_40_8_2 100.0 (898.6) 0.0 (nan) 0.0 (nan)
rus_20_40_8_3 98.9 (701.8) 0.0 (nan) 0.0 (nan)
rus_20_40_9_1 90.5 (849.4) 0.0 (nan) 0.0 (nan)
rus_20_40_9_2 93.9 (821.6) 0.0 (nan) 0.0 (nan)
rus_20_40_9_3 100.0 (864.0) 0.0 (nan) 0.0 (nan)

Updated: