AbstractionSampling Scores
The results below are organized as follows:
- each table displays the solver’s performance for individual problem instances for the given task under different time limits
- table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria
PR
overall
| Problem | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| 1a7w | 99.7 (-1.0) | 0.0 (nan) | 0.0 (nan) |
| 1aie | 98.1 (2.8) | 0.0 (nan) | 0.0 (nan) |
| 1b2v | 99.9 (-53.4) | 0.0 (nan) | 100.0 (-53.4) |
| 1bxv | 99.9 (-9.7) | 100.0 (-9.7) | 0.0 (nan) |
| 1cpq | 99.8 (-21.7) | 100.0 (-21.7) | 0.0 (nan) |
| 1ctj | 99.8 (-13.3) | 0.0 (nan) | 100.0 (-13.3) |
| 1cxy | 100.0 (-38.0) | 0.0 (nan) | 0.0 (nan) |
| 1f94 | 100.0 (-61.4) | 0.0 (nan) | 100.0 (-61.4) |
| 1fas | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| 1g6x | 98.9 (-27.3) | 100.0 (-27.2) | 100.0 (-27.2) |
| 1jer | 100.0 (-31.5) | 100.0 (-31.5) | 100.0 (-31.5) |
| 1kp6 | 100.0 (-45.1) | 0.0 (nan) | 0.0 (nan) |
| 1nkd | 99.9 (-8.8) | 0.0 (nan) | 0.0 (nan) |
| 1npl | 100.0 (-42.9) | 100.0 (-42.9) | 100.0 (-42.9) |
| 1qt9 | 0.0 (nan) | 100.0 (-59.7) | 100.0 (-59.7) |
| 1rfs | 100.0 (-50.6) | 0.0 (nan) | 100.0 (-50.6) |
| 1svy | 100.0 (-38.6) | 100.0 (-38.6) | 100.0 (-38.6) |
| 1vfy | 100.0 (-20.1) | 0.0 (nan) | 100.0 (-20.1) |
| 2rta | 100.0 (-55.2) | 100.0 (-55.2) | 100.0 (-55.2) |
| BN_12 | 100.0 (-3.6) | 100.0 (-3.6) | 100.0 (-3.6) |
| BN_13 | 100.0 (-2.3) | 100.0 (-2.3) | 100.0 (-2.3) |
| BN_15 | 100.0 (-5.7) | 100.0 (-5.7) | 100.0 (-5.7) |
| BN_31 | 98.6 (-12.8) | 100.0 (-11.0) | 100.0 (-11.0) |
| BN_74 | 98.4 (-2.5) | 100.0 (-0.0) | 0.0 (nan) |
| blockmap_10_01-0006 | 0.0 (-inf) | 99.9 (-1.0) | 100.0 (-1.0) |
| blockmap_10_01-0009 | 99.8 (-1.1) | 100.0 (-1.0) | 100.0 (-1.0) |
| blockmap_10_02-0009 | 99.9 (-2.1) | 100.0 (-2.2) | 100.0 (-2.2) |
| blockmap_10_03-0009 | 100.0 (-3.2) | 100.0 (-3.2) | 100.0 (-3.2) |
| blockmap_10_03-0010 | 100.0 (-2.9) | 100.0 (-2.9) | 100.0 (-2.9) |
| blockmap_15_01-0008 | 100.0 (-1.2) | 100.0 (-1.2) | 100.0 (-1.2) |
| blockmap_15_02-0008 | 100.0 (-2.4) | 100.0 (-2.4) | 100.0 (-2.4) |
| blockmap_15_03-0008 | 0.0 (-inf) | 100.0 (-3.2) | 100.0 (-3.2) |
| blockmap_15_03-0010 | 100.0 (-3.1) | 100.0 (-3.1) | 100.0 (-3.1) |
| blockmap_20_01-0009 | 100.0 (-1.3) | 100.0 (-1.3) | 100.0 (-1.3) |
| blockmap_20_02-0009 | 0.0 (nan) | 100.0 (-2.6) | 100.0 (-2.6) |
| blockmap_20_03-0010 | 0.0 (nan) | 100.0 (-4.0) | 100.0 (-4.0) |
| blockmap_22_01-0006 | 0.0 (nan) | 100.0 (-1.3) | 100.0 (-1.3) |
| blockmap_22_03-0009 | 100.0 (-4.1) | 0.0 (nan) | 100.0 (-4.1) |
| bwt3ac.wcsp | 98.8 (0.2) | 100.0 (0.0) | 100.0 (0.0) |
| fs-07 | 100.0 (-14.8) | 100.0 (-14.8) | 100.0 (-14.8) |
| mastermind_03_08_04-0000 | 0.0 (-inf) | 100.0 (0.0) | 100.0 (0.0) |
| mastermind_03_08_04-0001 | 99.8 (-12.8) | 100.0 (-12.7) | 100.0 (-12.7) |
| mastermind_03_08_04-0002 | 99.9 (-14.4) | 100.0 (-14.5) | 100.0 (-14.5) |
| mastermind_03_08_04-0003 | 100.0 (-11.5) | 100.0 (-11.5) | 100.0 (-11.5) |
| mastermind_03_08_04-0007 | 100.0 (-16.6) | 100.0 (-16.6) | 100.0 (-16.6) |
| mastermind_03_08_04-0008 | 100.0 (-16.3) | 100.0 (-16.3) | 100.0 (-16.3) |
| mastermind_03_08_04-0010 | 99.7 (-15.0) | 100.0 (-14.5) | 100.0 (-14.5) |
| mastermind_03_08_04-0011 | 0.0 (-inf) | 99.8 (-10.2) | 100.0 (-9.8) |
| mastermind_03_08_04-0012 | 0.0 (-inf) | 100.0 (-8.2) | 100.0 (-8.1) |
| mastermind_03_08_04-0013 | 0.0 (-inf) | 100.0 (-8.5) | 100.0 (-8.5) |
| mastermind_03_08_04-0014 | 0.0 (-inf) | 100.0 (-8.8) | 100.0 (-9.0) |
| mastermind_03_08_04-0015 | 0.0 (-inf) | 100.0 (-8.0) | 99.7 (-8.8) |
| mastermind_03_08_05-0000 | 0.0 (-inf) | 0.0 (-inf) | 99.1 (-3.7) |
| mastermind_03_08_05-0001 | 0.0 (-inf) | 100.0 (-15.5) | 100.0 (-15.4) |
| mastermind_03_08_05-0003 | 0.0 (-inf) | 100.0 (-15.1) | 100.0 (-15.2) |
| mastermind_03_08_05-0004 | 0.0 (-inf) | 100.0 (-14.3) | 100.0 (-14.2) |
| mastermind_03_08_05-0008 | 0.0 (-inf) | 99.9 (-14.8) | 100.0 (-14.6) |
| mastermind_03_08_05-0009 | 100.0 (-21.5) | 100.0 (-21.5) | 100.0 (-21.5) |
| mastermind_03_08_05-0011 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_03_08_05-0012 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_03_08_05-0013 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_03_08_05-0014 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_03_08_05-0015 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_04_08_03-0000 | 100.0 (0.0) | 100.0 (0.0) | 100.0 (0.0) |
| mastermind_04_08_03-0011 | 99.6 (-5.2) | 100.0 (-4.7) | 100.0 (-4.7) |
| mastermind_04_08_03-0012 | 100.0 (-6.0) | 100.0 (-6.0) | 100.0 (-6.0) |
| mastermind_04_08_03-0013 | 99.9 (-7.1) | 100.0 (-7.2) | 100.0 (-7.2) |
| mastermind_04_08_03-0014 | 99.9 (-6.6) | 100.0 (-6.5) | 100.0 (-6.5) |
| mastermind_04_08_03-0015 | 100.0 (-7.1) | 100.0 (-7.1) | 100.0 (-7.1) |
| mastermind_04_08_04-0000 | 0.0 (-inf) | 100.0 (0.0) | 100.0 (-0.0) |
| mastermind_04_08_04-0001 | 0.0 (-inf) | 100.0 (-11.7) | 100.0 (-11.7) |
| mastermind_04_08_04-0002 | 97.9 (-15.1) | 100.0 (-12.1) | 100.0 (-12.1) |
| mastermind_04_08_04-0003 | 0.0 (-inf) | 100.0 (-13.6) | 100.0 (-13.6) |
| mastermind_04_08_04-0004 | 0.0 (-inf) | 100.0 (-11.2) | 100.0 (-11.2) |
| mastermind_04_08_04-0005 | 0.0 (-inf) | 100.0 (-16.0) | 100.0 (-15.9) |
| mastermind_04_08_04-0011 | 0.0 (-inf) | 99.7 (-9.1) | 99.2 (-10.3) |
| mastermind_04_08_04-0012 | 0.0 (-inf) | 99.9 (-8.4) | 0.0 (-inf) |
| mastermind_04_08_04-0013 | 0.0 (-inf) | 99.4 (-8.6) | 99.8 (-7.5) |
| mastermind_04_08_04-0014 | 0.0 (-inf) | 100.0 (-8.7) | 99.4 (-10.5) |
| mastermind_04_08_04-0015 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| mastermind_05_08_03-0000 | 99.8 (-0.3) | 100.0 (0.0) | 100.0 (0.0) |
| mastermind_05_08_03-0001 | 100.0 (-9.8) | 100.0 (-9.8) | 100.0 (-9.8) |
| mastermind_05_08_03-0002 | 100.0 (-12.2) | 100.0 (-12.2) | 100.0 (-12.2) |
| mastermind_05_08_03-0003 | 100.0 (-8.6) | 100.0 (-8.6) | 100.0 (-8.6) |
| mastermind_05_08_03-0004 | 100.0 (-11.7) | 100.0 (-11.6) | 100.0 (-11.6) |
| mastermind_05_08_03-0009 | 100.0 (-7.9) | 100.0 (-8.0) | 100.0 (-8.0) |
| mastermind_05_08_03-0011 | 99.8 (-6.5) | 100.0 (-6.2) | 100.0 (-6.2) |
| mastermind_05_08_03-0012 | 99.9 (-7.5) | 100.0 (-7.6) | 100.0 (-7.6) |
| mastermind_05_08_03-0013 | 99.2 (-7.8) | 100.0 (-6.6) | 100.0 (-6.6) |
| mastermind_05_08_03-0014 | 99.5 (-5.6) | 100.0 (-4.9) | 100.0 (-4.9) |
| mastermind_05_08_03-0015 | 99.0 (-7.7) | 100.0 (-6.3) | 100.0 (-6.3) |
| mastermind_06_08_03-0000 | 0.0 (-inf) | 100.0 (-0.0) | 100.0 (0.0) |
| mastermind_06_08_03-0001 | 100.0 (-12.6) | 100.0 (-12.6) | 100.0 (-12.6) |
| mastermind_06_08_03-0002 | 100.0 (-10.5) | 100.0 (-10.5) | 100.0 (-10.5) |
| mastermind_06_08_03-0003 | 100.0 (-11.2) | 100.0 (-11.2) | 100.0 (-11.2) |
| mastermind_06_08_03-0005 | 100.0 (-11.9) | 100.0 (-11.9) | 100.0 (-11.9) |
| mastermind_06_08_03-0009 | 100.0 (-8.4) | 100.0 (-8.4) | 100.0 (-8.4) |
| mastermind_06_08_03-0011 | 98.4 (-7.1) | 100.0 (-4.7) | 100.0 (-4.7) |
| mastermind_06_08_03-0012 | 0.0 (-inf) | 100.0 (-5.9) | 100.0 (-5.9) |
| mastermind_06_08_03-0014 | 0.0 (-inf) | 100.0 (-6.3) | 100.0 (-6.3) |
| mastermind_06_08_03-0015 | 0.0 (-inf) | 100.0 (-7.8) | 100.0 (-7.8) |
| mastermind_10_08_03-0001 | 0.0 (-inf) | 100.0 (-13.6) | 100.0 (-13.6) |
| mastermind_10_08_03-0003 | 0.0 (-inf) | 100.0 (-15.5) | 100.0 (-15.5) |
| mastermind_10_08_03-0004 | 0.0 (-inf) | 100.0 (-16.5) | 100.0 (-16.5) |
| mastermind_10_08_03-0008 | 100.0 (-23.2) | 100.0 (-23.2) | 100.0 (-23.2) |
| mastermind_10_08_03-0009 | 100.0 (-17.8) | 100.0 (-17.8) | 100.0 (-17.8) |
| mastermind_10_08_03-0010 | 0.0 (-inf) | 100.0 (-10.1) | 100.0 (-10.1) |
| myciel5g_3.wcsp | 94.8 (-65.1) | 99.5 (-61.6) | 99.4 (-61.7) |
| myciel5g_4.wcsp | 96.0 (-9.3) | 98.3 (-8.6) | 99.9 (-8.1) |
| or_chain_1.fg | 97.4 (-10.9) | 90.3 (-11.4) | 94.8 (-11.1) |
| or_chain_102.fg | 88.7 (-10.6) | 94.0 (-10.4) | 94.1 (-10.3) |
| or_chain_106.fg | 90.0 (-10.7) | 96.1 (-10.3) | 95.3 (-10.4) |
| or_chain_107.fg | 89.6 (-12.3) | 99.4 (-11.5) | 99.9 (-11.6) |
| or_chain_12.fg | 77.9 (-13.8) | 95.8 (-12.5) | 95.8 (-12.5) |
| or_chain_128.fg | 89.4 (-11.8) | 96.3 (-11.2) | 96.3 (-11.2) |
| or_chain_132.fg | 98.9 (-8.4) | 99.9 (-8.4) | 100.0 (-8.4) |
| or_chain_138.fg | 96.8 (-10.3) | 100.0 (-10.1) | 100.0 (-10.1) |
| or_chain_140.fg | 35.4 (-9.4) | 96.8 (-5.7) | 96.8 (-5.7) |
| or_chain_149.fg | 96.1 (-8.3) | 100.0 (-8.6) | 100.0 (-8.6) |
| or_chain_15.fg | 91.0 (-12.5) | 100.0 (-11.8) | 100.0 (-11.8) |
| or_chain_150.fg | 72.6 (-8.3) | 99.8 (-7.3) | 99.8 (-7.2) |
| or_chain_153.fg | 81.5 (-10.0) | 100.0 (-8.7) | 100.0 (-8.7) |
| or_chain_155.fg | 97.8 (-10.1) | 100.0 (-9.9) | 100.0 (-9.9) |
| or_chain_161.fg | 93.3 (-9.6) | 99.6 (-9.2) | 100.0 (-9.2) |
| or_chain_186.fg | 94.1 (-9.7) | 100.0 (-9.5) | 100.0 (-9.5) |
| or_chain_188.fg | 94.1 (-12.7) | 100.0 (-12.3) | 100.0 (-12.3) |
| or_chain_198.fg | 98.2 (-4.4) | 100.0 (-4.3) | 100.0 (-4.3) |
| or_chain_209.fg | 96.2 (-11.7) | 99.9 (-11.5) | 100.0 (-11.5) |
| or_chain_242.fg | 94.6 (-5.8) | 100.0 (-5.4) | 99.8 (-5.3) |
| or_chain_4.fg | 89.2 (-12.5) | 99.9 (-11.7) | 100.0 (-11.7) |
| or_chain_53.fg | 98.9 (-9.3) | 99.9 (-9.2) | 100.0 (-9.2) |
| or_chain_61.fg | 99.4 (-8.0) | 99.8 (-8.0) | 99.8 (-8.0) |
| or_chain_64.fg | 91.7 (-9.4) | 100.0 (-8.8) | 100.0 (-8.8) |
| or_chain_90.fg | 88.4 (-9.3) | 95.3 (-8.8) | 95.3 (-8.8) |
| pedigree51 | 94.9 (-73.8) | 95.0 (-73.9) | 95.0 (-73.9) |
| queen5_5_3.wcsp | 97.9 (-115.6) | 100.0 (-113.0) | 99.9 (-112.9) |
| queen5_5_4.wcsp | 99.1 (-46.3) | 99.3 (-46.1) | 99.4 (-46.1) |
| rus2_20_40_0_1 | 100.0 (106.9) | 100.0 (106.9) | 99.9 (106.9) |
| rus2_20_40_0_2 | 99.9 (120.6) | 100.0 (120.6) | 99.9 (120.6) |
| rus2_20_40_0_3 | 100.0 (88.2) | 100.0 (88.2) | 100.0 (88.2) |
| rus2_20_40_1_1 | 100.0 (107.4) | 100.0 (107.4) | 100.0 (107.4) |
| rus2_20_40_1_2 | 100.0 (104.5) | 100.0 (104.5) | 100.0 (104.5) |
| rus2_20_40_1_3 | 99.8 (101.5) | 100.0 (101.6) | 100.0 (101.6) |
| rus2_20_40_2_1 | 99.0 (118.7) | 99.6 (119.1) | 98.4 (118.4) |
| rus2_20_40_2_2 | 97.3 (117.0) | 98.3 (117.6) | 96.7 (116.7) |
| rus2_20_40_2_3 | 99.9 (112.4) | 99.9 (112.5) | 99.9 (112.4) |
| rus2_20_40_3_1 | 99.0 (109.5) | 99.7 (110.0) | 99.7 (110.0) |
| rus2_20_40_3_2 | 99.5 (141.5) | 99.6 (141.4) | 100.0 (141.2) |
| rus2_20_40_3_3 | 99.9 (123.6) | 99.6 (123.3) | 99.7 (123.3) |
| rus2_20_40_4_1 | 99.8 (99.3) | 100.0 (99.2) | 100.0 (99.2) |
| rus2_20_40_4_2 | 100.0 (107.1) | 100.0 (107.0) | 100.0 (107.1) |
| rus2_20_40_4_3 | 99.8 (95.3) | 100.0 (95.2) | 100.0 (95.2) |
| rus2_20_40_5_1 | 100.0 (95.4) | 100.0 (95.4) | 100.0 (95.4) |
| rus2_20_40_5_2 | 99.9 (88.3) | 99.5 (88.0) | 100.0 (88.3) |
| rus2_20_40_5_3 | 100.0 (169.9) | 99.9 (170.0) | 100.0 (169.9) |
| rus2_20_40_6_1 | 99.9 (119.1) | 99.8 (119.3) | 100.0 (119.1) |
| rus2_20_40_6_2 | 99.7 (68.2) | 100.0 (68.3) | 100.0 (68.3) |
| rus2_20_40_6_3 | 100.0 (134.1) | 99.9 (134.2) | 100.0 (134.1) |
| rus2_20_40_7_1 | 99.9 (88.4) | 99.7 (88.6) | 100.0 (88.5) |
| rus2_20_40_7_2 | 99.8 (111.6) | 100.0 (111.5) | 100.0 (111.6) |
| rus2_20_40_7_3 | 100.0 (81.4) | 100.0 (81.4) | 100.0 (81.4) |
| rus2_20_40_8_1 | 100.0 (106.9) | 100.0 (106.9) | 100.0 (106.9) |
| rus2_20_40_8_2 | 99.9 (99.9) | 100.0 (99.8) | 99.9 (99.8) |
| rus2_20_40_8_3 | 100.0 (100.6) | 99.9 (100.5) | 99.9 (100.6) |
| rus2_20_40_9_1 | 100.0 (94.5) | 100.0 (94.5) | 100.0 (94.5) |
| rus2_20_40_9_2 | 99.3 (95.1) | 100.0 (94.6) | 100.0 (94.6) |
| rus2_20_40_9_3 | 99.7 (63.4) | 99.9 (63.5) | 99.9 (63.7) |
| rus_20_40_0_1 | 100.0 (617.3) | 100.0 (617.3) | 100.0 (617.3) |
| rus_20_40_0_2 | 100.0 (791.6) | 100.0 (791.6) | 100.0 (791.6) |
| rus_20_40_0_3 | 100.0 (903.0) | 100.0 (903.0) | 100.0 (903.0) |
| rus_20_40_1_1 | 100.0 (1004.6) | 97.3 (999.8) | 100.0 (1004.6) |
| rus_20_40_1_2 | 100.0 (910.1) | 97.3 (905.2) | 99.6 (910.9) |
| rus_20_40_1_3 | 95.3 (891.4) | 95.7 (892.1) | 95.5 (891.8) |
| rus_20_40_2_1 | 100.0 (797.8) | 97.4 (792.8) | 97.4 (792.8) |
| rus_20_40_2_2 | 97.2 (683.7) | 97.2 (683.7) | 97.3 (684.1) |
| rus_20_40_2_3 | 95.4 (739.8) | 93.8 (736.3) | 93.6 (735.8) |
| rus_20_40_3_1 | 99.8 (825.5) | 95.5 (817.1) | 99.6 (825.1) |
| rus_20_40_3_2 | 100.0 (839.0) | 100.0 (839.0) | 100.0 (839.0) |
| rus_20_40_3_3 | 100.0 (841.9) | 99.8 (842.3) | 99.9 (841.7) |
| rus_20_40_4_1 | 95.2 (922.7) | 99.6 (930.8) | 99.6 (930.8) |
| rus_20_40_4_2 | 99.3 (934.4) | 99.4 (934.7) | 96.6 (929.4) |
| rus_20_40_4_3 | 100.0 (853.3) | 99.6 (854.0) | 95.0 (843.2) |
| rus_20_40_5_1 | 100.0 (875.9) | 98.5 (873.2) | 99.7 (875.4) |
| rus_20_40_5_2 | 99.9 (839.4) | 98.2 (836.2) | 97.7 (835.3) |
| rus_20_40_5_3 | 98.5 (878.8) | 97.0 (875.7) | 98.4 (878.6) |
| rus_20_40_6_1 | 95.1 (713.5) | 99.9 (723.6) | 100.0 (723.8) |
| rus_20_40_6_2 | 100.0 (982.4) | 97.3 (977.4) | 97.3 (977.4) |
| rus_20_40_6_3 | 96.3 (959.9) | 94.6 (956.7) | 94.6 (956.7) |
| rus_20_40_7_1 | 100.0 (752.1) | 100.0 (752.1) | 100.0 (752.1) |
| rus_20_40_7_2 | 100.0 (756.2) | 98.6 (753.4) | 98.6 (753.4) |
| rus_20_40_7_3 | 100.0 (869.7) | 97.5 (864.9) | 97.5 (864.9) |
| rus_20_40_8_1 | 99.5 (725.8) | 97.5 (721.8) | 97.5 (721.8) |
| rus_20_40_8_2 | 100.0 (898.6) | 100.0 (898.7) | 97.3 (893.7) |
| rus_20_40_8_3 | 98.9 (701.8) | 99.2 (702.2) | 96.3 (696.8) |
| rus_20_40_9_1 | 90.5 (849.4) | 100.0 (866.6) | 100.0 (866.6) |
| rus_20_40_9_2 | 93.9 (821.6) | 100.0 (834.5) | 100.0 (834.5) |
| rus_20_40_9_3 | 100.0 (863.9) | 99.9 (864.3) | 97.3 (859.0) |