Abstraction-Sampling Scores
The results below are organized as follows:
- each table displays the solver’s performance for individual problem instances for the given task under different time limits
- table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria
PR
overall
| Problem | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| Alchemy_11 | 99.8 (606.3) | 100.0 (606.3) | 100.0 (606.3) |
| CSP_11 | 100.0 (13.6) | 100.0 (13.6) | 100.0 (13.6) |
| CSP_12 | 100.0 (16.5) | 100.0 (16.5) | 100.0 (16.5) |
| CSP_13 | 100.0 (15.3) | 100.0 (15.3) | 100.0 (15.3) |
| DBN_11 | 99.3 (57.9) | 100.0 (58.5) | 100.0 (58.5) |
| DBN_12 | 100.0 (63.1) | 100.0 (63.1) | 100.0 (63.1) |
| DBN_13 | 99.9 (66.4) | 100.0 (66.6) | 100.0 (66.6) |
| DBN_14 | 100.0 (151.2) | 100.0 (151.2) | 100.0 (151.2) |
| DBN_15 | 100.0 (152.6) | 100.0 (152.6) | 100.0 (152.6) |
| DBN_16 | 100.0 (166.1) | 100.0 (166.1) | 100.0 (166.1) |
| Grids_11 | 99.9 (169.4) | 100.0 (169.4) | 100.0 (169.4) |
| Grids_12 | 100.0 (303.1) | 100.0 (303.1) | 100.0 (303.1) |
| Grids_13 | 99.4 (332.8) | 100.0 (333.3) | 100.0 (333.3) |
| Grids_14 | 99.9 (497.6) | 100.0 (497.8) | 100.0 (497.8) |
| Grids_15 | 100.0 (291.7) | 100.0 (291.7) | 100.0 (291.7) |
| Grids_16 | 99.2 (663.8) | 100.0 (665.1) | 100.0 (665.1) |
| Grids_17 | 97.7 (1304.6) | 100.0 (1312.0) | 100.0 (1312.0) |
| Grids_18 | 97.2 (1949.5) | 100.0 (1963.0) | 100.0 (1963.0) |
| ObjectDetection_11 | 100.0 (-74.9) | 100.0 (-74.9) | 100.0 (-74.9) |
| ObjectDetection_12 | 99.8 (-67.4) | 100.0 (-67.4) | 100.0 (-67.4) |
| ObjectDetection_13 | 99.9 (-70.4) | 100.0 (-70.4) | 100.0 (-70.4) |
| ObjectDetection_14 | 100.0 (-64.1) | 100.0 (-64.1) | 100.0 (-64.1) |
| ObjectDetection_15 | 100.0 (-51.9) | 100.0 (-51.9) | 100.0 (-51.9) |
| ObjectDetection_16 | 100.0 (-34.4) | 100.0 (-34.4) | 100.0 (-34.4) |
| ObjectDetection_17 | 100.0 (-29.8) | 100.0 (-29.8) | 100.0 (-29.8) |
| ObjectDetection_18 | 99.9 (-37.1) | 100.0 (-37.0) | 100.0 (-37.0) |
| ObjectDetection_19 | 99.4 (-38.4) | 100.0 (-38.5) | 100.0 (-38.5) |
| ObjectDetection_20 | 99.9 (-34.8) | 100.0 (-34.8) | 100.0 (-34.8) |
| ObjectDetection_21 | 100.0 (-24.4) | 100.0 (-24.4) | 100.0 (-24.4) |
| ObjectDetection_22 | 100.0 (-36.7) | 100.0 (-36.7) | 100.0 (-36.7) |
| ObjectDetection_23 | 100.0 (-31.6) | 100.0 (-31.6) | 100.0 (-31.6) |
| ObjectDetection_24 | 100.0 (-26.4) | 100.0 (-26.4) | 100.0 (-26.4) |
| ObjectDetection_25 | 100.0 (-18.7) | 100.0 (-18.7) | 100.0 (-18.7) |
| ObjectDetection_26 | 99.9 (-33.8) | 100.0 (-33.8) | 100.0 (-33.8) |
| ObjectDetection_27 | 99.8 (-29.9) | 100.0 (-29.9) | 100.0 (-29.9) |
| ObjectDetection_28 | 100.0 (-46.8) | 100.0 (-46.8) | 100.0 (-46.8) |
| ObjectDetection_29 | 100.0 (-50.8) | 100.0 (-50.8) | 100.0 (-50.8) |
| ObjectDetection_30 | 99.9 (-52.4) | 100.0 (-52.4) | 100.0 (-52.4) |
| ObjectDetection_31 | 100.0 (-44.1) | 100.0 (-44.1) | 100.0 (-44.1) |
| ObjectDetection_32 | 99.9 (-46.7) | 100.0 (-46.7) | 100.0 (-46.7) |
| ObjectDetection_33 | 99.9 (-44.1) | 100.0 (-44.1) | 100.0 (-44.1) |
| ObjectDetection_34 | 100.0 (-35.4) | 100.0 (-35.4) | 100.0 (-35.4) |
| ObjectDetection_35 | 100.0 (-37.6) | 100.0 (-37.6) | 100.0 (-37.6) |
| ObjectDetection_36 | 99.8 (-30.7) | 100.0 (-30.7) | 100.0 (-30.7) |
| ObjectDetection_37 | 99.8 (-30.0) | 100.0 (-30.0) | 100.0 (-30.0) |
| ObjectDetection_38 | 100.0 (-75.1) | 100.0 (-75.1) | 100.0 (-75.1) |
| ObjectDetection_39 | 99.9 (-70.2) | 100.0 (-70.2) | 100.0 (-70.2) |
| ObjectDetection_40 | 99.9 (-72.3) | 100.0 (-72.3) | 100.0 (-72.3) |
| ObjectDetection_41 | 99.9 (-71.1) | 100.0 (-71.1) | 100.0 (-71.1) |
| ObjectDetection_42 | 100.0 (-45.2) | 100.0 (-45.2) | 100.0 (-45.2) |
| ObjectDetection_43 | 100.0 (-39.5) | 100.0 (-39.5) | 100.0 (-39.5) |
| ObjectDetection_44 | 100.0 (-41.0) | 100.0 (-41.0) | 100.0 (-41.0) |
| ObjectDetection_45 | 99.9 (-38.9) | 100.0 (-38.9) | 100.0 (-38.9) |
| ObjectDetection_46 | 100.0 (-42.0) | 100.0 (-42.0) | 100.0 (-42.0) |
| ObjectDetection_47 | 100.0 (-44.7) | 100.0 (-44.7) | 100.0 (-44.7) |
| ObjectDetection_48 | 99.9 (-36.6) | 100.0 (-36.6) | 100.0 (-36.6) |
| ObjectDetection_49 | 99.9 (-38.7) | 100.0 (-38.7) | 100.0 (-38.7) |
| ObjectDetection_50 | 100.0 (-36.1) | 100.0 (-36.1) | 100.0 (-36.1) |
| ObjectDetection_51 | 100.0 (-53.4) | 100.0 (-53.4) | 100.0 (-53.4) |
| ObjectDetection_52 | 100.0 (-57.0) | 100.0 (-57.0) | 100.0 (-57.0) |
| ObjectDetection_53 | 99.9 (-47.7) | 100.0 (-47.7) | 100.0 (-47.7) |
| ObjectDetection_54 | 100.0 (-41.5) | 100.0 (-41.5) | 100.0 (-41.5) |
| ObjectDetection_55 | 100.0 (-34.1) | 100.0 (-34.1) | 100.0 (-34.1) |
| ObjectDetection_56 | 99.9 (-36.3) | 100.0 (-36.3) | 100.0 (-36.3) |
| ObjectDetection_57 | 100.0 (-33.4) | 100.0 (-33.4) | 100.0 (-33.4) |
| ObjectDetection_58 | 100.0 (-41.7) | 100.0 (-41.7) | 100.0 (-41.7) |
| ObjectDetection_59 | 99.9 (-35.9) | 100.0 (-35.9) | 100.0 (-35.9) |
| ObjectDetection_60 | 99.9 (-35.3) | 100.0 (-35.3) | 100.0 (-35.3) |
| ObjectDetection_61 | 100.0 (-23.2) | 100.0 (-23.2) | 100.0 (-23.2) |
| ObjectDetection_62 | 100.0 (-24.8) | 100.0 (-24.8) | 100.0 (-24.8) |
| ObjectDetection_63 | 99.9 (-19.9) | 100.0 (-19.9) | 100.0 (-19.9) |
| ObjectDetection_64 | 100.0 (-21.3) | 100.0 (-21.3) | 100.0 (-21.3) |
| ObjectDetection_65 | 100.0 (-19.7) | 100.0 (-19.7) | 100.0 (-19.7) |
| ObjectDetection_66 | 100.0 (-47.0) | 100.0 (-47.0) | 100.0 (-47.0) |
| ObjectDetection_67 | 100.0 (-40.6) | 100.0 (-40.6) | 100.0 (-40.6) |
| ObjectDetection_68 | 99.9 (-42.7) | 100.0 (-42.7) | 100.0 (-42.7) |
| ObjectDetection_69 | 99.9 (-40.3) | 100.0 (-40.3) | 100.0 (-40.3) |
| ObjectDetection_70 | 100.0 (-31.2) | 100.0 (-31.2) | 100.0 (-31.2) |
| ObjectDetection_71 | 99.9 (-33.6) | 100.0 (-33.6) | 100.0 (-33.6) |
| ObjectDetection_72 | 99.6 (-26.4) | 100.0 (-26.4) | 100.0 (-26.4) |
| ObjectDetection_73 | 99.8 (-25.6) | 100.0 (-25.6) | 100.0 (-25.6) |
| ObjectDetection_74 | 100.0 (-31.8) | 100.0 (-31.8) | 100.0 (-31.8) |
| ObjectDetection_75 | 100.0 (-33.8) | 100.0 (-33.8) | 100.0 (-33.8) |
| Pedigree_11 | 100.0 (-17.2) | 100.0 (-17.2) | 100.0 (-17.2) |
| Pedigree_12 | 100.0 (-11.5) | 100.0 (-11.5) | 100.0 (-11.5) |
| Pedigree_13 | 100.0 (-15.2) | 100.0 (-15.2) | 100.0 (-15.2) |
| Promedus_11 | 100.0 (-8.4) | 100.0 (-8.4) | 100.0 (-8.4) |
| Promedus_12 | 97.9 (-3.2) | 100.0 (-3.2) | 100.0 (-3.2) |
| Promedus_13 | 100.0 (-4.5) | 100.0 (-4.5) | 100.0 (-4.5) |
| Promedus_14 | 99.4 (-7.8) | 100.0 (-7.8) | 100.0 (-7.8) |
| Promedus_15 | 100.0 (-3.6) | 100.0 (-3.6) | 100.0 (-3.6) |
| Promedus_16 | 100.0 (-7.0) | 100.0 (-7.0) | 100.0 (-7.0) |
| Promedus_17 | 99.6 (-9.4) | 100.0 (-9.5) | 100.0 (-9.5) |
| Promedus_18 | 99.8 (-4.7) | 100.0 (-4.7) | 100.0 (-4.7) |
| Promedus_19 | 99.4 (-4.3) | 100.0 (-4.3) | 100.0 (-4.3) |
| Promedus_20 | 91.7 (-7.4) | 100.0 (-7.1) | 100.0 (-7.1) |
| Promedus_21 | 100.0 (-5.6) | 100.0 (-5.6) | 100.0 (-5.6) |
| Promedus_22 | 100.0 (-2.5) | 100.0 (-2.5) | 100.0 (-2.5) |
| Promedus_23 | 99.9 (-11.3) | 100.0 (-11.3) | 100.0 (-11.3) |
| Promedus_24 | 100.0 (-5.9) | 100.0 (-5.9) | 100.0 (-5.9) |
| Promedus_25 | 98.6 (-9.4) | 100.0 (-9.4) | 100.0 (-9.4) |
| Promedus_26 | 100.0 (-7.3) | 100.0 (-7.3) | 100.0 (-7.3) |
| Promedus_27 | 100.0 (-8.1) | 100.0 (-8.1) | 100.0 (-8.1) |
| Promedus_28 | 100.0 (-8.1) | 100.0 (-8.1) | 100.0 (-8.1) |
| Promedus_29 | 100.0 (-10.5) | 100.0 (-10.5) | 100.0 (-10.5) |
| Promedus_30 | 100.0 (-22.1) | 100.0 (-22.1) | 100.0 (-22.1) |
| Promedus_31 | 100.0 (-1.8) | 100.0 (-1.8) | 100.0 (-1.8) |
| Promedus_32 | 100.0 (-2.2) | 100.0 (-2.2) | 100.0 (-2.2) |
| Promedus_33 | 100.0 (-2.8) | 100.0 (-2.8) | 100.0 (-2.8) |
| Promedus_34 | 100.0 (-3.1) | 100.0 (-3.1) | 100.0 (-3.1) |
| Promedus_35 | 100.0 (-1.7) | 100.0 (-1.7) | 100.0 (-1.7) |
| Promedus_36 | 100.0 (-1.7) | 100.0 (-1.7) | 100.0 (-1.7) |
| Promedus_37 | 95.9 (-4.3) | 100.0 (-4.2) | 100.0 (-4.2) |
| Promedus_38 | 93.1 (-5.3) | 100.0 (-5.0) | 100.0 (-5.0) |
| Segmentation_11 | 100.0 (-24.0) | 100.0 (-24.0) | 100.0 (-24.0) |
| Segmentation_12 | 100.0 (-10.3) | 100.0 (-10.3) | 100.0 (-10.3) |
| Segmentation_13 | 100.0 (-33.4) | 100.0 (-33.4) | 100.0 (-33.4) |
| Segmentation_14 | 100.0 (-39.5) | 100.0 (-39.5) | 100.0 (-39.5) |
| Segmentation_15 | 100.0 (-26.2) | 100.0 (-26.2) | 100.0 (-26.2) |
| Segmentation_16 | 100.0 (-38.1) | 100.0 (-38.1) | 100.0 (-38.1) |
| grid10x10.f10 | 100.0 (303.1) | 100.0 (303.1) | 100.0 (303.1) |
| linkage_12 | 99.7 (-69.5) | 0.0 (nan) | 0.0 (nan) |
| linkage_13 | 99.5 (-76.4) | 0.0 (nan) | 0.0 (nan) |
| linkage_14 | 99.9 (-30.8) | 0.0 (nan) | 100.0 (-30.8) |
| linkage_16 | 100.0 (-38.6) | 100.0 (-38.6) | 100.0 (-38.6) |
| linkage_18 | 99.9 (-78.5) | 100.0 (-78.5) | 100.0 (-78.5) |
| linkage_19 | 98.7 (-59.8) | 100.0 (-59.0) | 100.0 (-59.0) |
| linkage_21 | 100.0 (-29.6) | 100.0 (-29.6) | 100.0 (-29.6) |
| linkage_22 | 99.6 (-54.5) | 100.0 (-54.3) | 0.0 (nan) |
| linkage_23 | 99.8 (-116.7) | 0.0 (nan) | 100.0 (-116.6) |
| linkage_24 | 100.0 (-83.7) | 100.0 (-83.7) | 100.0 (-83.7) |
| linkage_26 | 100.0 (-115.8) | 100.0 (-115.8) | 100.0 (-115.8) |
| linkage_27 | 100.0 (-63.5) | 100.0 (-63.5) | 100.0 (-63.5) |