The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

PR

overall

Problem 20sec 1200sec 3600sec
Alchemy_11 100.0 (606.3) 100.0 (606.3) 100.0 (606.3)
CSP_11 100.0 (13.6) 100.0 (13.6) 100.0 (13.6)
CSP_12 100.0 (16.5) 100.0 (16.5) 100.0 (16.5)
CSP_13 100.0 (15.3) 100.0 (15.3) 100.0 (15.3)
DBN_11 100.0 (58.5) 100.0 (58.5) 100.0 (58.5)
DBN_12 100.0 (63.1) 100.0 (63.1) 100.0 (63.1)
DBN_13 100.0 (66.6) 100.0 (66.6) 100.0 (66.6)
DBN_14 100.0 (151.2) 100.0 (151.2) 100.0 (151.2)
DBN_15 100.0 (152.6) 100.0 (152.6) 100.0 (152.6)
DBN_16 100.0 (166.1) 100.0 (166.1) 100.0 (166.1)
Grids_11 100.0 (169.4) 100.0 (169.4) 100.0 (169.4)
Grids_12 100.0 (303.1) 100.0 (303.1) 100.0 (303.1)
Grids_13 100.0 (333.3) 100.0 (333.3) 100.0 (333.3)
Grids_14 100.0 (497.8) 100.0 (497.8) 100.0 (497.8)
Grids_15 100.0 (291.7) 100.0 (291.7) 100.0 (291.7)
Grids_16 100.0 (665.1) 100.0 (665.1) 100.0 (665.1)
Grids_17 100.0 (1312.0) 100.0 (1312.0) 100.0 (1312.0)
Grids_18 99.9 (1963.5) 100.0 (1963.0) 100.0 (1963.0)
ObjectDetection_11 100.0 (-74.9) 100.0 (-74.9) 100.0 (-74.9)
ObjectDetection_12 100.0 (-67.4) 100.0 (-67.4) 100.0 (-67.4)
ObjectDetection_13 99.3 (-70.4) 100.0 (-70.4) 100.0 (-70.4)
ObjectDetection_14 99.1 (-64.1) 100.0 (-64.1) 100.0 (-64.1)
ObjectDetection_15 100.0 (-51.9) 100.0 (-51.9) 100.0 (-51.9)
ObjectDetection_16 99.4 (-34.4) 100.0 (-34.4) 100.0 (-34.4)
ObjectDetection_17 99.4 (-29.8) 100.0 (-29.8) 100.0 (-29.8)
ObjectDetection_18 100.0 (-37.0) 100.0 (-37.0) 100.0 (-37.0)
ObjectDetection_19 95.3 (-38.5) 100.0 (-38.5) 100.0 (-38.5)
ObjectDetection_20 97.9 (-34.9) 100.0 (-34.8) 100.0 (-34.8)
ObjectDetection_21 100.0 (-24.4) 100.0 (-24.4) 100.0 (-24.4)
ObjectDetection_22 100.0 (-36.7) 100.0 (-36.7) 100.0 (-36.7)
ObjectDetection_23 99.2 (-31.6) 100.0 (-31.6) 100.0 (-31.6)
ObjectDetection_24 99.2 (-26.4) 100.0 (-26.4) 100.0 (-26.4)
ObjectDetection_25 100.0 (-18.7) 100.0 (-18.7) 100.0 (-18.7)
ObjectDetection_26 99.1 (-33.8) 100.0 (-33.8) 100.0 (-33.8)
ObjectDetection_27 99.3 (-29.9) 100.0 (-29.9) 100.0 (-29.9)
ObjectDetection_28 99.9 (-46.8) 99.9 (-46.8) 99.9 (-46.8)
ObjectDetection_29 100.0 (-50.8) 100.0 (-50.8) 100.0 (-50.8)
ObjectDetection_30 100.0 (-52.4) 100.0 (-52.4) 100.0 (-52.4)
ObjectDetection_31 100.0 (-44.1) 100.0 (-44.1) 100.0 (-44.1)
ObjectDetection_32 100.0 (-46.7) 100.0 (-46.7) 100.0 (-46.7)
ObjectDetection_33 100.0 (-44.1) 100.0 (-44.1) 100.0 (-44.1)
ObjectDetection_34 100.0 (-35.4) 100.0 (-35.4) 100.0 (-35.4)
ObjectDetection_35 100.0 (-37.6) 100.0 (-37.6) 100.0 (-37.6)
ObjectDetection_36 100.0 (-30.7) 100.0 (-30.7) 100.0 (-30.7)
ObjectDetection_37 100.0 (-30.0) 100.0 (-30.0) 100.0 (-30.0)
ObjectDetection_38 100.0 (-75.1) 100.0 (-75.1) 100.0 (-75.1)
ObjectDetection_39 100.0 (-70.2) 100.0 (-70.2) 100.0 (-70.2)
ObjectDetection_40 100.0 (-72.3) 100.0 (-72.3) 100.0 (-72.3)
ObjectDetection_41 100.0 (-71.1) 100.0 (-71.1) 100.0 (-71.1)
ObjectDetection_42 100.0 (-45.2) 100.0 (-45.2) 100.0 (-45.2)
ObjectDetection_43 100.0 (-39.5) 100.0 (-39.5) 100.0 (-39.5)
ObjectDetection_44 100.0 (-41.0) 100.0 (-41.0) 100.0 (-41.0)
ObjectDetection_45 100.0 (-38.9) 100.0 (-38.9) 100.0 (-38.9)
ObjectDetection_46 100.0 (-42.0) 100.0 (-42.0) 100.0 (-42.0)
ObjectDetection_47 100.0 (-44.7) 100.0 (-44.7) 100.0 (-44.7)
ObjectDetection_48 100.0 (-36.6) 100.0 (-36.6) 100.0 (-36.6)
ObjectDetection_49 100.0 (-38.7) 100.0 (-38.7) 100.0 (-38.7)
ObjectDetection_50 100.0 (-36.1) 100.0 (-36.1) 100.0 (-36.1)
ObjectDetection_51 100.0 (-53.4) 100.0 (-53.4) 100.0 (-53.4)
ObjectDetection_52 100.0 (-57.0) 100.0 (-57.0) 100.0 (-57.0)
ObjectDetection_53 100.0 (-47.7) 100.0 (-47.7) 100.0 (-47.7)
ObjectDetection_54 100.0 (-41.5) 100.0 (-41.5) 100.0 (-41.5)
ObjectDetection_55 100.0 (-34.1) 100.0 (-34.1) 100.0 (-34.1)
ObjectDetection_56 100.0 (-36.3) 100.0 (-36.3) 100.0 (-36.3)
ObjectDetection_57 100.0 (-33.4) 100.0 (-33.4) 100.0 (-33.4)
ObjectDetection_58 100.0 (-41.7) 100.0 (-41.7) 100.0 (-41.7)
ObjectDetection_59 100.0 (-35.9) 100.0 (-35.9) 100.0 (-35.9)
ObjectDetection_60 100.0 (-35.3) 100.0 (-35.3) 100.0 (-35.3)
ObjectDetection_61 100.0 (-23.2) 100.0 (-23.2) 100.0 (-23.2)
ObjectDetection_62 100.0 (-24.8) 100.0 (-24.8) 100.0 (-24.8)
ObjectDetection_63 100.0 (-19.9) 100.0 (-19.9) 100.0 (-19.9)
ObjectDetection_64 100.0 (-21.3) 100.0 (-21.3) 100.0 (-21.3)
ObjectDetection_65 100.0 (-19.7) 100.0 (-19.7) 100.0 (-19.7)
ObjectDetection_66 100.0 (-47.0) 100.0 (-47.0) 100.0 (-47.0)
ObjectDetection_67 100.0 (-40.6) 100.0 (-40.6) 100.0 (-40.6)
ObjectDetection_68 100.0 (-42.7) 100.0 (-42.7) 100.0 (-42.7)
ObjectDetection_69 100.0 (-40.3) 100.0 (-40.3) 100.0 (-40.3)
ObjectDetection_70 100.0 (-31.2) 100.0 (-31.2) 100.0 (-31.2)
ObjectDetection_71 100.0 (-33.6) 100.0 (-33.6) 100.0 (-33.6)
ObjectDetection_72 100.0 (-26.4) 100.0 (-26.4) 100.0 (-26.4)
ObjectDetection_73 100.0 (-25.6) 100.0 (-25.6) 100.0 (-25.6)
ObjectDetection_74 100.0 (-31.8) 100.0 (-31.8) 100.0 (-31.8)
ObjectDetection_75 100.0 (-33.8) 100.0 (-33.8) 100.0 (-33.8)
Pedigree_11 100.0 (-17.2) 100.0 (-17.2) 100.0 (-17.2)
Pedigree_12 100.0 (-11.5) 100.0 (-11.5) 100.0 (-11.5)
Pedigree_13 100.0 (-15.2) 100.0 (-15.2) 100.0 (-15.2)
Promedus_11 100.0 (-8.4) 100.0 (-8.4) 100.0 (-8.4)
Promedus_12 100.0 (-3.2) 100.0 (-3.2) 100.0 (-3.2)
Promedus_13 100.0 (-4.5) 100.0 (-4.5) 100.0 (-4.5)
Promedus_14 100.0 (-7.8) 100.0 (-7.8) 100.0 (-7.8)
Promedus_15 100.0 (-3.6) 100.0 (-3.6) 100.0 (-3.6)
Promedus_16 100.0 (-7.0) 100.0 (-7.0) 100.0 (-7.0)
Promedus_17 100.0 (-9.5) 100.0 (-9.5) 100.0 (-9.5)
Promedus_18 99.9 (-4.7) 100.0 (-4.7) 100.0 (-4.7)
Promedus_19 99.9 (-4.3) 100.0 (-4.3) 100.0 (-4.3)
Promedus_20 100.0 (-7.1) 100.0 (-7.1) 100.0 (-7.1)
Promedus_21 100.0 (-5.6) 100.0 (-5.6) 100.0 (-5.6)
Promedus_22 100.0 (-2.5) 100.0 (-2.5) 100.0 (-2.5)
Promedus_23 100.0 (-11.3) 100.0 (-11.3) 100.0 (-11.3)
Promedus_24 100.0 (-5.9) 100.0 (-5.9) 100.0 (-5.9)
Promedus_25 100.0 (-9.4) 100.0 (-9.4) 100.0 (-9.4)
Promedus_26 100.0 (-7.3) 100.0 (-7.3) 100.0 (-7.3)
Promedus_27 100.0 (-8.1) 100.0 (-8.1) 100.0 (-8.1)
Promedus_28 100.0 (-8.1) 100.0 (-8.1) 100.0 (-8.1)
Promedus_29 100.0 (-10.5) 100.0 (-10.5) 100.0 (-10.5)
Promedus_30 100.0 (-22.1) 100.0 (-22.1) 100.0 (-22.1)
Promedus_31 100.0 (-1.8) 100.0 (-1.8) 100.0 (-1.8)
Promedus_32 100.0 (-2.2) 100.0 (-2.2) 100.0 (-2.2)
Promedus_33 100.0 (-2.8) 100.0 (-2.8) 100.0 (-2.8)
Promedus_34 100.0 (-3.1) 100.0 (-3.1) 100.0 (-3.1)
Promedus_35 100.0 (-1.7) 100.0 (-1.7) 100.0 (-1.7)
Promedus_36 100.0 (-1.7) 100.0 (-1.7) 100.0 (-1.7)
Promedus_37 100.0 (-4.2) 100.0 (-4.2) 100.0 (-4.2)
Promedus_38 100.0 (-5.0) 100.0 (-5.0) 100.0 (-5.0)
Segmentation_11 100.0 (-24.0) 100.0 (-24.0) 100.0 (-24.0)
Segmentation_12 100.0 (-10.3) 100.0 (-10.3) 100.0 (-10.3)
Segmentation_13 100.0 (-33.4) 100.0 (-33.4) 100.0 (-33.4)
Segmentation_14 100.0 (-39.5) 100.0 (-39.5) 100.0 (-39.5)
Segmentation_15 100.0 (-26.2) 100.0 (-26.2) 100.0 (-26.2)
Segmentation_16 100.0 (-38.1) 100.0 (-38.1) 100.0 (-38.1)
grid10x10.f10 100.0 (303.1) 100.0 (303.1) 100.0 (303.1)
linkage_12 99.9 (-69.6) 100.0 (-69.7) 100.0 (-69.7)
linkage_13 99.9 (-76.0) 99.9 (-76.1) 99.9 (-76.1)
linkage_14 99.7 (-30.8) 100.0 (-30.8) 100.0 (-30.8)
linkage_16 100.0 (-38.6) 100.0 (-38.6) 100.0 (-38.6)
linkage_18 99.9 (-78.6) 99.9 (-78.5) 99.9 (-78.5)
linkage_19 99.9 (-59.1) 100.0 (-59.0) 100.0 (-59.0)
linkage_21 99.9 (-29.7) 99.9 (-29.7) 99.9 (-29.7)
linkage_22 100.0 (-54.3) 99.9 (-54.2) 99.9 (-54.2)
linkage_23 99.5 (-116.9) 99.9 (-116.6) 99.9 (-116.6)
linkage_24 100.0 (-83.7) 100.0 (-83.7) 100.0 (-83.7)
linkage_26 100.0 (-115.8) 100.0 (-115.8) 100.0 (-115.8)
linkage_27 99.9 (-63.5) 100.0 (-63.5) 100.0 (-63.5)

Updated: