The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

MPE

overall

Problem 20sec 1200sec 3600sec
Grids_18 100.0 (5674.0) 100.0 (5674.4) 100.0 (5674.4)
Grids_19 100.0 (1179.1) 100.0 (1179.2) 100.0 (1179.2)
Grids_20 92.9 (4588.4) 100.0 (4677.7) 100.0 (4677.6)
Grids_21 100.0 (8495.2) 100.0 (8496.7) 100.0 (8496.7)
Grids_22 99.3 (2817.8) 100.0 (2825.5) 100.0 (2825.5)
Grids_23 100.0 (2781.6) 100.0 (2781.6) 100.0 (2781.6)
Grids_24 99.8 (8227.6) 100.0 (8234.3) 100.0 (8234.3)
Grids_25 100.0 (1167.8) 100.0 (1167.8) 100.0 (1167.8)
Grids_26 100.0 (1325.0) 100.0 (1325.0) 100.0 (1325.0)
Grids_27 100.0 (5504.4) 100.0 (5504.4) 100.0 (5504.4)
Grids_28 100.0 (1981.8) 100.0 (1981.8) 100.0 (1981.8)
Grids_29 100.0 (670.0) 100.0 (670.0) 100.0 (670.0)
Grids_30 100.0 (1309.7) 100.0 (1309.7) 100.0 (1309.7)
ImageAlignment_11 100.0 (-824.2) 100.0 (-824.2) 100.0 (-824.2)
ImageAlignment_12 100.0 (-436.7) 100.0 (-436.7) 100.0 (-436.7)
ImageAlignment_13 100.0 (-2999.9) 100.0 (-2999.9) 100.0 (-2999.9)
ImageAlignment_14 100.0 (-1557.5) 100.0 (-1557.5) 100.0 (-1557.5)
ImageAlignment_15 100.0 (-1177.5) 100.0 (-1177.5) 100.0 (-1177.5)
ObjectDetection_11 85.6 (4237.9) 85.6 (4237.9) 85.6 (4237.9)
ObjectDetection_12 88.8 (6685.0) 88.8 (6685.0) 88.8 (6685.0)
ObjectDetection_13 82.9 (9854.2) 82.9 (9854.2) 82.9 (9854.2)
ObjectDetection_14 94.3 (9020.8) 94.3 (9020.8) 94.3 (9020.8)
ObjectDetection_15 93.3 (12478.6) 93.3 (12478.6) 93.3 (12478.6)
ObjectDetection_16 62.6 (13326.1) 70.3 (13536.2) 70.3 (13536.2)
ObjectDetection_17 97.4 (4816.0) 97.4 (4816.0) 97.4 (4816.0)
ObjectDetection_18 92.0 (8849.5) 92.0 (8849.5) 92.0 (8849.5)
Promedas_11 86.3 (-27.7) 99.1 (-21.5) 100.0 (-21.0)
Promedas_12 98.6 (-15.9) 100.0 (-15.3) 99.7 (-15.4)
Promedas_13 91.0 (-32.9) 100.0 (-27.1) 100.0 (-27.1)
Promedas_14 90.2 (-22.9) 100.0 (-18.6) 100.0 (-18.6)
Promedas_15 95.6 (-26.8) 100.0 (-24.5) 100.0 (-24.5)
Promedas_16 97.3 (-20.9) 100.0 (-19.7) 100.0 (-19.7)
Promedas_17 99.5 (-20.0) 100.0 (-19.8) 100.0 (-19.8)
Promedas_18 98.5 (-25.1) 100.0 (-24.4) 100.0 (-24.4)
Promedas_19 100.0 (-26.1) 100.0 (-26.1) 100.0 (-26.1)
Promedas_20 87.6 (-17.4) 100.0 (-12.2) 100.0 (-12.2)
Promedas_21 100.0 (-17.4) 100.0 (-17.4) 100.0 (-17.4)
Promedas_22 99.0 (-16.6) 100.0 (-16.1) 100.0 (-16.1)
Promedas_23 100.0 (-23.5) 100.0 (-23.5) 100.0 (-23.5)
Promedas_24 100.0 (-12.5) 100.0 (-12.5) 100.0 (-12.5)
Promedas_25 100.0 (-14.7) 100.0 (-14.7) 100.0 (-14.7)
Promedas_26 99.1 (-19.6) 100.0 (-19.2) 100.0 (-19.2)
Promedas_27 99.5 (-10.2) 100.0 (-10.0) 100.0 (-10.0)
Promedas_28 100.0 (-6.0) 100.0 (-6.0) 100.0 (-6.0)
Promedas_29 98.5 (-20.3) 100.0 (-19.8) 100.0 (-19.8)
Promedas_30 100.0 (-10.4) 100.0 (-10.4) 100.0 (-10.4)
Promedas_31 100.0 (-12.2) 100.0 (-12.2) 100.0 (-12.2)
Promedas_32 100.0 (-13.9) 100.0 (-13.9) 100.0 (-13.9)
Promedas_33 100.0 (-14.0) 100.0 (-14.0) 100.0 (-14.0)
Promedas_34 100.0 (-17.0) 100.0 (-17.0) 100.0 (-17.0)
Promedas_35 100.0 (-11.3) 100.0 (-11.3) 100.0 (-11.3)
Promedas_36 100.0 (-8.2) 100.0 (-8.2) 100.0 (-8.2)
Promedas_37 100.0 (-15.4) 100.0 (-15.4) 100.0 (-15.4)
Promedas_38 100.0 (-12.1) 100.0 (-12.1) 100.0 (-12.1)
Promedas_39 99.1 (-21.0) 100.0 (-20.7) 100.0 (-20.7)
Promedas_40 100.0 (-7.3) 100.0 (-7.3) 100.0 (-7.3)
Promedas_41 100.0 (-9.7) 100.0 (-9.7) 100.0 (-9.7)
Promedas_42 100.0 (-12.8) 100.0 (-12.8) 100.0 (-12.8)
Promedas_43 100.0 (-7.5) 100.0 (-7.5) 100.0 (-7.5)
Promedas_44 100.0 (-9.3) 100.0 (-9.3) 100.0 (-9.3)
Promedas_45 100.0 (-7.5) 100.0 (-7.5) 100.0 (-7.5)
Promedas_46 100.0 (-12.8) 100.0 (-12.8) 100.0 (-12.8)
Promedas_47 100.0 (-14.0) 100.0 (-14.0) 100.0 (-14.0)
Promedas_48 100.0 (-10.8) 100.0 (-10.8) 100.0 (-10.8)
Promedas_49 100.0 (-8.4) 100.0 (-8.4) 100.0 (-8.4)
Promedas_50 100.0 (-9.0) 100.0 (-9.0) 100.0 (-9.0)
Promedas_51 100.0 (-12.3) 100.0 (-12.3) 100.0 (-12.3)
Promedas_52 100.0 (-10.8) 100.0 (-10.8) 100.0 (-10.8)
Promedas_53 100.0 (-9.1) 100.0 (-9.1) 100.0 (-9.1)
Promedas_54 100.0 (-10.3) 100.0 (-10.3) 100.0 (-10.3)
Promedas_55 100.0 (-12.7) 100.0 (-12.7) 100.0 (-12.7)
Promedas_56 100.0 (-10.8) 100.0 (-10.8) 100.0 (-10.8)
Promedas_57 100.0 (-10.4) 100.0 (-10.4) 100.0 (-10.4)
Promedas_58 100.0 (-19.9) 100.0 (-19.9) 100.0 (-19.9)
Promedas_59 100.0 (-10.6) 100.0 (-10.6) 100.0 (-10.6)
Promedas_60 100.0 (-10.0) 100.0 (-10.0) 100.0 (-10.0)
Promedas_61 100.0 (-6.9) 100.0 (-6.9) 100.0 (-6.9)
Promedas_62 100.0 (-4.7) 100.0 (-4.7) 100.0 (-4.7)
Promedas_63 100.0 (-6.1) 100.0 (-6.1) 100.0 (-6.1)
Promedas_64 100.0 (-11.5) 100.0 (-11.5) 100.0 (-11.5)
Promedas_65 100.0 (-5.2) 100.0 (-5.2) 100.0 (-5.2)
Promedas_66 100.0 (-10.4) 100.0 (-10.4) 100.0 (-10.4)
Promedas_67 100.0 (-13.4) 100.0 (-13.4) 100.0 (-13.4)
Promedas_68 100.0 (-18.9) 100.0 (-18.9) 100.0 (-18.9)
Promedas_69 100.0 (-11.2) 100.0 (-11.2) 100.0 (-11.2)
Promedas_70 100.0 (-4.1) 100.0 (-4.1) 100.0 (-4.1)
Promedas_71 100.0 (-6.0) 100.0 (-6.0) 100.0 (-6.0)
ProteinFolding_11 100.0 (1961.8) 100.0 (1961.8) 100.0 (1961.8)
ProteinFolding_12 100.0 (-1547.1) 100.0 (-1547.1) 100.0 (-1547.1)
ProteinFolding_13 100.0 (-143.3) 100.0 (-143.3) 100.0 (-143.3)
ProteinFolding_14 100.0 (-331.7) 100.0 (-331.7) 100.0 (-331.7)
ProteinFolding_15 100.0 (-51.6) 100.0 (-51.6) 100.0 (-51.6)
ProteinFolding_16 100.0 (-855.5) 100.0 (-855.5) 100.0 (-855.5)
ProteinFolding_17 100.0 (16544.2) 100.0 (16544.2) 100.0 (16544.2)
ProteinFolding_18 100.0 (18606.3) 100.0 (18606.3) 100.0 (18606.3)
ProteinFolding_19 72.1 (2448.7) 72.1 (2448.7) 72.1 (2448.7)
ProteinFolding_20 99.9 (5577.3) 99.9 (5577.3) 99.9 (5577.3)
ProteinFolding_21 100.0 (7958.7) 100.0 (7958.7) 100.0 (7958.7)
ProteinFolding_22 100.0 (10547.2) 100.0 (10547.2) 100.0 (10547.2)
ProteinFolding_23 75.0 (4049.8) 92.5 (4210.2) 93.0 (4215.1)
ProteinFolding_24 98.1 (5868.8) 98.1 (5868.8) 98.1 (5868.8)
ProteinFolding_25 100.0 (8656.3) 100.0 (8656.3) 100.0 (8656.3)
Segmentation_11 100.0 (-135.8) 100.0 (-135.8) 100.0 (-135.8)
Segmentation_12 100.0 (-22.2) 100.0 (-22.2) 100.0 (-22.2)
Segmentation_13 100.0 (-21.7) 100.0 (-21.7) 100.0 (-21.7)
Segmentation_14 100.0 (-40.1) 100.0 (-40.1) 100.0 (-40.1)
Segmentation_15 100.0 (-165.2) 100.0 (-165.2) 100.0 (-165.2)
Segmentation_16 100.0 (-41.3) 100.0 (-41.3) 100.0 (-41.3)
Segmentation_17 100.0 (-176.8) 100.0 (-176.8) 100.0 (-176.8)
Segmentation_18 100.0 (-35.9) 100.0 (-35.9) 100.0 (-35.9)
Segmentation_19 100.0 (-25.5) 100.0 (-25.5) 100.0 (-25.5)
Segmentation_20 100.0 (-113.9) 100.0 (-113.9) 100.0 (-113.9)
wcsp_11 55.6 (-3.4) 55.6 (-3.4) 100.0 (-3.3)
wcsp_12 99.6 (-2.6) 99.9 (-2.6) 99.9 (-2.6)
wcsp_13 100.0 (-0.2) 100.0 (-0.2) 100.0 (-0.2)
wcsp_14 81.3 (-24.1) 100.0 (-0.1) 100.0 (-0.1)
wcsp_15 100.0 (-76.0) 100.0 (-76.0) 100.0 (-76.0)
wcsp_16 98.2 (-18.3) 100.0 (-16.5) 100.0 (-16.5)
wcsp_17 93.4 (-25.5) 99.6 (-13.2) 99.6 (-13.2)
wcsp_18 70.7 (-20.0) 100.0 (-0.1) 100.0 (-0.1)

MMAP

overall

Problem 20sec 1200sec 3600sec
Grids_18 100.0 (5678.1) 100.0 (5678.7) 100.0 (5678.7)
Grids_19 99.7 (1221.3) 99.8 (1221.6) 99.8 (1221.6)
Grids_20 98.2 (4813.4) 100.0 (4838.4) 100.0 (4839.0)
Grids_21 100.0 (8498.3) 100.0 (8499.7) 100.0 (8499.7)
Grids_22 99.8 (2833.3) 100.0 (2835.2) 100.0 (2835.2)
Grids_23 100.0 (2793.0) 100.0 (2793.0) 100.0 (2793.0)
Grids_24 99.8 (8231.1) 100.0 (8237.3) 100.0 (8237.3)
Grids_25 99.9 (1209.3) 99.9 (1209.3) 99.9 (1209.3)
Grids_26 100.0 (1326.3) 100.0 (1326.3) 100.0 (1326.3)
Grids_27 100.0 (5508.9) 100.0 (5508.9) 100.0 (5508.9)
Grids_28 100.0 (1982.5) 100.0 (1982.5) 100.0 (1982.5)
Grids_29 100.0 (673.0) 100.0 (673.0) 100.0 (673.0)
Grids_30 100.0 (1311.5) 100.0 (1311.5) 100.0 (1311.5)
ImageAlignment_11 100.0 (-824.2) 100.0 (-824.2) 100.0 (-824.2)
ImageAlignment_12 100.0 (-436.7) 100.0 (-436.7) 100.0 (-436.7)
ImageAlignment_13 99.8 (-2999.8) 99.8 (-2999.8) 99.8 (-2999.8)
ImageAlignment_14 100.0 (-1557.5) 100.0 (-1557.5) 100.0 (-1557.5)
ImageAlignment_15 100.0 (-1177.5) 100.0 (-1177.5) 100.0 (-1177.5)
ObjectDetection_11 85.6 (4237.9) 85.6 (4237.9) 85.6 (4237.9)
ObjectDetection_12 88.8 (6685.0) 88.8 (6685.0) 88.8 (6685.0)
ObjectDetection_13 82.9 (9854.2) 82.9 (9854.2) 82.9 (9854.2)
ObjectDetection_14 94.3 (9020.8) 94.3 (9020.8) 94.3 (9020.8)
ObjectDetection_15 93.3 (12478.6) 93.3 (12478.6) 93.3 (12478.6)
ObjectDetection_16 62.6 (13326.1) 70.3 (13536.2) 70.3 (13536.2)
ObjectDetection_17 97.4 (4816.0) 97.4 (4816.0) 97.4 (4816.0)
ObjectDetection_18 92.0 (8849.5) 92.0 (8849.5) 92.0 (8849.5)
Promedas_11 88.3 (-20.0) 99.2 (-13.8) 100.0 (-13.4)
Promedas_12 93.1 (-11.3) 100.0 (-7.6) 94.0 (-10.8)
Promedas_13 93.9 (-25.2) 100.0 (-21.0) 100.0 (-21.0)
Promedas_14 86.2 (-19.8) 98.0 (-14.0) 98.0 (-14.0)
Promedas_15 95.7 (-23.4) 99.9 (-21.2) 99.9 (-21.2)
Promedas_16 100.0 (-14.8) 99.3 (-15.1) 99.3 (-15.1)
Promedas_17 99.5 (-16.7) 100.0 (-16.5) 100.0 (-16.5)
Promedas_18 98.6 (-22.0) 100.0 (-21.3) 100.0 (-21.3)
Promedas_19 96.3 (-23.1) 96.3 (-23.1) 96.3 (-23.1)
Promedas_20 88.0 (-15.8) 100.0 (-10.7) 100.0 (-10.7)
Promedas_21 97.5 (-14.4) 97.5 (-14.4) 97.5 (-14.4)
Promedas_22 99.0 (-15.0) 100.0 (-14.5) 100.0 (-14.5)
Promedas_23 96.9 (-18.5) 96.9 (-18.5) 96.9 (-18.5)
Promedas_24 100.0 (-9.5) 100.0 (-9.5) 100.0 (-9.5)
Promedas_25 100.0 (-11.6) 100.0 (-11.6) 100.0 (-11.6)
Promedas_26 100.0 (-11.9) 91.6 (-16.1) 91.6 (-16.1)
Promedas_27 96.2 (-10.2) 96.7 (-10.0) 96.7 (-10.0)
Promedas_28 100.0 (-4.4) 100.0 (-4.4) 100.0 (-4.4)
Promedas_29 91.5 (-17.2) 100.0 (-13.6) 100.0 (-13.6)
Promedas_30 100.0 (-10.4) 100.0 (-10.4) 100.0 (-10.4)
Promedas_31 100.0 (-10.7) 100.0 (-10.7) 100.0 (-10.7)
Promedas_32 100.0 (-10.8) 92.1 (-13.9) 100.0 (-10.8)
Promedas_33 100.0 (-12.5) 100.0 (-12.5) 100.0 (-12.5)
Promedas_34 100.0 (-14.0) 100.0 (-14.0) 100.0 (-14.0)
Promedas_35 100.0 (-9.7) 100.0 (-9.7) 100.0 (-9.7)
Promedas_36 100.0 (-8.2) 100.0 (-8.2) 100.0 (-8.2)
Promedas_37 100.0 (-10.8) 100.0 (-10.8) 100.0 (-10.8)
Promedas_38 100.0 (-12.1) 100.0 (-12.1) 100.0 (-12.1)
Promedas_39 93.3 (-17.9) 94.1 (-17.6) 94.1 (-17.6)
Promedas_40 98.5 (-7.3) 98.5 (-7.3) 98.5 (-7.3)
Promedas_41 100.0 (-8.1) 100.0 (-8.1) 100.0 (-8.1)
Promedas_42 92.4 (-11.2) 92.4 (-11.2) 92.4 (-11.2)
Promedas_43 100.0 (-7.5) 100.0 (-7.5) 100.0 (-7.5)
Promedas_44 100.0 (-6.2) 100.0 (-6.2) 100.0 (-6.2)
Promedas_45 100.0 (-6.0) 100.0 (-6.0) 100.0 (-6.0)
Promedas_46 100.0 (-8.2) 100.0 (-8.2) 100.0 (-8.2)
Promedas_47 100.0 (-6.4) 100.0 (-6.4) 100.0 (-6.4)
Promedas_48 88.4 (-10.7) 88.4 (-10.7) 88.4 (-10.7)
Promedas_49 100.0 (-6.8) 100.0 (-6.8) 100.0 (-6.8)
Promedas_50 100.0 (-7.4) 100.0 (-7.4) 100.0 (-7.4)
Promedas_51 100.0 (-7.7) 100.0 (-7.7) 100.0 (-7.7)
Promedas_52 100.0 (-10.7) 100.0 (-10.7) 100.0 (-10.7)
Promedas_53 100.0 (-9.1) 100.0 (-9.1) 100.0 (-9.1)
Promedas_54 100.0 (-8.8) 100.0 (-8.8) 100.0 (-8.8)
Promedas_55 100.0 (-11.2) 100.0 (-11.2) 100.0 (-11.2)
Promedas_56 100.0 (-7.8) 100.0 (-7.8) 100.0 (-7.8)
Promedas_57 100.0 (-10.3) 100.0 (-10.3) 100.0 (-10.3)
Promedas_58 100.0 (-15.3) 100.0 (-15.3) 100.0 (-15.3)
Promedas_59 100.0 (-10.5) 100.0 (-10.5) 100.0 (-10.5)
Promedas_60 100.0 (-10.0) 100.0 (-10.0) 100.0 (-10.0)
Promedas_61 100.0 (-5.4) 100.0 (-5.4) 100.0 (-5.4)
Promedas_62 100.0 (-4.7) 100.0 (-4.7) 100.0 (-4.7)
Promedas_63 100.0 (-4.6) 100.0 (-4.6) 100.0 (-4.6)
Promedas_64 100.0 (-9.9) 100.0 (-9.9) 100.0 (-9.9)
Promedas_65 100.0 (-5.1) 100.0 (-5.1) 100.0 (-5.1)
Promedas_66 100.0 (-10.4) 100.0 (-10.4) 100.0 (-10.4)
Promedas_67 95.9 (-11.8) 95.9 (-11.8) 95.9 (-11.8)
Promedas_68 100.0 (-18.8) 100.0 (-18.8) 100.0 (-18.8)
Promedas_69 100.0 (-9.7) 100.0 (-9.7) 100.0 (-9.7)
Promedas_70 100.0 (-4.1) 100.0 (-4.1) 100.0 (-4.1)
Promedas_71 100.0 (-6.0) 100.0 (-6.0) 100.0 (-6.0)
ProteinFolding_11 100.0 (1962.3) 100.0 (1962.3) 100.0 (1962.3)
ProteinFolding_12 100.0 (-1547.0) 100.0 (-1547.0) 100.0 (-1547.0)
ProteinFolding_13 100.0 (-143.3) 100.0 (-143.3) 100.0 (-143.3)
ProteinFolding_14 100.0 (-331.7) 100.0 (-331.7) 100.0 (-331.7)
ProteinFolding_15 100.0 (-51.6) 100.0 (-51.6) 100.0 (-51.6)
ProteinFolding_16 100.0 (-855.5) 100.0 (-855.5) 100.0 (-855.5)
ProteinFolding_17 100.0 (16544.2) 100.0 (16544.2) 100.0 (16544.2)
ProteinFolding_18 100.0 (18606.3) 100.0 (18606.3) 100.0 (18606.3)
ProteinFolding_19 72.1 (2448.7) 72.1 (2448.7) 72.1 (2448.7)
ProteinFolding_20 99.9 (5577.3) 99.9 (5577.3) 99.9 (5577.3)
ProteinFolding_21 100.0 (7958.7) 100.0 (7958.7) 100.0 (7958.7)
ProteinFolding_22 100.0 (10547.2) 100.0 (10547.2) 100.0 (10547.2)
ProteinFolding_23 75.0 (4050.0) 92.5 (4210.2) 93.0 (4215.1)
ProteinFolding_24 98.1 (5868.8) 98.1 (5868.8) 98.1 (5868.8)
ProteinFolding_25 100.0 (8656.3) 100.0 (8656.3) 100.0 (8656.3)
Segmentation_11 98.6 (-132.3) 98.6 (-132.3) 98.6 (-132.3)
Segmentation_12 100.0 (-21.9) 100.0 (-21.9) 100.0 (-21.9)
Segmentation_13 100.0 (-21.4) 100.0 (-21.4) 100.0 (-21.4)
Segmentation_14 100.0 (-39.3) 100.0 (-39.3) 100.0 (-39.3)
Segmentation_15 99.4 (-163.8) 99.4 (-163.8) 99.4 (-163.8)
Segmentation_16 100.0 (-40.4) 100.0 (-40.4) 100.0 (-40.4)
Segmentation_17 100.0 (-174.3) 100.0 (-174.3) 100.0 (-174.3)
Segmentation_18 100.0 (-34.0) 100.0 (-34.0) 100.0 (-34.0)
Segmentation_19 100.0 (-24.1) 100.0 (-24.1) 100.0 (-24.1)
Segmentation_20 100.0 (-112.0) 100.0 (-112.0) 100.0 (-112.0)
wcsp_11 59.2 (5.7) 58.2 (5.6) 65.9 (6.8)
wcsp_12 84.6 (13.2) 85.7 (13.4) 85.5 (13.3)
wcsp_13 100.0 (0.7) 100.0 (0.7) 100.0 (0.7)
wcsp_14 81.9 (-22.0) 99.6 (1.1) 100.0 (1.5)
wcsp_15 99.6 (-75.1) 100.0 (-74.6) 100.0 (-74.6)
wcsp_16 99.5 (22.8) 100.0 (23.4) 99.9 (23.2)
wcsp_17 94.1 (20.6) 99.3 (33.4) 99.3 (33.3)
wcsp_18 78.9 (-14.2) 100.0 (0.2) 100.0 (0.2)

Updated: