The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

MMAP

overall

Problem 20sec 1200sec 3600sec
Grids_18 93.9 (5538.3) 91.5 (5481.7) 92.9 (5513.9)
Grids_19 99.3 (1219.6) 98.3 (1215.8) 0.0 (nan)
Grids_20 96.6 (4790.5) 0.0 (nan) 0.0 (nan)
Grids_21 91.1 (8185.4) 0.0 (nan) 91.6 (8204.6)
Grids_22 94.4 (2777.8) 96.6 (2799.9) 93.8 (2771.3)
Grids_23 95.5 (2750.8) 96.2 (2757.2) 95.8 (2753.5)
Grids_24 92.6 (8021.2) 93.2 (8038.1) 92.1 (8005.0)
Grids_25 98.5 (1204.2) 98.1 (1202.5) 99.9 (1209.3)
Grids_26 90.7 (1279.1) 93.7 (1294.1) 91.8 (1284.5)
Grids_27 93.2 (5378.2) 94.1 (5395.8) 90.8 (5331.2)
Grids_28 92.9 (1927.0) 94.9 (1942.6) 95.5 (1947.4)
Grids_29 94.4 (659.4) 95.4 (661.8) 99.2 (671.2)
Grids_30 98.6 (1304.9) 94.9 (1287.1) 93.8 (1281.8)
ImageAlignment_11 100.0 (-824.2) 100.0 (-824.2) 100.0 (-824.2)
ImageAlignment_12 100.0 (-436.7) 100.0 (-436.7) 100.0 (-436.7)
ImageAlignment_13 99.0 (-3004.4) 100.0 (-2998.9) 100.0 (-2998.9)
ImageAlignment_14 100.0 (-1557.5) 100.0 (-1557.5) 100.0 (-1557.5)
ImageAlignment_15 100.0 (-1177.5) 0.0 (nan) 0.0 (nan)
ObjectDetection_11 15.4 (3383.2) 0.0 (3176.1) 0.0 (3176.1)
ObjectDetection_12 0.0 (5342.9) 0.0 (5503.1) 0.0 (5503.1)
ObjectDetection_13 0.0 (8569.1) 0.0 (8883.1) 0.0 (8883.1)
ObjectDetection_14 64.6 (8638.2) 58.7 (8562.9) 58.7 (8562.9)
ObjectDetection_15 68.9 (11916.6) 72.7 (12004.1) 71.5 (11976.2)
ObjectDetection_16 83.6 (13900.5) 88.1 (14021.5) 88.1 (14021.5)
ObjectDetection_17 67.1 (3992.8) 86.4 (4518.2) 86.4 (4518.2)
ObjectDetection_18 95.1 (8914.2) 82.5 (8650.9) 83.5 (8673.1)
Promedas_11 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_12 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_13 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_14 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_15 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_16 99.0 (-15.2) 0.0 (nan) 0.0 (nan)
Promedas_17 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_18 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_19 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_20 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_21 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_22 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_23 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_24 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_25 0.0 (-inf) 72.3 (-21.3) 0.0 (-inf)
Promedas_26 96.3 (-13.7) 96.3 (-13.7) 96.3 (-13.7)
Promedas_27 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_28 99.5 (-4.6) 99.5 (-4.6) 99.5 (-4.6)
Promedas_29 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_30 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_31 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_32 96.4 (-12.2) 96.4 (-12.2) 96.4 (-12.2)
Promedas_33 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_34 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_35 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_36 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_37 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_38 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_39 0.0 (-inf) 0.0 (nan) 0.0 (nan)
Promedas_40 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_41 67.6 (-17.3) 0.0 (-inf) 0.0 (-inf)
Promedas_42 0.0 (-inf) 0.0 (-inf) 93.2 (-10.9)
Promedas_43 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_44 95.7 (-7.6) 95.7 (-7.6) 95.7 (-7.6)
Promedas_45 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_46 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_47 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_48 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_49 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_50 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_51 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_52 89.0 (-13.0) 89.0 (-13.0) 89.0 (-13.0)
Promedas_53 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_54 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_55 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_56 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_57 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_58 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_59 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_60 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_61 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_62 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_63 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_64 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_65 0.0 (-inf) 73.3 (-10.0) 73.3 (-10.0)
Promedas_66 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
Promedas_67 100.0 (-10.7) 100.0 (-10.7) 100.0 (-10.7)
Promedas_68 100.0 (-18.8) 100.0 (-18.8) 100.0 (-18.8)
Promedas_69 0.0 (-inf) 39.2 (-18.3) 39.2 (-18.3)
Promedas_70 0.0 (-inf) 86.8 (-5.1) 86.8 (-5.1)
Promedas_71 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
ProteinFolding_11 4.1 (1260.9) 26.3 (1423.3) 30.2 (1452.0)
ProteinFolding_12 99.3 (-1549.1) 0.0 (nan) 0.0 (nan)
ProteinFolding_13 100.0 (-143.3) 100.0 (-143.3) 100.0 (-143.3)
ProteinFolding_14 100.0 (-331.7) 100.0 (-331.7) 100.0 (-331.7)
ProteinFolding_15 100.0 (-51.6) 100.0 (-51.6) 100.0 (-51.6)
ProteinFolding_16 100.0 (-855.5) 100.0 (-855.5) 100.0 (-855.5)
ProteinFolding_17 48.3 (12517.3) 63.4 (13693.6) 63.4 (13697.2)
ProteinFolding_18 32.1 (12289.2) 48.1 (13772.5) 48.1 (13772.5)
ProteinFolding_19 0.0 (1540.4) 0.0 (1298.7) 0.0 (1298.7)
ProteinFolding_20 12.3 (4301.6) 4.4 (4187.5) 5.6 (4204.3)
ProteinFolding_21 0.0 (4209.7) 0.0 (6687.1) 0.0 (6687.1)
ProteinFolding_22 0.0 (9753.2) 95.2 (10521.2) 95.2 (10521.2)
ProteinFolding_23 91.1 (4197.5) 80.4 (4099.4) 80.4 (4099.4)
ProteinFolding_24 95.1 (5821.1) 96.4 (5841.7) 96.4 (5841.7)
ProteinFolding_25 97.8 (8607.2) 100.0 (8656.3) 100.0 (8656.3)
Segmentation_11 73.0 (-135.6) 90.3 (-133.4) 90.3 (-133.4)
Segmentation_12 100.0 (-21.9) 100.0 (-21.9) 100.0 (-21.9)
Segmentation_13 100.0 (-21.4) 100.0 (-21.4) 100.0 (-21.4)
Segmentation_14 75.4 (-41.1) 92.2 (-39.8) 92.2 (-39.8)
Segmentation_15 77.3 (-169.2) 78.7 (-168.8) 78.7 (-168.8)
Segmentation_16 93.9 (-40.8) 93.9 (-40.8) 93.9 (-40.8)
Segmentation_17 92.4 (-175.9) 93.6 (-175.7) 92.7 (-175.9)
Segmentation_18 77.3 (-38.1) 100.0 (-34.0) 100.0 (-34.0)
Segmentation_19 100.0 (-24.1) 100.0 (-24.1) 100.0 (-24.1)
Segmentation_20 78.1 (-116.7) 93.3 (-113.5) 93.3 (-113.5)
wcsp_11 0.0 (nan) 0.0 (nan) 0.0 (nan)
wcsp_12 0.0 (nan) 0.0 (nan) 0.0 (nan)
wcsp_13 30.8 (-91.1) 83.8 (-20.9) 89.4 (-13.3)
wcsp_14 0.0 (nan) 81.9 (-21.9) 82.4 (-21.3)
wcsp_15 0.0 (-409.3) 0.0 (-194.5) 0.0 (-193.8)
wcsp_16 82.5 (-0.7) 82.3 (-0.8) 84.1 (1.5)
wcsp_17 73.4 (-30.4) 0.0 (nan) 0.0 (nan)
wcsp_18 0.0 (-73.4) 77.0 (-15.5) 69.7 (-20.5)

Updated: