The results below are organized as follows:

  • each table displays the solver’s performance for individual problem instances for the given task under different time limits
  • table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria

MMAP

overall

Problem 20sec 1200sec 3600sec
75-17-5.Q0.5.I4 92.7 (-8.5) 0.0 (-inf) 0.0 (-inf)
75-19-5.Q0.5.I2 0.0 (-inf) 100.0 (-9.5) 100.0 (-9.5)
75-22-5.Q0.5.I2 100.0 (-11.5) 0.0 (-inf) 0.0 (-inf)
75-23-5.Q0.5.I3 0.0 (-inf) 99.8 (-12.5) 99.8 (-12.5)
75-26-5.Q0.5.I4 0.0 (nan) 96.8 (-19.0) 96.8 (-19.0)
90-22-5.Q0.5.I4 0.0 (-inf) 0.0 (-inf) 100.0 (-5.6)
90-24-5.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
90-25-5.Q0.5.I2 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-26-5.Q0.5.I1 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
90-30-5.Q0.5.I1 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-34-5.Q0.5.I2 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-38-5.Q0.5.I4 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-42-5.Q0.5.I4 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-46-5.Q0.5.I4 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
90-50-5.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (-inf)
Grids_20 0.0 (nan) 0.0 (4552.1) 0.0 (4551.3)
Grids_21 0.0 (nan) 1.5 (8031.5) 0.0 (7952.7)
Grids_22 0.0 (2682.0) 0.0 (2690.8) 0.0 (2690.8)
Grids_23 0.0 (2649.1) 0.0 (2644.2) 0.0 (2639.9)
Grids_24 0.0 (7761.6) 0.0 (7724.2) 0.0 (7765.5)
Grids_25 0.0 (1138.0) 0.0 (1147.1) 8.1 (1167.2)
Grids_26 11.5 (1279.2) 3.5 (1275.0) 3.5 (1275.0)
Grids_27 0.0 (5131.1) 0.0 (5114.8) 0.0 (5121.6)
Grids_28 0.0 (1906.2) 0.0 (1900.8) 0.0 (1899.7)
Grids_29 0.0 (651.2) 0.0 (651.2) 0.0 (651.2)
ImageAlignment_11 0.0 (-8256.1) 0.0 (-5852.0) 0.0 (nan)
ImageAlignment_12 100.0 (-436.7) 0.0 (nan) 0.0 (nan)
ImageAlignment_13 0.0 (-14566.4) 0.0 (-9121.9) 0.0 (-9180.4)
ImageAlignment_14 0.0 (-2454.4) 0.0 (nan) 0.0 (nan)
ImageAlignment_15 0.0 (-1675.5) 0.0 (nan) 0.0 (nan)
ObjectDetection_13 0.0 (nan) 0.0 (nan) 0.0 (nan)
ObjectDetection_14 0.0 (4931.7) 0.0 (nan) 0.0 (nan)
ObjectDetection_15 0.0 (6985.6) 0.0 (7045.5) 0.0 (nan)
ObjectDetection_16 0.0 (nan) 0.0 (nan) 0.0 (nan)
ObjectDetection_17 0.0 (-822.5) 0.0 (nan) 0.0 (nan)
ProteinFolding_11 61.6 (1727.6) 61.7 (1728.4) 61.7 (1728.4)
ProteinFolding_12 0.0 (-2426.5) 0.0 (nan) 0.0 (nan)
ProteinFolding_13 100.0 (-143.3) 100.0 (-143.3) 100.0 (-143.3)
ProteinFolding_14 100.0 (-331.7) 100.0 (-331.7) 0.0 (nan)
ProteinFolding_15 100.0 (-600.5) 100.0 (-600.5) 100.0 (-51.6)
Segmentation_11 0.0 (-207.6) 0.0 (-207.6) 100.0 (-132.1)
Segmentation_12 100.0 (-21.9) 100.0 (-21.9) 100.0 (-21.9)
Segmentation_13 100.0 (-21.4) 100.0 (-21.4) 100.0 (-21.4)
Segmentation_14 100.0 (-39.3) 100.0 (-39.3) 100.0 (-39.3)
Segmentation_15 0.0 (-318.1) 0.0 (-318.1) 100.0 (-163.7)
Segmentation_16 100.0 (-40.4) 100.0 (-40.4) 100.0 (-40.4)
Segmentation_17 0.0 (-386.4) 0.0 (-386.4) 98.2 (-174.6)
Segmentation_18 100.0 (-34.0) 100.0 (-34.0) 100.0 (-34.0)
Segmentation_19 100.0 (-24.1) 100.0 (-24.1) 100.0 (-24.1)
Segmentation_20 0.0 (-151.4) 0.0 (-151.4) 100.0 (-112.0)
bw_p24_16 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p24_20 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p34_15 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p34_20 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p44_15 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p44_19 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p54_10 0.0 (nan) 0.0 (nan) 0.0 (nan)
bw_p54_16 0.0 (nan) 0.0 (nan) 0.0 (nan)
comm_p01_16 0.0 (nan) 0.0 (nan) 0.0 (nan)
comm_p01_20 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_11.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_16.fg.Q0.5.I3 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
or_chain_22.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_24.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_25.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_32.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_36.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_39.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_40.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_41.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_43.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_6.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_60.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_63.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
or_chain_8.fg.Q0.5.I3 0.0 (nan) 0.0 (nan) 0.0 (nan)
pedigree1.Q0.5.I3 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree13.Q0.5.I1 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree18.Q0.5.I1 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree19.Q0.5.I4 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree20.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree25.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree30.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree31.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree33.Q0.5.I2 90.4 (-74.8) 99.6 (-69.4) 99.6 (-69.4)
pedigree38.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree41.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree44.Q0.5.I4 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree50.Q0.5.I1 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree7.Q0.5.I2 0.0 (-inf) 0.0 (-inf) 0.0 (-inf)
pedigree9.Q0.5.I3 0.0 (nan) 0.0 (-inf) 0.0 (-inf)
pomdp10-12_7_3_8_4.mmap 0.0 (nan) 0.0 (nan) 0.0 (nan)
pomdp6-12_6_2_6_3.mmap 95.3 (1.0) 97.4 (1.0) 95.9 (1.0)
pomdp7-20_10_2_10_3.mmap 0.0 (nan) 0.0 (nan) 0.0 (nan)
pomdp8-14_9_3_12_4.mmap 0.0 (nan) 0.0 (nan) 0.0 (nan)
pomdp9-14_8_3_10_4.mmap 0.0 (nan) 0.0 (nan) 0.0 (nan)
wcsp_14 0.0 (nan) 0.0 (-857.9) 82.1 (-21.5)
wcsp_15 0.0 (-467.6) 42.8 (-137.6) 42.8 (-137.6)
wcsp_16 0.0 (-398.5) 0.0 (-396.7) 0.0 (-372.1)
wcsp_17 0.0 (-704.2) 0.0 (-766.5) 32.4 (-92.6)
wcsp_18 0.0 (-698.7) 0.0 (-451.9) 0.0 (-631.8)

Updated: