braobb-mmap Scores
The results below are organized as follows:
- each table displays the solver’s performance for individual problem instances for the given task under different time limits
- table values are normalized scores for each evaluated problem as outlined in Evaluation Criteria
MMAP
overall
| Problem | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| 75-17-5.Q0.5.I4 | 92.4 (-8.3) | 100.0 (-7.5) | 100.0 (-7.5) |
| 75-19-5.Q0.5.I2 | 76.8 (-12.9) | 99.6 (-9.7) | 99.6 (-9.7) |
| 75-22-5.Q0.5.I2 | 100.0 (-11.3) | 0.0 (-inf) | 0.0 (-inf) |
| 75-23-5.Q0.5.I3 | 0.0 (-inf) | 94.0 (-13.8) | 94.0 (-13.8) |
| 75-26-5.Q0.5.I4 | 80.0 (-23.9) | 99.3 (-18.3) | 99.4 (-18.3) |
| 90-22-5.Q0.5.I4 | 96.6 (-6.0) | 100.0 (-5.6) | 100.0 (-5.6) |
| 90-24-5.Q0.5.I2 | 0.0 (nan) | 100.0 (-5.6) | 100.0 (-5.6) |
| 90-25-5.Q0.5.I2 | 92.6 (-9.2) | 0.0 (nan) | 0.0 (nan) |
| 90-26-5.Q0.5.I1 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| 90-30-5.Q0.5.I1 | 0.0 (nan) | 95.6 (-11.6) | 96.5 (-11.5) |
| 90-34-5.Q0.5.I2 | 0.0 (-inf) | 0.0 (nan) | 0.0 (nan) |
| 90-38-5.Q0.5.I4 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| 90-42-5.Q0.5.I4 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| 90-46-5.Q0.5.I4 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| 90-50-5.Q0.5.I3 | 0.0 (nan) | 76.9 (-36.9) | 76.9 (-36.9) |
| Grids_20 | 0.0 (4551.8) | 0.0 (4551.2) | 0.0 (4518.7) |
| Grids_21 | 1.2 (8029.6) | 4.5 (8045.3) | 0.0 (8002.4) |
| Grids_22 | 0.0 (2673.2) | 0.0 (2686.7) | 0.0 (2687.2) |
| Grids_23 | 0.0 (2652.1) | 0.0 (2654.3) | 94.5 (2787.3) |
| Grids_24 | 0.0 (7772.1) | 0.0 (7746.2) | 92.0 (8204.5) |
| Grids_25 | 0.0 (1148.2) | 100.0 (1209.9) | 100.0 (1209.9) |
| Grids_26 | 11.5 (1279.2) | 92.4 (1322.2) | 92.4 (1322.2) |
| Grids_27 | 0.0 (5097.4) | 99.1 (5507.0) | 99.1 (5507.0) |
| Grids_28 | 0.0 (1906.2) | 66.7 (1969.1) | 66.7 (1969.1) |
| Grids_29 | 0.0 (651.2) | 81.6 (669.2) | 81.6 (669.2) |
| ImageAlignment_11 | 100.0 (-824.2) | 100.0 (-824.2) | 100.0 (-824.2) |
| ImageAlignment_12 | 100.0 (-436.7) | 100.0 (-436.7) | 100.0 (-436.7) |
| ImageAlignment_13 | 100.0 (-2998.9) | 100.0 (-2998.9) | 100.0 (-2998.9) |
| ImageAlignment_14 | 100.0 (-1557.5) | 100.0 (-1557.5) | 100.0 (-1557.5) |
| ImageAlignment_15 | 100.0 (-1177.5) | 100.0 (-1177.5) | 100.0 (-1177.5) |
| ObjectDetection_13 | 0.0 (3426.7) | 0.0 (6684.3) | 0.0 (6684.3) |
| ObjectDetection_14 | 0.0 (4931.7) | 0.0 (6712.5) | 0.0 (6712.5) |
| ObjectDetection_15 | 0.0 (6985.6) | 13.1 (10627.8) | 13.1 (10627.8) |
| ObjectDetection_16 | 0.0 (6928.8) | 14.3 (12023.3) | 14.3 (12023.3) |
| ObjectDetection_17 | 0.0 (-837.6) | 8.6 (2454.9) | 8.6 (2454.9) |
| ProteinFolding_11 | 100.0 (1962.3) | 100.0 (1962.3) | 100.0 (1962.3) |
| ProteinFolding_12 | 100.0 (-1547.0) | 100.0 (-1547.0) | 100.0 (-1547.0) |
| ProteinFolding_13 | 100.0 (-143.3) | 100.0 (-143.3) | 100.0 (-143.3) |
| ProteinFolding_14 | 100.0 (-331.7) | 100.0 (-331.7) | 100.0 (-331.7) |
| ProteinFolding_15 | 100.0 (-51.6) | 100.0 (-51.6) | 100.0 (-51.6) |
| Segmentation_11 | 75.8 (-134.1) | 100.0 (-132.1) | 100.0 (-132.1) |
| Segmentation_12 | 100.0 (-21.9) | 100.0 (-21.9) | 100.0 (-21.9) |
| Segmentation_13 | 100.0 (-21.4) | 100.0 (-21.4) | 100.0 (-21.4) |
| Segmentation_14 | 100.0 (-39.3) | 100.0 (-39.3) | 100.0 (-39.3) |
| Segmentation_15 | 96.3 (-164.5) | 71.3 (-169.6) | 71.3 (-169.6) |
| Segmentation_16 | 100.0 (-40.4) | 100.0 (-40.4) | 100.0 (-40.4) |
| Segmentation_17 | 93.0 (-175.4) | 88.0 (-176.2) | 88.0 (-176.2) |
| Segmentation_18 | 100.0 (-34.0) | 100.0 (-34.0) | 100.0 (-34.0) |
| Segmentation_19 | 100.0 (-24.1) | 100.0 (-24.1) | 100.0 (-24.1) |
| Segmentation_20 | 99.8 (-112.0) | 100.0 (-112.0) | 100.0 (-112.0) |
| bw_p24_16 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p24_20 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p34_15 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p34_20 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p44_15 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p44_19 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p54_10 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| bw_p54_16 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| comm_p01_16 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| comm_p01_20 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_11.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_16.fg.Q0.5.I3 | 0.0 (nan) | 44.1 (-51.2) | 38.8 (-53.8) |
| or_chain_22.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_24.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_25.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_32.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_36.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_39.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_40.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_41.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_43.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_6.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_60.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_63.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| or_chain_8.fg.Q0.5.I3 | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| pedigree1.Q0.5.I3 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree13.Q0.5.I1 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree18.Q0.5.I1 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree19.Q0.5.I4 | 86.7 (-95.9) | 87.4 (-95.4) | 87.5 (-95.3) |
| pedigree20.Q0.5.I2 | 88.7 (-51.0) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree25.Q0.5.I2 | 70.4 (-159.8) | 81.1 (-154.6) | 81.1 (-154.6) |
| pedigree30.Q0.5.I2 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree31.Q0.5.I2 | 88.7 (-124.5) | 96.1 (-118.5) | 96.3 (-118.4) |
| pedigree33.Q0.5.I2 | 80.4 (-79.4) | 100.0 (-67.8) | 97.3 (-69.4) |
| pedigree38.Q0.5.I2 | 62.8 (-95.0) | 70.3 (-91.4) | 70.3 (-91.4) |
| pedigree41.Q0.5.I2 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree44.Q0.5.I4 | 85.6 (-94.5) | 77.9 (-98.2) | 77.9 (-98.2) |
| pedigree50.Q0.5.I1 | 95.7 (-53.3) | 88.2 (-55.7) | 88.2 (-55.7) |
| pedigree7.Q0.5.I2 | 0.0 (-inf) | 0.0 (-inf) | 0.0 (-inf) |
| pedigree9.Q0.5.I3 | 0.0 (nan) | 0.0 (-inf) | 0.0 (-inf) |
| pomdp10-12_7_3_8_4.mmap | 0.0 (nan) | 0.0 (nan) | 100.0 (1.2) |
| pomdp6-12_6_2_6_3.mmap | 95.2 (1.0) | 95.2 (1.0) | 86.8 (1.0) |
| pomdp7-20_10_2_10_3.mmap | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| pomdp8-14_9_3_12_4.mmap | 0.0 (nan) | 0.0 (nan) | 0.3 (1.4) |
| pomdp9-14_8_3_10_4.mmap | 0.0 (nan) | 0.0 (nan) | 0.0 (nan) |
| wcsp_14 | 78.5 (-26.1) | 76.0 (-29.2) | 76.0 (-29.2) |
| wcsp_15 | 0.0 (-185.1) | 19.2 (-163.6) | 19.2 (-163.6) |
| wcsp_16 | 52.8 (-21.7) | 70.6 (-4.8) | 70.0 (-5.3) |
| wcsp_17 | 0.0 (-744.5) | 26.5 (-103.7) | 27.5 (-101.8) |
| wcsp_18 | 0.0 (-698.7) | 0.0 (-508.3) | 0.0 (-540.0) |