Final Leader Board
The results below are organized as follows:
- each table displays the cumulative results for the given task under different time limits
- table values are average normalized scores across all evaluated problems as outlined in Evaluation Criteria
- clicking on a solver name links to the results of the solver on the individual problem instances
PR
| Solver | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| uai14-pr | 61.7 | 96.8 | 96.7 |
| ibia-pr | 53.6 | 96.6 | 97.1 |
| AbstractionSampling | 78.9 | 91.7 | 93.9 |
| lbp-pr | 90.3 | 89.9 | 90.2 |
MAR
| Solver | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| uai14-mar | 58.4 | 80.5 | 81.9 |
| ibia-mar | 46.5 | 78.2 | 78.6 |
| lbp-mar | 58.6 | 58.4 | 58.4 |
MPE
| Solver | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| toulbar2-vacint-mpe | 85.1 | 95.5 | 97.3 |
| toulbar2-vns-mpe | 86.0 | 94.7 | 94.9 |
| toulbar2-ipr-mpe | 68.2 | 94.8 | 95.0 |
| daoopt-mpe | 80.7 | 89.9 | 90.6 |
| daoopt-lh-mpe | 77.8 | 88.5 | 90.0 |
MMAP
| Solver | 20sec | 1200sec | 3600sec |
|---|---|---|---|
| toulbar2-ipr-mmap | 67.7 | 72.7 | 73.3 |
| toulbar2-vns-mmap | 70.4 | 72.1 | 72.2 |
| daoopt-mmap | 68.5 | 72.1 | 71.7 |
| toulbar2-vacint-mmap | 68.9 | 71.3 | 71.4 |
| daoopt-lh-mmap | 64.5 | 66.4 | 67.6 |
| braobb-mmap | 34.0 | 41.4 | 44.1 |
| uai14-mmap | 28.9 | 33.8 | 35.4 |
| lbp-mmap | 28.3 | 29.4 | 29.6 |